Skip to main content
NCBI home page
The BMJ logoLink to The BMJ
. 2001 Jul 14;323(7304):110.

Peer led programme for asthma education in adolescents

Issues of design and analysis are crucial in cluster randomised trials

Tim J Peters 1,2,3, Anna Graham 1,2,3, Chris Salisbury 1,2,3, Laurence Moore 1,2,3
PMCID: PMC1120722  PMID: 11484679

Editor—We have concerns about the design and analysis of Shah et al's cluster randomised trial of a peer led education programme for asthma.1 Neither the printed nor the (longer) electronic version mentioned how clustering was accounted for in the trial design. The sample size was not justified—neither the number of clusters (six) nor numbers of children in them. This may seem unimportant since confidence intervals were provided for the comparisons between arms, but the omission is crucial.

The authors did not specify the magnitude of differences considered in advance as clinically important. The small intracluster correlations observed could just be fortuitous. With so few clusters, any estimate of between-cluster variance (and hence intracluster correlation) will be extremely imprecise. Without proper details of trial design, the danger of publication bias remains, where a study with low power is more likely to be published when significance is attained. The widths of the comparative confidence intervals are not reassuring here.

It is unlikely that a fully considered trial design would include only six clusters, with apparently no attention being paid to stratification, given the sex and year imbalances.2 Important information was also omitted regarding the analysis plan, with between-arm comparisons in table 2 for quality of life as a total score and three subdomains. Only the electronic version states that the total score was the primary outcome.

More fundamentally, comparisons for each of the four outcomes were presented first for all children and then for four subgroups (males and females in years 7 and 10). Separate tests for subgroups rather than formal interaction tests are highly prone to false positive results. There was no indication that these subgroup analyses were established in advance, nor specifically whether others were conducted. The analysis section of the electronic version refers to sex and year as potential confounders, not effect modifiers.

If the principle of “electronic long, paper short” publishing is to be successful, it is important that short versions do not omit crucial methodological information. It cannot be assumed that all readers have access to electronic versions, and Plummer has commented on the paper (in his rapid response) without apparently checking the long version.3

We appreciate that the exclusion of cluster adjusted results was justified here on grounds of space,4 but we contend that issues of design and analysis are vital in cluster randomised trials and should have been mentioned in the short version.

References

  • 1.Shah S, Peat JK, Mazurski EJ, Wang H, Sindhusake D, Bruce C, et al. Effect of peer led programme for asthma education in adolescents: cluster randomised controlled trial. BMJ. 2001;322:583–585. doi: 10.1136/bmj.322.7286.583. . (10 March.) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Ukoumunne OC, Gulliford MC, Chinn S, Sterne JAC, Burney PGJ. Methods for evaluating area-wide and organisation-based interventions in health and health care: a systematic review. Health Technol Assess 1999;3(5). [PubMed]
  • 3.Plummer W. Analysis should reflect the clustered study design [electronic response to Shah S et al. Effect of peer led programme for asthma education in adolescents: cluster randomised controlled trial]. www.bmj.com/cgi/eletters/322/7286/583#EL1 (accessed 12 June 2001). [DOI] [PMC free article] [PubMed]
  • 4.Wang H, Peat J, Shah S. Analysis reflects clustered study design [electronic response to Shah S et al. Effect of peer led programme for asthma education in adolescents: cluster randomised controlled trial]. www.bmj.com/cgi/eletters/322/7286/583#EL2 (accessed 12 June 2001). [DOI] [PMC free article] [PubMed]
BMJ. 2001 Jul 14;323(7304):110.

Papers describing cluster randomised trials must be peer reviewed by statisticians

Martin Underwood 1, Sandra Eldridge 1

Editor—We believe that Shah et al's paper on the effect of a peer led programme for asthma education in adolescents has serious weaknesses.1-1 The authors seem not to have taken account of articles highlighting issues surrounding the design and analysis of cluster randomised trials.1-2–1-4 There is no evidence from the paper of a sample size calculation, let alone one that allows for clustering.

A cluster randomised study generally lacks power compared with a non-clustered study of the same size, due to homogeneity of individuals within clusters. Not accounting for this in the analysis could alter the authors' conclusions. The variance of an estimate from a cluster randomised trial is increased relative to the variance in a non-clustered design by a factor of 1+(<n>-1)ρ, where <n> is the average cluster size and ρ, the intracluster correlation coefficient, is a measure of the between-cluster variation in the outcome being considered.

In this study, average cluster size is about 40. We cannot estimate ρ accurately, but even a modest (in these circumstances) ρ of 0.01 would lead to a 40% increase in variance and hence an increase in standard error of about 20%. This could alter the conclusions concerning the quality of life measure.

We note from the information on prepublication history that although there was a statistician on the hanging committee, this paper was not reviewed by a statistician. We recommend that to avoid weaknesses in future publications of this sort, all papers describing cluster randomised trials are in future peer reviewed by a statistician, preferably one with experience in this field.

References

  • 1-1.Shah S, Peat JK, Mazurski EJ, Wang H, Sindhusake D, Bruce C, et al. Effect of peer led programme for asthma education in adolescents: cluster randomised controlled trial. BMJ. 2001;322:583–585. doi: 10.1136/bmj.322.7286.583. . (10 March.) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 1-2.Kerry SM, Bland JM. Statistics notes: The intracluster correlation coefficient in cluster randomisation. BMJ. 1998;316:1455. doi: 10.1136/bmj.316.7142.1455. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 1-3.Kerry SM, Bland JM. Statistics notes: Sample size in cluster randomisation. BMJ. 1998;316:549. doi: 10.1136/bmj.316.7130.549. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 1-4.Kerry SM, Bland JM. Statistics notes: Analysis of a trial randomised in clusters. BMJ. 1998;316:54. doi: 10.1136/bmj.316.7124.54. [DOI] [PMC free article] [PubMed] [Google Scholar]
BMJ. 2001 Jul 14;323(7304):110.

Authors' reply

Peter G Gibson 1,2,3,4,5, Smita Shah 1,2,3,4,5, Doungkarnol Sindhusake 1,2,3,4,5, Han Wang 1,2,3,4,5, Jennifer K Peat 1,2,3,4,5, Richard L Henry 1,2,3,4,5

Editor—In his rapid response to our paper on bmj.com Plummer emphasises the pitfalls that can arise in the statistical analysis of cluster randomisation studies.2-1 In fact, we did calculate intraclass correlations to explore any effects due to clustering, although space precluded us from including both these data and more extensive within-school baseline data. For all outcomes, the values were exceedingly small—for example, those for the mean differences in quality of life scores were <0.002 for all domains. Clearly, values such as this have a negligible effect on P values.

Randomisation by clusters was necessary to avoid contamination of the control group and to make the intervention feasible. This is a strength of the design that adds to the generalisability of the results. It was considerably easier to educate a whole class about asthma than to identify students with asthma and offer separate classes for them. This strategy also provides the opportunity for asthma education to be incorporated into the school curriculum.2-2,2-3

We conducted a trial of an important intervention (asthma education) in a problematic group (adolescents in a rural high school) and measured outcomes that are acknowledged to be relevant to people with asthma (quality of life, asthma attacks, and school absenteeism). We found that a peer led approach to asthma education led to beneficial results.

Although Peters (letter here in the paper journal) and Plummer2-1 find important omissions from the analysis plan in the written version of the paper, these are reported in the electronic version. The sample size calculation and justification are reported in our study protocol, which was the basis for the peer reviewed funding application. Journals seldom publish these. We based these calculations on an earlier study, in which we measured quality of life before and after the intervention in a smaller number of adolescents.2-2 A priori we chose the subgroups of male/female and years 7 and 10 because of our prior results, and because of the design of the intervention (year 10 students are active participants in the education process, whereas year 7 students are passive recipients).2-3,2-4

Finally, there are clear advantages to electronic publication. As experience with the principle of “electronic long, paper short” publishing grows, we should see increasingly detailed electronic publications that allow full description of the research process and additional results. This system works well with the electronic versions of systematic reviews in the Cochrane library.

From this correspondence we have learnt that, given the opportunity of electronic publication, we should add rather than subtract detail and ask the editors to show us the printed version before publication. The downside is that, with no restriction on space, all sorts of critical letters can be published on bmj.com, which may confuse readers and detract from the valid message of the research (that peer led education is a valid and potentially successful approach in health promotion).

Footnotes

A longer version of this letter is available on bmj.com

References

  • 2-1.Plummer W. Analysis should reflect the clustered study design [electronic response to Shah S et al. Effect of peer led programme for asthma education in adolescents: cluster randomised controlled trial]. www.bmj.com/cgi/eletters/322/7286/583#EL1 (accessed 12 June 2001). [DOI] [PMC free article] [PubMed]
  • 2-2.Henry RL, Gibson PG, Hazell J, Vimpani G, Leggatt R, Mowbray C, et al. Integrated health and education input in the development of an educational package about asthma for schools. J Paed Child Health. 1994;30:492–496. doi: 10.1111/j.1440-1754.1994.tb00719.x. [DOI] [PubMed] [Google Scholar]
  • 2-3.Shah S, Mamoon H, Gibson PG. Peer-led health education for adolescents: 1. Development and implementation. Health Prom J Aust. 1998;8:177–182. doi: 10.1016/S1054-139X(97)00203-6. [DOI] [PubMed] [Google Scholar]
  • 2-4.Gibson PG, Shah S, Mamoon HA. Peer-led asthma education for adolescents. J Adolesc Health. 1998;22:66–72. doi: 10.1016/S1054-139X(97)00203-6. [DOI] [PubMed] [Google Scholar]

Articles from BMJ : British Medical Journal are provided here courtesy of BMJ Publishing Group

RESOURCES