Relative or Absolute? A Significant Intervention for Chlamydia Screening with Small Absolute Benefit

“This complex intervention within the English chlamydia screening programme led to a 76% increase in chlamydia screening test rates across all practices offered the intervention, with a 40% increase in infections detected.”[1] Thus begins the Discussion section of the article by McNulty, et al. in this edition of Sexually Transmitted Infections. On the surface, the finding appears dramatic, offering hope for a major advance in the coverage of screening for chlamydial infection under the United Kingdom’s National Chlamydia Screening Programme. Look a little deeper, however, and chlamydia screening clearly remains a major challenge.

McNulty and colleagues conducted a cluster randomized controlled trial using a “Modified Zelen” design to assess the impact of an intervention developed to alter the screening behaviors of general practices in the southwest of England.[1] Over 150 practices were randomized: 77 to receive the intervention and 83 to serve as control practices. The intervention, developed using the Theory of Planned Behavior,[2] incorporated an outreach educational workshop, posters, invitation cards, feedback on practice performance, and ongoing support to the practices. Grounding the intervention in the Theory of Planned Behavior provided a strong framework and justification for each component of the intervention and is a fundamental strength of this study.

The uptake of the intervention was modest: only 62% of intervention practices agreed to three contacts with a chlamydia support worker and 17% refused all contacts. Additionally, only 45% of practices used computer prompts and about two-thirds used invitation cards. Use of posters was more common, with about 80% of practices using posters.

Given the modest uptake of the intervention by the intervention practices, what was its impact? During the intervention period, the incidence rate ratio for chlamydia testing was 1.76 (i.e. a 76% relative increase) comparing the intervention practices to the control practices. During the 9 months after the intervention, the incidence rate ratio for chlamydia testing was 1.57. These increases are statistically significant, but are they clinically significant? We would argue only weakly so. The rate of testing in both groups was remarkably low, and the absolute difference between the intervention and control practices was very small. The rate of testing was 4.34 per 100 patients 15–24 years of age in the intervention practices, compared to 3.00 per 100 patients 15–24 years of age in the control practices, an absolute difference of less than 1.5 per 100 patients.

Ultimately, the effectiveness of a chlamydia screening program is judged not by the number of persons tested, but by the number of infections identified. In both the intervention and control practices, the number of infections identified was low, with only about two infections per practice detected during the intervention period. In the intervention group, 164 episodes of chlamydia were detected, corresponding to a rate of 2.5 infections per 1000 patients 15–24 years of age. The corresponding findings in the control group were 182 episodes of chlamydia with a rate of 2.3 per 1000 patients 15–24 years of age.

Furthermore, we would expect that the rate of infection would correspond roughly to the population prevalence. But, the observed rate is about 10–20 times lower than would be expected based on population-based or clinic-based studies.[3–7] For example, in the recently released results of Natsal 3, the prevalence of chlamydial infection in the UK was 3.1% in women and 2.3% in men aged 16–24 years.[7]

Overall, McNulty et al. view their results much more positively than we do, despite acknowledgement of a small absolute intervention effect. Their view appears to be based on the relative impact of the intervention and the statistical significance of the effect. But despite the affection of many clinical researchers, epidemiologists, and biostatisticians for relative measures, such as the incidence rate ratio, the absolute effect is often substantially more important from a clinical or public health perspective. In this particular case, baseline rates of screening were so low that an intervention would need a many fold higher effect to achieve a meaningful impact at the community level.

Taken together, we have an intervention that yielded a modest relative increase in screening, a small absolute increase in screening, and a minimal increase in the number of infected persons identified. These effects varied somewhat with uptake of the intervention: practices that adopted more components of the intervention tended to have higher screening rates. Even considering this variation, however, the maximum absolute increase was only 29% from baseline.

Given the small effects, what are the primary take home messages from the study? Simply put, provider behavior is exceedingly difficult to modify. Screening rates were exceptionally low before the study began, the intervention had a small effect, and the rates, although remaining “significantly higher” in the intervention practices compared to the control practices, dropped after the intervention period. The intervention used here – straightforward, practical, and theory-based – would seemingly be ideal for boosting chlamydia screening. But in many ways, the results are quite disappointing. The small effect and the limited number of infections identified suggest that chlamydia screening is not a priority for these practices – even with a carefully developed and well-planned intervention. This observation implies that changing chlamydia screening behavior to achieve a reasonable level of coverage will require a substantially more intensive and sustained intervention, or perhaps, an alternative economically-based motivation.

Before concluding, we wish to bring attention to one unique feature of the study design – the “modified Zelen” design, an approach that is unlikely to be familiar to many researchers. The traditional Zelen design, also known as the randomized consent design, randomizes prospective participants in a randomized controlled trial before consent is obtained and then seeks consent only from those assigned to the intervention group.[8,9] With this design, persons in the standard of care arm are not aware of their participation in a trial. One modification of the Zelen design allows consent for participation in an observational study first, followed by a second consent for the intervention for participants assigned to that group.[8,9]

McNulty, et al. used a further modification of the Zelen design: no informed consent was obtained from the practices involved in either the intervention or standard of care groups.[1] Thus, the clinicians were unaware that they were engaged in a research study. The practitioners could, however, accept or refuse any component of the intervention. The design received appropriate ethical review and approval.

While a full review of the ethical issues of this modification of Zelen’s design is beyond the scope of this commentary, we believe two points are worth consideration. First, a key question in any research study that involves any form of deception is whether the deception was necessary.[10,11] The rationale for the design in this case was that the behavior of the providers might change with knowledge that their clinical behaviors were under study. Was that deception necessary? Possibly – whether screening rates would have been substantially higher in the intervention or control groups if the study was revealed is unclear. Second, one consequence of informed consent may be to build and maintain trust between the participant and the researcher.[12] In this case, the research team included both academics and government employees. Loss of providers’ trust because of concerns related to being studied or “manipulated” is one plausible outcome of this study. Only those providers who engaged in the study, and others who subsequently learn of it, can answer that question.

In conclusion, this carefully designed intervention resulted in modest increases in chlamydia screening among general practices in southwest England. The emphasis on relative effects tends to overstate the small gains in screening and identified infections. The small absolute effects reflect the difficulty in changing clinicians’ behavior. Undoubtedly, greater coverage of young persons at risk for chlamydial infection will require creative solutions.