Methodological issues in pragmatic trials of complex interventions in primary care

A distinction in trial design is made between explanatory (or efficacy) and pragmatic (or effectiveness) trials.¹ These terms, originally coined by Schwartz and Lellouch in 1967, are used to describe trials that either test causal research hypotheses to determine whether an intervention works in tightly controlled conditions to achieve optimal efficacy and/or to understand the key mechanism of action (explanatory); or that help choose between options for care under the usual conditions in which those options may be offered (pragmatic).

The pragmatic–explanatory distinction really comprises a continuous spectrum¹ with many elements, from the breadth of eligibility criteria, the flexibility in intervention delivery, expertise of those delivering treatment, degree of standardisation of intervention protocol, the efforts to ensure compliance, through to specific approaches to analysis. Pragmatic trials are increasingly adopted to test the effectiveness of complex interventions for patients in primary care, but may have explanatory elements.

COMPLEX INTERVENTIONS

The Medical Research Council Framework for the development and evaluation of complex interventions provides helpful guidance, as complex interventions involve a number of separate but interacting components which are likely to be important to the success of the intervention, although the ‘active ingredients’ are often difficult to specify.^2,3 In developing a novel, complex intervention comprising separate elements — for example, an education package with exercises for back pain — a rigorous process of development is needed to inform both the intervention and trial design.

A ‘good idea’ can easily lead to a superficial, ineffective intervention that is poorly evidence-based, with a poor theoretical basis and insufficient empirical development work. Ensuring components work well together is vital, as it is easy for components either to dilute the effect of each other or work against each other.⁴ ‘Simple’ interventions, such as Alexander Technique, cognitive–behavioural therapy, and manual therapy, may also in practice be complex as they encompass several elements which require the components of the intervention to be informed by knowledge and detailed assessment of the patient.

EVALUATION OF COMPLEX INTERVENTIONS

Evaluation of complex interventions is a particular challenge,^2,3 and a key question is to what extent trialists attempt to evaluate how a complex intervention works, that is, to ‘tease out’ the effective ingredients in the ‘black box’ For example, exercise for a variety of conditions may potentially work by some or all of improved muscle strength, balance and stability, decreased anxiety, improved self-confidence and control over symptoms, increased social interaction and participation, time, support, health professionals’ empathy, reassurance and attention, and so on. Trial teams need to decide whether to measure these factors to be able to model an explanatory element, bearing in mind that the greater the burden on measurement the more likely selection and attrition bias will operate.

Bias — that is, the systematic distortion of the estimates due to poor design, conduct, or analysis of a trial⁵ — is particularly an issue for pragmatic trials of complex interventions where ‘real world’ estimates are paramount. Maximising generalisability and access to the key target group is important by careful attention to how the trial is communicated to potentially eligible patients, and recruitment methods that avoid reliance on busy clinicians (for example, using Read-Code activated electronic tags, or mailed invitation or population screening surveys of registered patients). Where the unit of randomisation is not the patient (cluster randomised trials) using such methods will also lessen selection bias, as clinicians in the control group may be less enthusiastic in recruiting patients, leading to differential selection bias.

CONTROLLING BIAS

The choice of control intervention is a difficult issue in trials of complex interventions. Trials with waiting list controls, usual care, or ongoing ‘stable’ medication provide intervention estimates which reflect the combined specific and non-specific effects that will accrue in practice, and are more likely to show between-group differences,⁶ but are less likely to motivate eligible patients to participate, and make attrition bias more likely. Conversely, using a ‘credible’ control or comparison intervention can help control for the known non-specific effects of complex interventions, ensure that the trial results are not simply explained by regression to the mean, natural history of the complaint or attention from a health professional, but provide an underestimate of the total effect of the intervention. More than one control group can be useful to tease out some of these issues,⁷ but equally it can be very difficult to separate fully the specific and non-specific effects of an intervention.⁸

Performance bias is a particular issue since an integral component of complex interventions is frequently patient–practitioner interaction or the ‘therapeutic relationship’. Complex interventions are rarely in standardised formats like a simple pill, rather most are individualised depending on presenting features, history, and response to the intervention. Factors such as the initial starting ‘dose’, the intensity of treatment progression, the frequency of sessions and contact time, degree of adherence required of the patient, and delivery as monotherapy or in parallel with other interventions may vary.

Investigators need to decide whether to try to constrain patient–practitioner interaction, attention effects and patient expectations. Explanatory trials, or those that aim to find out what components of the intervention ‘work’ are more likely to try to constrain and/or measure the interaction, and to include placebo or ‘attention only’ comparisons that control for the attention and interaction with the health professional. Pragmatic trials, however, are more likely to test the complex intervention as it would normally be delivered in clinical practice (for example, Paterson et al⁹).

Blinding, or the masking of patients, practitioners, outcome assessors, and statisticians about the treatment to which an individual patient has been allocated is the traditional approach to try to prevent performance and ascertainment bias in trials.^5,10 However, complex interventions, by their nature, often mean that the usual types of blinding (patient and practitioner) are simply impossible. In fully pragmatic trials, in which the specific ‘active’ ingredient of the intervention is not of interest, placebo interventions are rarely included, yet some blinding is often still possible¹ particularly of outcome assessment and during analysis.

The traditional explanatory, placebo controlled, double blinded randomised trial has been the legacy of drug evaluations. While pragmatic randomised trials are an increasingly popular design to test primary care interventions, the ‘shackles of simple intervention thinking prove hard to throw off’.¹¹ In designing pragmatic trials of complex interventions in primary care, there are few ‘right’ answers. The key is to be clear at the outset of the trial, and conduct and report the trial robustly¹² to facilitate appropriate interpretation. Clear thinking about the key question and careful decision-making will maximise the chance of the results influencing clinical practice for the better.

Provenance

Freely submitted; not externally peer reviewed.