Adherence-adjusted estimates of benefits and harms from treatment with amoxicillin for LRTI: secondary analysis of a 12-country randomised placebo-controlled trial using randomisation-based efficacy estimators

Abstract

Objectives

Estimate the efficacy of amoxicillin for acute uncomplicated lower-respiratory-tract infection (LRTI) in primary care and demonstrate the use of randomisation-based efficacy estimators.

Design

Secondary analysis of a two-arm individually-randomised placebo-controlled trial.

Setting

Primary care practices in 12 European countries.

Participants

Patients aged 18 or older consulting with an acute LRTI in whom pneumonia was not suspected by the clinician.

Interventions

Amoxicillin (two 500 mg tablets three times a day for 7 days) or matched placebo.

Main outcome measures

Clinician-rated symptom severity between days 2–4; new/worsening symptoms and presence of side effects at 4-weeks. Adherence was captured using self-report and tablet counts.

Results

2061 participants were randomised to the amoxicillin or placebo group. On average, 88% of the prescribed amoxicillin was taken. The original analysis demonstrated small increases in both benefits and harms from amoxicillin. Minor improvements in the benefits of amoxicillin were observed when an adjustments for adherence were made (mean difference in symptom severity −0.08, 95% CI −0.17 to 0.01, OR for new/worsening symptoms 0.81, 95% CI 0.66 to 0.98) as well as minor increases in harms (OR for side effects 1.32, 95% CI 1.12 to 1.57).

Conclusions

Adherence to amoxicillin was high, and the findings from the original analysis were robust to non-adherence. Participants consulting to primary care with an acute uncomplicated LRTI can on average expect minor improvements in outcome from taking amoxicillin. However, they are also at an increased risk of experiencing side effects.

Trial registration numbers

Eudract-CT 2007-001586-15 and ISRCTN52261229.

The trial was registered at EudraCT in 2007 due to an administrative misunderstanding that EudraCT was a suitable registry—which it was not in 2007, but has become since. On discovery of this error, the trial was also registered at ISRCTN (January 2009). Trial procedures did not change between the two registrations.

Strengths and limitations of this study.

• This is the largest randomised placebo-controlled trial evaluating amoxicillin for acute, uncomplicated lower-respiratory-tract infection in primary care to date.

• Consideration of the benefits and harms of amoxicillin allowed for a balanced assessment of this treatment.

• Multiple types of adherence measures meant that agreement between measures could be assessed.

• As is often the case in research, indirect measures of medication adherence were collected. These rely heavily on their inherent assumptions (eg, accurate patient recall, returning of all unused medication). Direct measures (eg, direct observation) are preferable, but often not feasible in practice.

• Structural mean models enabled an adjustment for treatment non-adherence while maintaining a comparison of groups as randomised.

Introduction

Acute uncomplicated lower-respiratory-tract infection (LRTI) is one of the most common reasons for patients consulting in primary care.^{1 2} Antibiotics are prescribed to the majority of consulting patients, with amoxicillin being the most common across Europe.³ Evidence for the benefits and harms of antibiotic treatment has been unclear, primarily due to underpowered and inappropriately designed studies.⁴ With antibiotic resistance becoming a growing problem worldwide, the need for clear evidence for the benefits and harms of antibiotics for this condition has never been more of a priority.^{5 6}

A recently published trial of amoxicillin for acute uncomplicated LRTI in primary care concluded that amoxicillin provides little clinical benefit and causes slight harms.⁷ The findings of this trial were based on a comparison of participants in the arm to which they were originally randomised (ie, using the intention to treat (ITT) principle). While an ITT analysis is an important part of the analysis of any trial, as it reflects the design of the trial and uses randomisation to avoid selection bias,⁸ this approach does not take into account deviations that occur following randomisation, such as lack of adherence to treatment.

Adherence to antibiotic treatment in primary care is poor.⁹ Less than 60% of patients prescribed an antibiotic for an acute cough/LRTI in primary care initiated their treatment, and less than half took the full course.¹⁰ Poor levels of adherence to antibiotics wastes healthcare resources, could negatively impact on clinical outcomes and could increase the selective pressures for antibiotic resistance. When issues with adherence are present in a trial, analysis based on the ITT principle underestimates treatment effects, and can only provide an unbiased estimate of the effect of prescribing treatment (effectiveness), rather than the effect of treatment itself (efficacy).¹¹

Two traditional approaches to estimating treatment efficacy include per-protocol analysis, where participants who do not adhere to their allocated treatment are excluded from analyses, and on-treatment analysis, where participants are analysed in the group corresponding to the treatment they took (regardless of the group they were allocated to).¹² Both methods make the implicit assumption that the groups of participants are equivalent with respect to observed and unobserved variables, something that is implausible in practice.¹³ Approaches to estimating efficacy without making this key assumption exist, and are becoming increasingly popular.^{14 15} However, these approaches are generally reported in specialist methodological journals, rather than the general medical literature and as such there still remains a reliance on more traditional and arguably inadequate methods.

The aim of this paper is to use the data set from the largest placebo-controlled trial of amoxicillin for acute uncomplicated LRTI in primary care to produce adherence-adjusted estimates of the benefits and harms from amoxicillin for adults consulting in primary care with an acute uncomplicated LRTI, while preserving a comparison of groups as randomised.

Methods

Study design and participants

A two-arm individually-randomised placebo-controlled trial was conducted between November 2007 and April 2010. Patients were recruited consecutively from primary care practices from 12 European countries (Belgium, England, France, Germany, Italy, the Netherlands, Poland, Slovakia, Slovenia, Spain, Sweden and Wales).

The trial has previously been described in detail elsewhere.⁷ A brief description about recruitment, randomisation, blinding, the interventions, data collection and follow-up are given below. Further analyses were performed in order to investigate our study question.

Recruitment, randomisation, blinding and interventions

Participants were eligible for inclusion if they were aged 18 years or older and consulting for the first time with either an acute cough (≤28 days’ duration) as their main symptom, for which non-infective diagnoses were judged very unlikely, or an illness in which cough was not the most prominent symptom but the clinician thought acute LRTI the most probable diagnosis.

Participants were deemed ineligible if their initial diagnosis was community-acquired pneumonia (ie, complicated LRTI) on the basis of focal chest signs (focal crepitations, bronchial breathing) and systemic features (high fever, vomiting, severe diarrhoea). Participants were also ineligible if their working diagnosis was cough of a non-infective cause (eg, pulmonary embolus, left ventricular failure, oesophageal reflux, allergy), they had used antibiotics in the previous month, were unable to provide informed consent or complete the diary (eg, they had dementia, psychosis or severe depression), were pregnant, allergic to penicillin or had immunological deficiencies.

Participants were allocated to groups on a 1:1 basis using block randomisation. As this was a double-blinded trial, clinicians and participants were blinded to the randomisation sequence and allocation. All outcome data were also collected without prior knowledge of the group to which participants were allocated.

Randomised participants received a prescription for amoxicillin, to be taken as two 500 mg tablets three times a day for 7 days, or a placebo identical in appearance, taste and texture.

Data collection and participant follow-up

Consenting participants had their comorbidities, clinical signs and symptoms recorded by the recruiting clinician.

Following recruitment, consent and randomisation, participants were given a daily symptom diary to complete for up to 28 days. The diary recorded the duration and severity of 12 symptoms (cough, phlegm, shortness of breath, wheeze, blocked or runny nose, chest pain, muscle aches, headaches, disturbed sleep, general feeling of being unwell, fever and interference with normal activities). Severity was scored on a scale from 0 to 6 (0=no problem, 1=very little problem, 2=slight problem, 3=moderately bad, 4=bad, 5=very bad, 6=as bad as it could be). Patients also recorded non-respiratory symptoms, such as diarrhoea, skin rash and vomiting.

Members of the research team telephoned participants after 4 days to offer support and answer questions about the completion of the diary. If the diary was not returned after 4 weeks, brief information was collected about symptom duration and severity. This information was collected with either a short questionnaire or a standardised telephone call.

Measures of adherence

Using their daily symptom diary, participants recorded whether or not they took their study medication on a given day, and whether they took their study medication according to the instructions. Where it was indicated that participants did not take their study medication according to the instructions, space was given to provide more detail. Participants for whom a diary was not returned were asked to state the number of days that they took their study medication. This information was collected using the short questionnaire/telephone call described in the previous section. Participants were also instructed to return their study medication bottles, complete with any unused medication, at the end of the trial. The number of tablets returned was recorded by members of the research team.

Randomised participants were prescribed 42 tablets. Adherence to study medication was defined as the percentage of the correct number of tablets taken during the first 7 days of the follow-up period (ie, the period for which the medication was prescribed). Three binary definitions of adherence were also constructed in order to provide sensitivity analyses around the continuous definition. The three binary definitions were full (100%) adherence versus not full adherence, at least the equivalent of a 5-day course (approximately 71.4%) versus less and at least one tablet versus no tablets.

Where participants indicated that they had taken medication on a particular day, in the absence of information to the contrary (eg, stating that they only took one tablet three times a day instead of two tablets), the assumption was made that a participant consumed all study medication as instructed. Where medication bottles were returned, it was assumed that the difference between the number of tablets prescribed and the number returned equated to the number of tablets consumed. We also assumed that all tablets were consumed during the first 7 days of the follow-up period. Where a short questionnaire or telephone call was conducted, it was assumed that the correct numbers of tablets were taken for the number of days medication was reportedly taken.

Where multiple types of adherence measures were available for a participant the agreement between measures, and the assumptions inherent in our definition of adherence, were investigated.

Outcomes

To demonstrate the benefits and harms of amoxicillin in this population, and to illustrate the use of randomisation-based efficacy estimators, the paper concentrates on three of the outcomes described in the original paper. The first was the mean clinician-rated symptom severity between days 2 and 4 after initial presentation. The second outcome was the development of new or worsening symptoms, defined as returning to the clinician with new or worsening symptoms, new signs or an illness requiring admission to hospital within the 4-week follow-up period. The third outcome was the presence of any non-respiratory symptoms (diarrhoea, skin rash or vomiting) during the 4-week follow-up period. These specific symptoms were recorded as they are known side effects of amoxicillin. The first two outcomes were used to demonstrate the clinical benefits of amoxicillin for patients with an acute uncomplicated LRTI in primary care, with the third used to demonstrate harms. The decision to exclude the outcome “time to resolution of moderately bad symptoms” from the analysis was made for two reasons. First, in order to reduce the number of assumptions made when deriving the definition of adherence (we have not made any assumptions about adherence on individual days, but would have to make this additional assumption to perform analysis on this outcome). The second reason was that standard techniques for adjusting time-to-event outcomes for non-adherence rely on fitting an accelerated failure time model. The original outcome was analysed using a Cox proportional hazards model, and therefore the outcome would initially require reanalysing using an accelerated failure time model before an adjustment could be made. As the results from this analysis cannot be directly compared with the findings from the main paper, the decision to exclude this outcome from consideration was made.

Statistical analysis

Participants and their adherence to study medication were described using means (SDs), medians (IQRs) and percentages as appropriate.

Participants for whom more than one measure of adherence was available had their agreement between measures compared using Bland and Altman limits of agreement.¹⁶ Bland and Altman plots are presented with jittering and semitransparency to highlight overlapping data points. Where multiple types of adherence measures were reported and there was disagreement between values, the minimum value was used for analysis.

The between-group mean difference in symptom severity on days 2 to 4 postrandomisation was estimated using linear regression. The mean clinician-rated symptom severity at baseline was controlled for as a covariate. The between-group odds of developing new or worsening symptoms and of reporting any non-respiratory symptoms in the 4 weeks following randomisation were compared using logistic regression without covariates. These analyses included participants on an intention-to-treat basis. That is, they did not adjust for deviations following randomisation. The analyses therefore provide an estimate of the effectiveness of amoxicillin for patients with an acute uncomplicated LRTI in primary care, and as an estimate of efficacy, are viewed as being biased towards the null.

To determine efficacy in a way that preserves randomisation (ie, provides a comparison of groups ie, independent of observed and, importantly, unobserved confounders), and is not biased towards the null, structural mean models (SMM) were used to compare the between-group differences in the aforementioned outcomes. By recognising that at the beginning of a trial, all participants have two potential outcomes—one if they are treated and one if they are not, a SMM relates a treated participant's observed outcome to their potentially counterfactual outcome that would have been observed had they received no treatment. Standard approaches to fitting a SMM rely on using observed levels of exposure, and treating randomisation as an instrument (ie, assuming that it is independent of both observed and unobserved confounders and only effects outcome through its effect on exposure). Estimation procedures therefore rely on finding a value of the treatment effect such that balance is achieved between groups on the outcome (or potential outcome) in participants who were not treated. The between-group mean difference in symptom severity on days 2 to 4 was estimated using a two-stage least squares instrumental variables regression model.¹⁷ To compare the odds of developing new or worsening symptoms and reporting any non-respiratory symptoms, a generalised linear (double logistic) SMM was estimated via a generalised method of moments procedure.¹⁸ The double logistic SMM involved a two-step process whereby the association between outcome (development of new or worsening symptoms or reporting of side effects), trial arm and adherence was modelled first, with estimates from this model used in the SMM in order to obtain correct SEs (and hence correct 95% CIs). For more information on the use of randomisation-based efficacy estimators and their core assumptions, including the Stata syntax used to implement the SMMs, please see the online supplementary appendices 1 and 2.

Results from the linear regression model are presented as adjusted mean differences with associated 95% CIs. Results from the logistic regression models are presented as ORs with associated 95% CIs. For the SMM (double logistic SMM), results are presented as both the adjusted mean difference (OR) per % increase in adherence and per 100% adherence, the latter of which can be interpreted as the maximum possible efficacy.

Additional analyses using the three binary definitions of adherence were performed to investigate the sensitivity of the main efficacy analyses to departures from the assumed linear relationship between adherence and outcome.

Data management and descriptive statistics were performed using IBM SPSS Statistics V.20.¹⁹ All other analyses were performed using Stata V.13.²⁰

Results

Participants

In total, 2061 participants were recruited and randomised to either the amoxicillin group (1038) or placebo (1023; figure 1). The groups were well matched on baseline characteristics (table 1).

Figure 1.

CONSORT flow diagram.

Table 1.

Baseline characteristics of trial participants

Baseline characteristic	Amoxicillin	Placebo
Women	624/1038 (60.1%)	600/1023 (58.7%)
Age (years)	48.6 (16.7)	49.3 (16.4)
Non-smoker (past or present)	477/1037 (46.0%)	483/1022 (47.3%)
Illness duration before index consultation (days)	9.5 (8.0)	9.3 (7.2)
Respiratory rate (breaths per minute)	16.9 (3.3)	16.9 (3.3)
Body temperature (°C)	36.7 (3.3)	36.8 (3.3)
Lung disease*	163/1037 (15.7%)	147/1023 (14.4%)
Mean severity score (all symptoms)†	2.1 (0.5)	2.1 (0.5)
Mean severity score (cough)†	3.1 (0.7)	3.2 (0.7)
Sputum production	814/1036 (78.6%)	824/1021 (80.7%)
Discoloured sputum‡	481/968 (49.7%)	468/957 (48.9%)

Data are n/N (%) or mean (SD).

*Chronic obstructive pulmonary disease or asthma.

†Severity of symptoms: 1=no problem; 2=mild problem; 3=moderate problem; 4=severe problem.

‡Green, yellow or blood stained.

Adherence to study medication

Adherence data were available for 1854 participants (90% of all randomised participants). The majority of participants had multiple types of measure recorded (1214, or 58.9% of all randomised; figure 2).

Figure 2.

Availability of different types of adherence data for all 2061 randomised participants.

Adherence to study medication was similar between trial arms and relatively high overall. Average levels of adherence were highest for responses obtained from self-reported diaries and lowest for responses from self-reported telephone. Adherence data were highly skewed for all three measures and spanned the entire range of possible responses (table 2).

Table 2.

Levels of adherence to study medication across all types of measures

	Amoxicillin			Placebo			Overall
	Mean (SD)	Median (IQR)	Minimum–maximum	Mean (SD)	Median (IQR)	Minimum–maximum	Mean (SD)	Median (IQR)	Minimum–maximum
Self-reported diary (n=1675)	91.6 (21.8)	100.0 (100.0–100.0)	0.0–100.0	90.8 (22.3)	100.0 (100.0–100.0)	0.0–100.0	91.2 (22.0)	100.0 (100.0–100.0)	0.0–100.0
Self-reported telephone (n=129)	80.4 (36.0)	100.0 (78.6–100.0)	0.0–100.0	74.7 (37.9)	100.0 (42.9–100.0)	0.0–100.0	77.5 (36.9)	100.0 (57.1–100.0)	0.0–100.0
Tablet count (n=1266)	90.0 (23.8)	100.0 (100.0–100.0)	0.0–100.0	87.0 (26.9)	100.0 (90.5–100.0)	0.0–100.0	88.5 (25.4)	100.0 (95.2–100.0)	0.0–100.0

Agreement between adherence measures

Where multiple types of adherence measures were available, self-reports (diary and telephone formats) provided slightly higher estimates of adherence on average compared to tablet counts (mean differences of 1.7 and 2.6 percentage points, respectively). The limits of agreement when comparing diary and tablet count adherence ranged from −26.8 (self-reported diary adherence was calculated as 26.8 percentage points lower than tablet count adherence) to 30.2 (self-reported diary adherence was calculated as 30.2 percentage points higher than tablet count adherence) and when comparing telephone and tablet count from −21.8 to 26.9 (table 3). Figure 3A, B provide an illustration of the level of agreement between different types of measures. What is clear from these figures is that adherence was high and was generally good (most data points on both plots are clustered around the coordinate (100, 0), indicating full adherence and no difference between measures). For the comparison of diary to tablet count adherence, 7% of participants were outside the limits of agreement; for the comparison of telephone to tablet count adherence, 5% of participants were outside the limits of agreement.

Table 3.

Difference between adherence measures and limits of agreement

Difference between adherence measures	Self-reported diary adherence minus tablet count adherence (n=1135)	Self-reported telephone adherence minus tablet count adherence (n=80)
Mean	1.7	2.6
SD	14.5	12.4
Lower 95% limit of agreement	−26.8	−21.8
Upper 95% limit of agreement	30.2	26.9

Figure 3.

(A and B) Bland and Altman plots illustrating the agreement between the self-reported (diary (A) and telephone (B)) and tablet count adherence measures. Red solid line represents perfect agreement between measures. Black solid line represents the mean difference (bias) between measures. Black dashed lines are the 95% limits of agreement. Where data points took the same value (ie, when more than one participant had both the same average and difference in adherence), semitransparency and jittering effects were applied to provide an illustration of the number of overlapping data points. There were a large number of data points at (100, 0), and this is illustrated by the large cluster of jittered points around this coordinate.

Taking the minimum reported adherence value (where multiple values were reported), adherence to study medication remained high and negatively skewed (table 4 and figure 4).

Table 4.

Levels of adherence to study medication used for statistical analyses (with the minimum value reported when participants had more than one type of measure)

	Amoxicillin (n=930)	Placebo (n=924)	Overall (n=1854)
Mean (SD)	88.0 (25.8)	86.6 (27.2)	87.3 (26.5)
Median (IQR)	100.0 (95.2–100.0)	100.0 (85.7–100)	100.0 (90.5–100.0)
Minimum–maximum	0.0–100.0	0.0–100.0	0.0–100.0

Figure 4.

Proportion of participants at each adherence level (with the minimum value reported when participants had more than one type of measure).

Outcomes

Table 5 provides descriptive statistics for each of the three clinical outcomes.

Table 5.

Descriptive statistics of the three outcome measures

Outcome	Amoxicillin	Placebo
Mean symptom severity between days 2 and 4 postrandomisation*	1.6 (0.8)	1.7 (0.8)
Development of new or worsening symptoms in the 4 weeks postrandomisation	162/1021 (15.9)	194/1006 (19.3)
Reported non-respiratory symptoms/side effects in the 4 weeks postrandomisation	249/867 (28.7)	206/860 (24.0)

Data are n/N (%) or mean (SD).

*Each symptom was scored from 0–6 (0=no problem, 1=very little problem, 2=slight problem, 3=moderately bad, 4=bad, 5=very bad, 6=as bad as it could be).

Effectiveness

Table 6 compares the effectiveness and efficacy of amoxicillin with respect to the various outcomes below.

Table 6.

Comparison of effectiveness and efficacy of amoxicillin for acute uncomplicated LRTI in primary care

Outcome	Effectiveness*	Effectiveness for whom adherence data were also available†	Efficacy per 10% increase in adherence†	Maximum efficacy (100% adherence)†
Adjusted between-group mean difference in symptom severity between days 2 and 4 postrandomisation	−0.07 (−0.15 to 0.01)	−0.07 (−0.15 to 0.01)	−0.008 (−0.017 to 0.001)	−0.08 (−0.17 to 0.01)
OR for developing new or worsening symptoms in the 4 weeks postrandomisation	0.79 (0.63 to 0.99)	0.81 (0.64 to 1.03)	0.978 (0.960 to 0.998)	0.81 (0.66 to 0.98)
OR for reporting non-respiratory symptoms/side effects in the 4 weeks postrandomisation	1.28 (1.03 to 1.59)	1.28 (1.04 to 1.59)	1.028 (1.011 to 1.046)	1.32 (1.12 to 1.57)

*Analysis based on 1789, 2027 and 1727 participants for the symptom severity, new symptoms and side effect outcomes, respectively.

†Analysis based on 1787, 1923 and 1725 participants for the symptom severity, new symptoms and side effect outcomes, respectively.

LRTI, lower-respiratory-tract infection.

As reported in the original paper, the adjusted between-group mean difference in symptom severity score on days 2 to 4 was slightly lower in the amoxicillin group than the placebo group (adjusted mean difference of −0.07, 95% CI −0.15 to 0.01).

Being allocated to the amoxicillin arm (ie, being prescribed amoxicillin) was associated with decreased odds of developing new or worsening symptoms in the 4 weeks postrandomisation follow-up period. The odds of developing new or worsening symptoms were 21% lower for participants who were prescribed amoxicillin than for those prescribed a matched placebo (OR=0.79, 95% CI 0.63 to 0.99). When the effectiveness analyses were only performed on participants for whom outcome and adherence data were available, there was a 19% decrease in the odds of developing new or worsening symptoms in participants prescribed amoxicillin (OR=0.81, 95% CI 0.64 to 1.03).

Being prescribed amoxicillin was associated with a 28% increase in the odds of reporting non-respiratory symptoms (side effects) in the 4 weeks postrandomisation (OR=1.28, 95% CI 1.03 to 1.59).

Efficacy

Adjusting for adherence using the SMM, a small increase in the between-group mean difference in symptom severity score for participants who complete their course of amoxicillin was found (−0.08, 95% CI −0.17 to 0.01).

Figure 5 provides an illustration of the effectiveness and efficacy of amoxicillin for the above outcome. The treatment efficacy when adherence is 0% is 0, the ITT (effectiveness) is illustrated by the diamonds (positioned at an adherence level of 88%—the patient-average), and the maximum efficacy when adherence is 100%.

Figure 5.

Graphical illustration of the effectiveness and efficacy of amoxicillin on mean symptom severity on days 2–4.

The odds of developing new or worsening symptoms remained lower in participants who took their full course of amoxicillin (OR for 100% adherence to amoxicillin=0.81, 95% CI 0.66 to 0.98).

A small increase in the odds of reporting non-respiratory symptoms was found when adjusting for adherence (OR for 100% adherence=1.32, 95% CI 1.12 to 1.57).

Sensitivity analyses

Refitting the above efficacy analyses with binary definitions of adherence, the results remained largely similar and did not alter the conclusions drawn by either the efficacy or indeed the effectiveness analyses. The most extreme definition of adherence (full vs not) yielded the largest between group differences and the least extreme (at least one tablet vs none) yielded the smallest (table 7).

Table 7.

Efficacy analyses with binary definitions of adherence (for sensitivity)

Outcome	Efficacy with binary definition of adherence (full vs not full)	Efficacy with binary definition of adherence (at least five day course vs less than five day course)	Efficacy with binary definition of adherence (at least one tablet vs no tablets)
Adjusted between-group mean difference in symptom severity between days 2 and 4 postrandomisation	−0.10 (−0.20 to 0.01)	−0.08 (−0.18 to 0.01)	−0.07 (−0.15 to 0.01)
OR for developing new or worsening symptoms in the 4 weeks postrandomisation	0.78 (0.62 to 0.98)	0.80 (0.65 to 0.98)	0.82 (0.69 to 0.98)
OR for reporting non-respiratory symptoms/side effects in the 4 weeks postrandomisation	1.43 (1.15 to 1.79)	1.35 (1.26 to 1.62)	1.29 (1.11 to 1.50)

Discussion

Principal findings

In this 12-country randomised placebo-controlled trial of amoxicillin for acute uncomplicated LRTI in primary care, reported levels of adherence to study medication was very high. Prescribing amoxicillin in this setting was shown to have modest improvements in symptom severity on days 2–4, and a decrease in the odds of developing new or worsening symptoms in the 4 weeks following index consultation. However, this has to be balanced with the odds of reporting non-respiratory symptoms (side effects) in the 4 weeks following index consultation, which also increased. Adjusting these findings for adherence, the effect of taking amoxicillin in this setting largely agreed with the effect of prescribing described above. Given the high level of adherence reported in the trial, the adjustments made were minor, though in the expected direction. Compared to the effect of prescribing amoxicillin (ie, including participants who may take their medication to a varying degree), taking amoxicillin was shown to further improve symptom severity on days 2–4, further decrease the odds of developing new or worsening symptoms and further increase the odds of reporting side effects.

Strengths and weaknesses

To date, this remains the largest randomised placebo-controlled trial evaluating amoxicillin for acute, uncomplicated LRTI in primary care. By maintaining a broad inclusion criteria, recruiting across a range of different countries, and recruiting participants similar in nature to previously conducted observational studies in this setting,³ the findings of this study are likely to be widely applicable.

This paper demonstrated that the findings of main effectiveness analysis were robust to non-adherence to treatment, and did so using a method of analysis that was not prone to the usual selection biases that arise when ITT findings are adjusted for treatment adherence traditionally (eg, per-protocol analysis).

By considering the benefits and harms, the study provided a comprehensive account of the consequences of taking amoxicillin for an acute uncomplicated LRTI in primary care.

Adherence to medication was assessed using self-report and tablet count data, and while both only provided indirect measures of medication adherence, relying heavily on various assumptions (eg, accurate participant recall, returning of all unused medication), both measures were often available for the same individual, allowing for the assessment of agreement between measures. Agreement was good, with adherence calculated as 100% for both measures for the majority of participants.

The use of SMMs to adjust trial findings for non-adherence was attractive as it allowed for a comparison of groups that was independent of measured and unmeasured confounders. However, for this comparison to be valid, it relied on the key assumption that for participants who were categorised as non-adherers, merely being allocated to receive treatment had no effect on outcome (the so-called exclusion restriction).²¹ While this was likely to be a valid assumption for this study, as participants and clinicians were blinded to allocation, this is less likely to be valid for non-blinded studies.

Defining adherence as a continuous measure made the exclusion restriction more plausible, as the lowest level of adherence could be defined as receiving no treatment, a level at which being allocated to either treatment group should really have no effect on outcome. However, this approach made the additional assumption that the effect of receiving an increasing amount of treatment on outcome increased linearly,²² which for a trial involving medication is unlikely to be true. Sensitivity analyses were conducted using various binary definitions of adherence, ranging from one or more tablets (vs no tablets) to full course (vs less than full course). While the former increased the plausibility of the exclusion restriction, the estimated treatment efficacy was too conservative. The latter analysis combined participants who would have taken 99% of their medication with participants who would have taken no medication and considered them all as not adhering (and therefore assumed they would have received no benefit from being allocated to the amoxicillin arm). This clearly violated the exclusion restriction. However, the findings from the sensitivity analyses largely agreed with the main findings (where adherence was measured continuously), adding further strength to the conclusions of the paper.

Despite the fact that incomplete outcome and adherence data were minimal, their impact on findings remains unknown. However, as the condition under investigation is generally self-limiting, and outcome data included worsening of illness (a composite outcome collected from medical notes that included hospitalisation), we do not believe that the small amount of missing data would have severely impacted on the findings or conclusions drawn from this study. Indeed, sensitivity analyses demonstrate that clinical conclusions remain largely unaltered even when taking an extreme assumption about missing adherence data (see online supplementary appendix 3 for further details).

Comparison to existing literature

The findings from this study concur with those reported in the main findings paper,⁷ both of which are consistent with a recently published Cochrane review of antibiotics for acute bronchitis.³

Adherence to amoxicillin in this study was considerably higher than that reported in an observational study of antibiotics for adults with acute cough/LRTI in primary care.¹⁰ However, the participants recruited into this trial were similar to those recruited into the aforementioned observational study in terms of their baseline characteristics.³

Approaches for adjusting treatment effects for non-adherence while preserving randomisation have been in existence for approximately 20 years.²¹ However, they have largely been consigned to specialist methodological journals, rarely used in practice and when used, generally focussed on non-pharmacological treatments.²³ A recent publication using the same SMM approach as this paper on a clinical trial involving patients with depression demonstrates further that these methods are becoming more mainstream and should be reported alongside standard ITT estimates of treatment effectiveness, when there is also interest in knowing the efficacy of treatment.²⁴

Implications

The slight benefits gained from taking amoxicillin in adults consulting to primary care with acute uncomplicated LRTI must be balanced against the slight harms that amoxicillin causes in terms of side effects, as well as the associated contribution to antibiotic resistance.

While estimating the effectiveness of treatment using the ITT principle remains the gold standard in clinical trials, an ITT analysis only tells us the population-average effect that prescribing treatment has. The analysis therefore provides the answer to a question that is of primary interest to clinicians and policymakers (“What are the effects when this drug is prescribed?”). However, to a patient, the analysis may not be as informative (“What are the effects when I take this drug as prescribed?”). Some of these prescriptions will not be taken in their entirety, others not at all. In general, an ITT analysis does not estimate how good the medication is at treating the illness under consideration. Adjusting for adherence does allow for the estimation of this. If an ITT analysis shows little evidence of benefit, but an adherence-adjusted analysis demonstrates benefit, then the attention of policymakers should turn to ensuring that patients take their treatment properly. Estimating efficacy can provide additional insight into the potential benefit from treatment, and can indicate whether additional resources need to be allocated to the improvement of adherence to medication for specific conditions.

As was seen in this paper, if an ITT analysis finds little evidence of any benefit, and these conclusions are not affected by an adherence-adjusted analysis, it can be concluded that the intervention does not work in practice or principle.

Estimating efficacy in clinical trials while preserving the random allocation of participants to treatment groups is vital for inferring causal treatment effects. Standard software is available for implementing methods such as the SMM, and should become more widely used and reported in the medical literature.

Future research

While the main findings paper reported that a subgroup of older participants (aged 60 years or older) received no differential effect of treatment, investigating the efficacy of amoxicillin in this subgroup may be beneficial.

The SMM as presented in this paper relies on the assumption of a linear relationship between adherence (dose) and treatment efficacy. The incorporation of non-linear dose–response relationships into SMMs may increase the applicability of these methods in clinical trials, and is something that needs further attention.