Development and internal validation of a clinical rule to improve antibiotic use in children presenting to primary care with acute respiratory tract infection and cough: a prognostic cohort study

Summary

Background

Antimicrobial resistance is a serious threat to public health, with most antibiotics prescribed in primary care. General practitioners (GPs) report defensive antibiotic prescribing to mitigate perceived risk of future hospital admission in children with respiratory tract infections. We developed a clinical rule aimed to reduce clinical uncertainty by stratifying risk of future hospital admission.

Methods

8394 children aged between 3 months and 16 years presenting with acute cough (for ≤28 days) and respiratory tract infection were recruited to a prognostic cohort study from 247 general practitioner practices in England. Exposure variables included demographic characteristics, parent-reported symptoms, and physical examination signs. The outcome was hospital admission for respiratory tract infection within 30 days, collected using a structured, blinded review of medical records.

Findings

8394 (100%) children were included in the analysis, with 78 (0·9%, 95% CI 0·7%–1·2%) admitted to hospital: 15 (19%) were admitted on the day of recruitment (day 1), 33 (42%) on days 2–7; and 30 (39%) on days 8–30. Seven characteristics were independently associated (p<0·01) with hospital admission: age <2 years, current asthma, illness duration of 3 days or less, parent-reported moderate or severe vomiting in the previous 24 h, parent-reported severe fever in the previous 24 h or a body temperature of 37·8°C or more at presentation, clinician-reported intercostal or subcostal recession, and clinician-reported wheeze on auscultation. The area under the receiver operating characteristic (AUROC) curve for the coefficient-based clinical rule was 0·82 (95% CI 0·77–0·87, bootstrap validated 0·81). Assigning one point per characteristic, a points-based clinical rule consisting of short illness, temperature, age, recession, wheeze, asthma, and vomiting (mnemonic STARWAVe; AUROC 0·81, 0·76–0·85) distinguished three hospital admission risk strata: very low (0·3%, 0·2–0·4%) with 1 point or less, normal (1·5%, 1·0–1·9%) with 2 or 3 points, and high (11·8%, 7·3–16·2%) with 4 points or more.

Interpretation

Clinical characteristics can distinguish children at very low, normal, and high risk of future hospital admission for respiratory tract infection and could be used to reduce antibiotic prescriptions in primary care for children at very low risk.

Funding

National Institute for Health Research (NIHR).

Introduction

Combating the present and future threats of antimicrobial resistance is high on policy agendas internationally.^{1, 2, 3} A key goal of the UK¹ strategy is to slow emergence of antimicrobial resistance through judicious antibiotic prescribing. The vast majority of infections are managed outside of secondary care, with more than 80% of all health-service antibiotics prescribed by primary care clinicians.⁴ Antibiotic prescribing in primary care is increasing⁵ and directly affects antimicrobial resistance.⁶ Given that up to 50% of prescribed antibiotics are considered unnecessary,² primary care is a high priority area for antimicrobial stewardship research.^{1, 7}

Respiratory tract infections and cough in children (due to upper respiratory tract infections, bronchitis, bronchiolitis, infective exacerbation of asthma, and pneumonia) are the most common respiratory tract infection syndromes managed by health services internationally.^{8, 9} Primary care clinicians acknowledge that they prescribe antibiotics for a range of medical and non-medical reasons,¹⁰ particularly in children, who are seen as vulnerable¹¹ and whose clinical state can change rapidly. Many clinicians report that they prescribe antibiotics just in case,¹² to mitigate perceived risk of future hospital admission and complications,^{13, 14} and that failing to provide a prescription for a child who subsequently becomes seriously unwell is professionally unacceptable.¹¹ If primary care clinicians could identify children at low (or very low) risk of such future complications, the reduced clinical uncertainty could lead to a reduced use of antibiotics in these groups of patients.

Research in context.

Evidence before this study

At the time of funding, we did a systematic review for previous studies investigating factors associated with poor prognosis in children presenting to primary care with acute cough and respiratory tract infection. We searched MEDLINE for articles published from 1966 to January, 2012) using the following search terms: “respiratory tract infection [MeSH Terms] OR respiratory infection* OR rti OR lrti OR urti OR lri OR uri OR chest infection* OR cough OR dyspnoea OR congestion OR (lung consolidation) OR (lobar pneumonia) OR (difficult breath*) OR respiration disorder* AND child* OR schoolchild* OR preschool* OR pediatric* OR paediatric* OR parent OR parents OR parental OR mother OR father OR mom OR dad OR mum OR caregiver OR guardian OR carer OR infant OR infancy AND primary care OR family practice OR general practice OR family medicine OR community healthcare OR primary healthcare OR ambulatory care NOT asthma OR malaria OR tuberculosis”. We found no prognostic evidence to support GPs in identifying children at lowest or highest risk of future complications.

Added value of this study

This is the first primary care cohort study to identify the baseline characteristics of children with acute respiratory tract infection and cough (the most common problem managed by health-care services internationally) at risk of future poor outcome. The study showed hospital admission for respiratory tract infection in the 30 days after recruitment was uncommon, but could be accurately predicted with a clinical rule consisting of short illness, temperature, age, recession, wheeze, asthma, and vomiting (mnemonic STARWAVe). Using a simple one-point-per-item, STARWAVe could be used to stratify children into one of three hospital admission risk groups: very low, normal, and high risk.

Implications of all the available evidence

STARWAVe could reduce clinical uncertainty, and thereby antibiotic use, in children at very low risk of future hospital admission who form the majority of children presenting to primary care with acute respiratory tract infection and cough. If antibiotic prescription in this group halved, remained static in the normal risk stratum, and increased to 90% in the high risk stratum, a 10% overall reduction in antibiotic prescribing would be achieved, similar to other contemporary antimicrobial stewardship interventions.

This Article reports the assembly and analysis of a prospective cohort to develop and internally validate a clinical rule to identify children presenting with a respiratory tract infection at risk of future hospital admission. Specifically, we postulated that baseline clinical characteristics could be used to distinguish good from poor prognosis in children presenting to primary care with acute respiratory tract infection and cough, and that clinically useful coefficients (for use with computer assistance) and points-based algorithms could be developed.

Methods

Study design and participants

This study¹⁵ was a prospective, prognostic cohort study recruiting participants between July, 2011, and June, 2013; the design of this study has been described previously.¹⁵ Briefly, general practitioners and prescribing nurse practitioners (referred to from here on as clinicians) working in general practitioner practices were trained by four UK university hubs (Bristol, London, Oxford, and Southampton). In order to minimise confounding by indication, clinicians who self-reported prescribing antibiotics in 30% or less of children with respiratory tract infections were invited to recruit and in order to minimise recruitment bias in the study, were asked to identify a priori their preferred recruitment strategy (consecutive or first eligible on the day).

Children were eligible for recruitment if they were aged between 3 months and 16 years and presented with the main symptom of acute (≤28 days) cough with other respiratory tract infection symptoms (such as fever and coryza). Children with an infected exacerbation of asthma and those who were severely unwell (eg, requiring same-day hospital assessment or admission) were included. Children were excluded if they presented with a non-infective exacerbation of asthma, were at high risk of serious infection (immunocompromised, for example with cystic fibrosis), required a throat swab for clinical management (which were taken for research purposes in a subgroup of children), had been previously recruited to the study or recently participated in other research, or had temporarily registered at the practice.

Procedures

Clinicians were asked to record reasons for not inviting potentially eligible children to participate in the study and to report a global illness severity (successfully used in a previous study¹⁶ and scored from zero to ten) for those invited but declining participation. After receiving informed consent from the children's parents (and assent from children aged ≥11 years), clinicians completed a structured online (or paper) case report form (appendix). This form recorded eight sociodemographic and four past medical history items, 33 parent-reported symptoms (including symptom severity of either mild, moderate, or severe in the previous 24 h), 14 physical examination signs (including vital signs and global illness severity), and the prescription of antibiotics. Children were assessed for current asthma at medical notes review, which was deemed present if asthma was noted in the medical notes and asthma medication was issued in the previous 12 months. All baseline data, except for asthma, were collected blinded to the main outcome.

Outcomes

The primary outcome was hospital admission (excluding emergency department attendance only) for any respiratory tract infection in the 30 days after recruitment and was done blind to baseline characteristics. This information was collected using a structured review of the primary care electronic medical record, including reading scanned hospital discharge summaries. Any practices not completing review forms were telephoned for the primary outcome. A double review of medical notes was done for all participants admitted to hospital and a further random 1% of participants to estimate inter-rater reliability.

Statistical analysis

Our original sample size estimate assumed a 5% two-sided significance level, 80% power, 2% of patients admitted to hospital, and the requirement to detect odds ratio (OR) of 2·4 (ORs of between 2·5 and 5 had been previously observed)¹⁷ with a predictor prevalence of 25%, giving a required analytical sample of 2588 participants. Assuming 10% attrition, 30% prescription of antibiotics (in case confounding by indication required their exclusion), and a 50:50 data split for derivation and validation, a sample of 8216 participants was needed. However, hospital admissions were lower than expected¹⁸ and because bootstrap internal validation is considered to be better than data splitting in the presence of few outcome events,¹⁹ a revised sample size of 5751 provided 80% power to detect an OR of 2·4 with a hospital admission rate of 1% and, in the absence of confounding by indication where all children contributed to analysis, 8216 children increased the power to 95%.

All data were analysed using Stata (version 13.1). We compared illness severity of recruited children with children declining study participation. We calculated kappa scores to assess inter-rater reliability of the main outcome. Univariable analyses to describe associations between candidate predictors and hospital admission, and to test for model entry, were done with logistic regression. χ², Fisher's exact tests, and non-parametric tests (such as the Mann-Whitney) were used to test several categories and non-normal distributions.

Multivariable analysis was done with logistic regression. Symptom severity (mild, moderate, or severe) was dichotomised between severe and other categories if more than 5% of symptoms were severe, or between moderate and mild if 5% or less were severe. Recognised clinical cutoffs were used for continuous data (eg, temperature >37·8°C,²⁰ age-related heart and respiratory rates,²¹ oxygen saturation ≤95%,²² and capillary refill time ≥3 s).²¹ Remaining continuous variables were dichotomised with upper or lower quartile thresholds as appropriate. Deprivation was assigned using the UK 2008 census Index of Multiple Deprivation score zip codes.²³

The initial model inclusion criterion was p<0·05 with putative predictors entered using backward-stepwise selection and retained where p<0·01. For sensitivity analyses, a slightly less conservative p value of <0·05 was used to explore effects of other putative predictors. The ability of the model to discriminate between hospitalised and non-hospitalised children was summarised using the area under the receiver operating characteristic (AUROC) curve, with 95% CIs. We internally validated coefficient-based models with the bootstrap procedure described by Steyerberg²⁴ using 200 iterations to calculate a validated AUROC and a calibration slope (shrinkage factor) by which we multiplied model coefficients to derive internally validated ORs.

Further sensitivity analyses were done to explore the effect on the AUROC of dichotomising continuous variables and collapsing multicategorical variables. Residual outliers of the final model were tested for goodness of fit. Potential interactions between all predictors in the model and variables excluded from the model but of borderline significance were also tested. The role of antibiotic prescription and potential for confounding by indication was investigated by calculating the AUROC of the final model excluding children prescribed antibiotics and using propensity scoring.

To develop a rule for clinical use without a computer, we developed a scoring system with Wald estimates (calculated by dividing the coefficient by its standard error, squaring it, and rounding it) of each predictor and a simpler scoring system in which one point was given to each predictor. For both approaches, the stratification of children into very low, normal, and high risk of hospital admission was decided using the sensitivity and specificity for each potential cutoff compared with the overall risk of hospital admission. This study is registered on UK Clinical Research Network Portfolio as the TARGET study, reference number 9334.

Role of the funding source

This study was funded by the National Institute for Health Research and sponsored by the University of Bristol. The funder and sponsor of this study had no role in the study design; data collection, analysis or interpretation; writing of the paper; patient recruitment; or any aspect pertinent to the study. The corresponding author had full access to all the data and took final responsibility to submit the paper for publication.

Results

519 general practitioners and nurse practitioners recruited children at 247 practices across England. Clinicians offered 9043 invitations to children for study recruitment, of which 8879 (98%) were accepted, and for which 8394 (95%) received the children's parents' valid consent; all these children met the eligibility criteria and made up the final analytical sample (figure 1). There was no difference in global illness severity score between the 164 children declining participation (median 3, IQR 2–4) and the final recruited sample (median 3, IQR 2–4).

Figure 1.

Flow chart

*Other reasons include perceived difficulties with completing symptom diary, language barriers, another child was already recruited, person attending with the child was not their carer, concerns about access to medical records, and child had complex needs.

In the final sample, children's median age was 3 years (IQR 1–6), 4365 (52%) were boys, 6547 (78%) were white (similar to UK Census 2011 data),²⁵ and 3121 (37·2%) were prescribed an antibiotic. Mothers' median age at the child's birth was 30 years (IQR 26–34) and 1511 (18·0%) mothers smoked (similar result to the 2012 English Household survey).²⁶ Families' median deprivation score was 16·7 (IQR 8·8–29·5), also similar to national data results.

Medical notes were reviewed for the primary outcome for 8394 (100%) children, of whom 78 (0·9%, 95% CI 0·7%–1·2%) were admitted to hospital for a respiratory tract infection in the 30 days after recruitment. Inter-rater agreement for admission to hospital for respiratory tract infection was 90% (kappa 0·80). Median time to hospital admission was 2 days (IQR 1–11), with 15 (19%) of children admitted to hospital on the day of recruitment (day 1), 33 (42%) admitted between days 2 and 7, 11 (14%) admitted between days 8 and 14, and 19 (24%) admitted between days 15 and 30. Hospital discharge diagnoses were lower respiratory tract infection (15 [19%]); bronchiolitis (14 [18%]); viral wheeze (12 [15%]); upper respiratory tract infection (ten [13%]); croup (six [8%]); infected exacerbation of asthma (six [8%]); tonsillitis (five [6%]); viral illness (four [5%]); febrile illness (two [3%]); pneumonia (one [1%]) and missing data (three [4%]), with 26·9% of all discharge diagnoses suggesting a possible bacterial cause (lower respiratory tract infection, tonsillitis, and pneumonia) for which antibiotics might have been needed. Missing data for candidate predictors was infrequent (<2%), with the exception of oxygen saturation (50%, due to unavailable oxygen saturation monitors).

Children who were admitted to hospital were younger, more likely to be boys and of white ethnicity, have current asthma, and have more consultations for respiratory tract infection in the previous 12 months than non-hospitalised children (table 1). The number of siblings, breastfeeding at 3 months, maternal age, and maternal smoking were not associated with hospital admission (data not shown).

Table 1.

Univariable associations (p<0·05) with hospital admission

	Hospitalised children	Non-hospitalised children	Odds ratio	95% CI	p value
Sociodemographics and past medical history
Current asthma*	19/78 (24%)	731/8316 (9%)	3·34	1·98–5·65	<0·001
Age <2 (years)	47/78 (60%)	2781/8316 (33%)	3·02	1·91–4·76	<0·001
≥6 RTI consults in previous 12 months	9/78 (12%)	437/8316 (5%)	2·35	1·17–4·74	0·01
Male	51/78 (65%)	4280/8316 (51%)	1·78	1·12–2·85	0·01
Non-white ethnicity	8/77 (10%)	1798/8272 (22%)	0·42	0·20–0·87	0·02
Carer-reported general symptoms
Illness duration ≤3 days	43/78 (55%)	2352/8312 (28%)	3·11	1·99–4·88	<0·001
Severity of illness ≥7/10	33/77 (43%)	2114/8291 (25%)	2·19	1·39–3·45	<0·001
Carer-reported symptoms (present any time during illness)
Breathing quickly†	44/78 (56%)	2394/8312 (29%)	2·37	1·51–3·72	<0·001
Wheezing	44/78 (56%)	3241/8310 (39%)	2·02	1·29–3·18	0·002
Vomiting‡	33/78 (42%)	2314/8312 (28%)	1·90	1·21–2·99	0·005
Change in crying	20/78 (26%)	1362/8285 (16%)	1·75	1·05–2·92	0·03
Carer-reported symptoms (in the previous 24 h)
Moderate-to-severe vomiting‡	20/78 (26%)	820/8303 (10%)	3·14	1·88–5·26	<0·001
Severe fever	12/78 (15%)	529/8283 (6%)	2·69	1·43–4·96	0·001
Severe disturbed sleep	23/78 (30%)	1328/8272 (16%)	2·19	1·34–3·57	0·001
Breathing quickly†	32/78 (41%)	1599/8298 (19%)	2·91	1·85–4·59	0·001
Moderate-to-severe wheezing in chest	27/78 (35%)	1589/8297 (19%)	2·23	1·40–3·57	0·001
Severe reduction in eating	10/78 (13%)	419/8277 (5%)	2·70	1·41–5·40	0·002
Moderate-to-severe reduction in urine passed	9/78 (12%)	455/8284 (5%)	2·24	1·11–4·52	0·02
Severe dry cough	10/78 (13%)	551/8285 (7%)	2·06	1·06–4·03	0·03
Moderate-to-severe diarrhoea	7/78 (9%)	337/8305 (4%)	2·33	1·06–5·12	0·03
Moderate-to-severe reduced fluid intake	17/78 (22%)	1140/8291 (14%)	1·75	1·02–3·00	0·04
General clinical examination
Irritable or drowsy	5/78 (6%)	123/8301 (1%)	4·55	1·81–11·47	0·007
Pallor	15/77 (20%)	808/8306 (10%)	2·25	1·27–3·96	0·004
Nasal flaring	5/78 (6%)	97/8306 (1·2%)	5·80	2·29–14·66	0·003
Grunting	4/78 (5%)	72/8304 (1%)	6·18	2·20–17·36	0·005
Temperature ≥37·8°C	20/78 (26%)	1026/8294 (12%)	2·75	1·64–4·58	<0·001
Oxygen saturation ≤95%	11/39 (28%)	392/4155 (9%)	3·77	1·86–7·63	<0·001
Severity of illness ≥ 4/10	38/78 (49%)	1691/8292 (20%)	3·71	2·37–5·80	<0·001
Clinician's gut feeling that something is wrong	30/78 (39%)	1676/8299 (20%)	2·47	1·56–3·91	<0·001
Chest examination
Tachypnoea§	21/78 (28%)	1222/8272 (15%)	2·12	1·28–3·52	0·002
Recession	25/78 (32%)	379/8304 (5%)	9·86	6·06–16·05	<0·001
Wheeze	34/78 (44%)	1203/8303 (14%)	4·56	2·90–7·16	<0·001
Crackles or crepitations	25/78 (32%)	1575/8303 (19%)	2·02	1·25–3·25	0·003

RTI=respiratory tract infections.

^*Defined as present if asthma in medical notes and asthma drugs issued in the previous 12 months²⁷

Children admitted to hospital had shorter illnesses before consultation and higher carer-reported illness severity scores than non-hospitalised children (table 1). Parents of children admitted to hospital were also more likely to report breathing faster, wheezing, vomiting, change in cry, fever, disturbed sleep, eating less, dry cough, diarrhoea, consuming fewer fluids, and passing less urine. There was no evidence of differences between hospitalised and non-hospitalised children with regard to recent illness deterioration, productive or wet cough, barking or croupy cough, blocked runny nose, and chills, shivering, low energy, fatigue, or lethargy (data not shown).

The following 12 examination findings were associated with increased risk of hospital admission: irritability or drowsiness; pallor; nasal flaring; grunting; body temperature of 37·8°C or more; oxygen saturation of 95% or less; increased clinician global impression of severity of illness; clinician gut feeling that there was something wrong; tachypnoea; intercostal or subcostal recession; wheezing; and crackles or crepitations (table 1). There was no evidence of differences between hospitalised and non-hospitalised children with regard to bronchial breathing (unilateral or bilateral), inflamed pharynx or tonsils, age-adjusted tachycardia, capillary refill time, or stridor (data not shown).

The initial multivariable model contained the following seven predictors with p values of less than 0·01: age of less than 2 years, current asthma, illness up to 3 days before consultation, moderate or severe vomiting in the previous 24 h, intercostal or subcostal recession, wheeze, and body temperature of 37·8°C or more. The initial model AUROC curve was 0·81 (95% CI 0·77–0·86). The model intercept (−6·65) suggests that the probability of hospital admission for children with no predictors was 0·14%.

Low oxygen saturation was added to the multivariable model but did not reach the prespecified threshold (p=0·16). Variables with p values between 0·01 and 0·1 using backward stepwise variable selection were examined by adding each one to the initial model. Two variables met this criterion, ethnicity (p=0·04) and carer-reported severe fever (p=0·08). Apart from an 11% higher prevalence of white ethnicity among the children admitted to hospital, a detailed analysis of ethnic groups revealed no further marked differences and no interaction with the variables already in the model. Carer-reported severe fever within 24 h was interchangeable with clinician-identified high temperature in the multivariable model, producing similar results. We therefore produced a combined variable using the presence of either of these markers, which identified 25 (32·1%) of 78 hospitalised children and 1338 (16·2%) of 8262 non-hospitalised children (p<0·001). This combined variable was included in the final multivariable model (p=0·006) and slightly increased the AUROC. Compared with children aged 3 years or younger, there was an incremental increased risk of hospital admission with decreasing age: 24–35 months OR 1·64 (95% CI 0·73–3·67), 12–23 months OR 3·45 (1·81–6·59), and less than 12 months OR 4·90 (2·57–9·30; p=0·03). We decided against including age as a continuous variable because it reduced the AUROC and would be less useful clinically. Unilateral or bilateral wheeze was investigated but showed insufficient superiority compared with presence or absence of wheeze to warrant separate categorisation. A goodness-of-fit test identified an outlier of five children with identical model parameters. When removed, parameter estimates changed very little and the goodness-of-fit test became insignificant (p=0·88). We found no evidence for multiplicative interactions between all predictors in the final model.

There was no evidence of association between hospital admission and the prescription of antibiotics at the recruitment consultation (32% in hospitalised children and 37% in non-hospitalised children, p=0·36). Removing the 3121 children who were prescribed antibiotics, backward stepwise regression yielded the same variables as the full cohort, with an additional clinician variable (gut feeling that something was wrong OR 2·23, 95% CI 1·00–5·01, p=0·05). Propensity scoring on the full cohort also yielded the same initial seven-variable model and forcing antibiotic prescription and gut feeling resulted in a slightly decreased AUROC. There was also no evidence of interactions between initial model variables and antibiotic prescribing or gut feeling.

The final multivariable model included a combined carer-reported severe fever with clinican measured temperature of 37·8°C or more variable (table 2). As a result, the AUROC increased from 0·81 to 0·82 (95% CI 0·77–0·87, figure 2). Bootstrap validation yielded a difference of −0·011 (95% CI −0·008 to −0·014), giving a final validated AUROC of 0·81. Because three of the final model characteristics could be linked to asthma (current asthma, wheezing, and intercostal recession), we did a post-hoc assessment of the final model in the 750 (8·9%) children with asthma, 19 of whom were admitted to hospital. Intercostal recession, age younger than 2 years, short illness duration, and wheeze remained associated with hospital admission, and 95% CIs for vomiting and high body temperature or fever in the previous 24 h were also consistent with the final model 95% CIs (appendix).

Table 2.

Final multivariable^* predictors of hospitalisation (all p<0·01)^*

	Data source	Odds ratio†	95% CI	p value
Current asthma‡	Notes review	3·93	2·20–7·03	<0·001
Inter and subcostal recession	Clinician	3·82	2·23–6·62	<0·001
Age of child (<2 years)	Parent	3·42	2·12–5·58	<0·001
Illness duration (<4 days)	Parent	2·77	1·77–4·35	<0·001
Moderate-to-severe vomiting in the last 24 h§	Parent	2·56	1·54–4·31	<0·001
Wheeze	Clinician	2·16	1·28–3·60	0·004
Body temperature >37·8°C or parent-reported severe fever in the last 24 h	Clinician or parent	1·99	1·22–3·25	0·006

^*Model includes 8340 (99·4%) of 8394 cohort participants; the original model intercept coefficient was −6·65 (95% CI −7·21 to −6·10), suggesting that the probability of hospital admission for children with no predictors was 0·14%.

^†Odds ratios calculated using shrunken estimates from the bootstrap internal validation calibration slope.

^‡Defined as present if asthma in medical notes and asthma drugs issued in the previous 12 months.

^§Including after cough.

Figure 2.

Final model receiver operator characteristic curves for (A) coefficient and (B) points-based models

Using weighted Wald estimates or a simple scoring system yielded a similar probability of hospital admission within comparative groups (table 3). The simpler, more clinically useful scoring system was therefore chosen; the score cutoffs for the three different groups selected were based on the sensitivity and specificity of using different thresholds. The final risk strata (table 3) were very low (0·3% risk of hospital admission) containing 5593 (67%) children, normal (1·5% risk, similar to the overall population) for 2520 (30%) children, and high (11·8% risk) for 204 (3%) children. The AUROC for this final points-based model was similar to the coefficient-based model (0·81, 95% CI 0·76–0·85). The percentage of children with discharge diagnoses suggesting possible bacterial infection by risk strata were very low (0·09%), normal (0·38%), and high (3·92%). The percentage of children prescribed an antibiotic (immediate and delayed) by risk strata were also low (33%), normal (44%), and high (65%).

Table 3.

Risk of hospital admission: simple scoring system

	Number of predictors	Hospitalised children	Non-hospitalised children	Risk of hospital admission*†
				Risk percentage	95% CI
Very low risk	0 to 1	17 (22%)	5576 (68%)	0·3% (1 in 328)	0·2%–0·4%
Normal risk	2 to 3	37 (47%)	2483 (30%)	1·5% (1 in 68)	1·0%–1·9%
High risk	4 or more	24 (31%)	180 (2%)	11·8% (1 in 8·5)	7·3%–16·2%
Total		78 (100%)	8239 (100%)	0·9% (1 in 106)	0·7%–1·2%

^*Risk of hospital admission using Wald estimates were 0·2% (or 1 in 449) for the very low risk group, 1·0% (or 1 in 104) for the normal risk group, and 4·3% (or 1 in 23) for the high risk group.

^†The sensitivity and specificity using the cutoff of (normal or high risk) versus (very low risk) were 78·2% and 67·7%. The sensitivity and specificity using the cutoff (high risk) versus (normal or very low risk) were 30·8% and 97·8%.

Discussion

Using a well characterised, large, representative cohort of children presenting to primary care with the most frequently managed acute paediatric health-care problem internationally, we found that subsequent hospital admission was uncommon, and that a simple, one-point-per-item rule consisting of short (≤3 days) illness; temperature; age (<24 months); recession; wheeze; asthma; and vomiting (mnemonic STARWAVe) can be used to identify children at very low (0·3%), normal (1·5%), and high (11·8%) risk of future hospital admission for respiratory tract infection. Most admissions were for lower respiratory tract infection, bronchiolitis, or viral wheeze, with only a quarter of discharge diagnoses suggesting a possible bacterial cause for which antibiotics might have been indicated.

Our previous systematic review²⁸ suggests that this is the largest and most rigorous prognostic study of children with respiratory tract infections in primary care. Participating children were no less unwell than were those who were invited but declined study participation. Baseline characteristics were pragmatic, measured according to routine clinical practice with standardised reporting forms blind to the primary outcome, and with high levels of data completeness. Our primary outcome is clinically relevant, reflects a key driver of primary care antibiotic prescribing,¹³ was assessed blind to baseline characteristics with 100% follow-up, and had high levels of inter-rater agreement. Because it is not possible to eliminate confounding by indication in observational studies (clinicians need to use antibiotics in some children), we used a less than 30% antibiotic prescribing threshold to minimise its effect, as well as testing for its presence by excluding children prescribed antibiotics and using propensity scoring. We used a stringent model retention criterion (p<0·01) because we had 55 candidate predictors and a low number of hospital admissions. We developed both coefficient (requiring computer assistance) and points-based models, which were surprisingly similar in their prognostic value.

The main limitation is the relatively small number of hospital admissions for respiratory tract infections, which affects two areas. First, we were not able to externally validate the final model. Although external validation is desirable before clinical application, bootstrap validation takes model over-optimism into account and is considered better than data splitting, especially in the presence of limited outcome events.²⁴ Our results are biologically plausible and the associations substantial, and by analogy, it is reasonable for clinical practice to change on the basis of a single, well done randomised trial. Second, our model development breached the widely quoted rule of ten events per candidate predictor. However, this is only a rule of thumb, with little theoretical or empirical justification,²⁹ and the consequences of variable selection are strongly dependent on the strength of association between candidate predictors and the outcome. Finally, it is possible that other biomarkers could be used to improve prognostic certainty, for example, pulse oximetry (our missing data might have impeded a full assessment of usefulness); inflammatory markers, which are diagnostically useful;²⁷ and rapid point-of-care pathogen tests. We chose not to include C-reactive protein testing because of the recruitment disincentive to children of taking blood. We considered but discounted assessing generalisability by comparing children's characteristics with those of routine datasets because of the variable coding of respiratory tract infections in primary care³⁰ and the fact that cohort children had acute cough and respiratory tract infection, representing a range of underlying diagnoses including upper and lower respiratory tract infections.

Although new evidence is providing early warning of impending critical illness in secondary care,³¹ we are not aware of similar, representative, large-scale studies to establish the prognostic usefulness of children's clinical parameters in primary care,²⁸ that is, studies that have developed algorithms to predict future complications. We are aware of two small primary care studies, the first of which also showed fever and abnormal chest signs predicted reconsultation for worsening illness,³² although the second did not find any prognostically useful characteristics.³³ Other studies^{34, 35} have assessed the diagnostic usefulness of symptoms and signs for a range of serious infections (including meningitis, septicaemia, pyelonephritis, and pneumonia) arising within shorter timeframes of index testing, with all but one³⁵ done in secondary care. In contrast to our results, these studies showed parental concern,³⁴ dyspnoea,³⁵ and clinician gut feeling³⁵ were useful for diagnosing pneumonia.

The main value of our results is to reduce clinical uncertainty and antibiotic use in children least likely to benefit from them, namely those at very low risk of future hospital admission. We believe a no antibiotic strategy will be safe in this group for two reasons: first, only a minority of discharge diagnoses suggested a bacterial cause, and second, the preventive value of antibiotics against future complications is small in children at normal risk of complications,³⁶ so is likely to be very small in the very low risk stratum. We believe a no antibiotic or delayed antibiotic prescribing strategy is appropriate for the middle (normal) risk stratum because this is the treatment strategy recommended by NICE for this group³⁷ and we believe children at high risk of hospital admission should be closely monitored for signs of deterioration, with consideration given to proactively arrange same-day or next-day follow-up and prescribe an immediate antibiotic.

Our data show that 1846 (33%) of the very-low-risk stratum children received antibiotics. Because these children represent the majority (67%) of all the participants, a 10% overall reduction in antibiotic prescription would be achieved if prescription in this group halved, remained static in the normal risk stratum, and increased to 90% in the high risk stratum, resulting in a similar effect size to other contemporary antimicrobial stewardship interventions.^{38, 39} Given that national data suggest that 50% of children with respiratory tract infection at present receive antibiotics,⁴⁰ effects could be increased in practices providing more prescriptions. The AUROC is useful, but not perfect, meaning that the rule should supplement, not supplant, clinical judgment. As a result, clinicians should use safety netting,⁴¹ advising parents about the conditions under which they should reconsult.

Our results complement existing evidence to support antimicrobial stewardship in primary care.^{42, 43, 44} Rule constituents can be easily measured, taught, and remembered by a wide range of health-care professionals, as well as incorporated into electronic health record templates and decision support tools.

Further research is needed to externally validate the rule and to investigate its effects on antibiotic prescription and children's clinical outcomes. Demographic characteristics, parent-reported symptoms, and clinical examination findings can distinguish children at very low, normal, and high risk of future hospital admission for respiratory tract infection. General practitioners and primary care nurses might find this clinical rule useful to reduce prescribing antibiotics in children at very low risk of future hospital admission.