Validity of British Thoracic Society guidance (the CRB-65 rule) for predicting the severity of pneumonia in general practice: systematic review and meta-analysis

Abstract

Background

The CRB-65 score is a clinical prediction rule that grades the severity of community-acquired pneumonia in terms of 30-day mortality.

Aim

The study sought to validate CRB-65 and assess its clinical value in community and hospital settings.

Design of study

Systematic review and meta-analysis of validation studies of CRB-65.

Method

Medline (1966 to June 2009), Embase (1988 to November 2008), British Nursing Index (BNI) and PsychINFO were searched, using a diagnostic accuracy search filter combined with subject-specific terms. The derived (index) rule was used as a predictive model and applied to all validation studies. Comparison was made between the observed and predicted number of deaths stratified by risk group (low, intermediate, and high) and setting of care (community or hospital). Pooled results are presented as risk ratios (RRs) in terms of over-prediction (RR>1) or under-prediction (RR<1) of 30-day mortality.

Results

Fourteen validation studies totalling 397 875 patients are included. CRB-65 performs well in hospitalised patients, particularly in those classified as intermediate (RR 0.91, 95% confidence interval [CI] = 0.71 to 1.17) or high risk (RR 1.01, 95% CI = 0.87 to 1.16). In community settings, CRB-65 over-predicts the probability of 30-day mortality across all strata of predicted risk, low (RR 9.41, 95% CI = 1.75 to 50.66), intermediate (RR 4.84, 95% CI = 2.61 to 8.69), and high (RR 1.58, 95% CI = 0.59 to 4.19).

Conclusion

CRB-65 performs well in stratifying severity of pneumonia and resultant 30-day mortality in hospital settings. In community settings, CRB-65 appears to over-predict the probability of 30-day mortality across all strata of predicted risk. Caution is needed when applying CRB-65 to patients in general practice.

INTRODUCTION

Community-acquired pneumonia (CAP) has an annual incidence of 5–11 per 1000, accounting for 5–12% of all cases of adult lower respiratory tract infection managed by GPs.¹ Appropriate management for patients with suspected CAP must be determined by GPs at initial presentation, particularly with regard to whether or not to manage a patient in the community or refer them to hospital.² The proportion of adults with CAP who require hospital admission in the UK varies between 22% and 42%.1 This proportion varies in other countries, possibly due to differing structures of primary and secondary healthcare systems.¹ Furthermore, of those initially managed in the community, less than 10% subsequently require hospital admission. At the other end of the severity spectrum, between 5% and 10% of patients admitted to hospital require management in an intensive care unit (ICU).¹

Severity assessment in patients with suspected CAP plays a key part in planning appropriate management in both community and hospital settings, as the setting of care has an impact on the level of treatment given to the patient as well as the overall costs of treatment.¹ The CRB-65 score (Figure 1) is a clinical prediction rule derived to determine the severity of CAP and to aid the physician in arranging appropriate treatment.³ It ranks patients into strata of low, intermediate, or high risk of mortality based on four criteria (new-onset confusion; respiratory rate ≥30/minutes; systolic blood pressure <90 mmHg or diastolic blood pressure ≤60 mmHg; and age ≥65 years), in terms of estimated mortality risk (Figure 1). It is also forms the basis of an algorithm to determine the most appropriate site of care.¹ The CRB-65 score is a simplified version of the CURB-65 rule (inclusion of urea estimation: raised >7 mmol/l), and has been advocated for use in community settings as its criteria depend on clinical assessment (history and physical examination) alone.⁴

Figure 1.

Severity assessment by the CRB-65 in terms of risk strata and subsequent management strategy for patients with suspected community-acquired pneumonia.^3,4

How this fits in

The CRB-65 rule grades the severity of community-acquired pneumonia in terms of 30-day mortality risk. Although it is intended for use in the community setting the majority of validation studies are hospital based. This systematic review suggests that the CRB-65 rule performs well in hospital settings but may over-predict the probability of 30-day mortality in community settings, and should be used with caution.

There are specific methodological standards that are applied to clinical prediction rules before their use in clinical practice can be recommended. Ideally, a rule undergoes three steps: derivation, narrow and broad validation, and impact analysis (randomised controlled trial assessing its effectiveness and cost-effectiveness).^5–7

The CRB-65 rule has been validated extensively in separate studies, although for a clinical prediction rule specifically intended for community use there have been few validation studies performed wholly in primary care.^1,8 As clinical prediction rules may not perform well in practice because of deficiencies in the development methods or because of differences between the original sample and the validation sample, it is good practice to assess their validity across a broad range of eligible patient groups.⁹ The aim of this study was to perform a comprehensive systematic review and meta-analysis of validation studies of CRB-65, to determine its accuracy in predicting 30-day mortality and assess how well it performs in community and hospital settings.

METHOD

Search strategy

A search strategy was designed for PubMed in order to retrieve all relevant articles. MeSH terms ‘community acquired infection’, ‘respiratory tract infection’, ‘pneumonia’, ‘severity indices’, and ‘prognosis’ were combined, and potential studies were retrieved. This subject-specific search was combined with a methodological filter for clinical prediction rules.¹⁰

PubMed was searched from 1966 to June 2009. In addition, Embase, the Cochrane Library, and Medline databases were searched for relevant studies. Google Scholar was used to track references to each selected study, and hand searches of references in the included studies were also performed. Authors of published studies were also contacted for additional data and for knowledge of any unpublished studies.

Selection

Inclusion criteria for the systematic review were as follows: (1) study design – cohort study; (2) patient population – adult patients (≥16 years) with a primary diagnosis of CAP; (3) explanatory variables – CRB-65 score calculated; (4) setting of care – community-based or hospital-based patients; and (5) outcome measure – death within 30 days. Studies that used the same dataset for more than one publication were included once in the meta-analysis.

Validity assessment

Quality assessment was independently performed by two researchers following the methodological standards of McGinn for validation studies of clinical prediction rules.⁵

Data abstraction

Abstracts of potential studies were independently screened by two researchers and those considered were read fully in duplicate and their suitability for inclusion in the study was independently determined by both researchers. Disagreements were managed by consensus and the involvement of a third researcher.

Statistical methods

The initial derivation study of the CRB-65 rule was used as the predictive model to which all validation studies were compared.³ The number of deaths as predicted by the CRB-65 severity index was compared to the observed number of deaths in each of the validation studies, across the three strata of risk in the CRB-65 rule – low risk (score = 0), intermediate risk (score = 1 or 2), and high risk (score = 3 or 4) (Figure 1). In order to calculate the predicted score, the proportionate mortality estimate from the original derivation study was applied according to the three risk strata – low risk (mortality = 1.2%), intermediate risk (mortality = 8.2%), and high risk (mortality = 31.3%),³ an approach previously used in a community-based validation study.⁸ The predicted:observed risk ratios for these three risk strata are presented separately for patients who were treated in the community or in hospital. Results are presented as risk ratios (RRs) with 95% confidence intervals (CIs). RR<1 indicated an under-prediction of death by the CRB-65 rule (the observed number of deaths is greater than the predicted number) and RR>1 indicated an over-prediction of death by the CRB-65 rule (the observed number of deaths is less than the predicted number). RR = 1 indicated a perfect calibration between observed and predicted deaths. Assessment of publication bias was also made by means of generating a funnel plot, and statistical testing for publication bias.

Review Manager 5 software from the Cochrane Collaboration was used to perform the analysis, determine heterogeneity, and produce forest plots. RRs were calculated with 95% CIs using the Mantel-Haenszel statistical method. A random-effects analysis was performed, and heterogeneity was described by the I² statistic.

RESULTS

Description of included studies

The search identified 14 studies with a total of 397 875 patients (Figure 2).^8,11–23 Table 1 summarises the characteristics of the included studies. Additional data were provided from three studies,^13,15,23 and clarification of included overlapping subjects obtained from two other studies.^12,16 Only one study was performed entirely in a primary care setting,⁸ although two other studies did include data from patients who were treated as outpatients.^12,14 The included studies ranged in size from 105 patients to 388 406 patients, with a broadly equal sex balance. Two studies limited the inclusion for age to ≥65 years.^8,23 Most studies also had mortality at 30 days as an outcome measure, corresponding directly in timing to the original derivation study.³ However, two studies reported in-hospital mortality and one reported mortality at 42 days.^12,23 Thirty-day mortality was obtained from the authors of one of these studies,²³ and in-hospital mortality was used as an approximation to 30-day mortality in the other two. The quality-assessment standards of the included studies are summarised in Figure 3. No publication bias exists in the studies (P>0.05) (Appendix 1).

Figure 2.

Selection of studies for inclusion in the meta-analysis.

Table 1.

Characteristics of included studies.

	Setting	Study type	Participants; age; sex	Inclusion	Exclusion	Outcome measure
Barlow et al, 200711	Hospitalised inpatients	Retrospective analysis of prospectively collected data (before and after study)	n = 503; median 74 years (range 16–98 years); male n = 197	Receiving antibiotics for a suspected lower respiratory tract infection +new infiltrate on chest radiograph, or clinically diagnosed as CAP by specialist registrar or consultant doctor	One or more of: non-pneumonia diagnosis non-pneumonia diagnosis aspiration, hypostatic or hospital-acquired pneumonia diagnosis of CAP changed before discharge HIV+, neutropenic or markedly immunosuppressed progressive malignancy chronic respiratory disease other than asthma or chronic obstructive pulmonary disease (COPD) age <16years	30-day mortality
Bauer et al, 200612	CAPNETZ centres, inpatients, outpatients	Consecutive prospective cohort	n = 2184: 1646 inpatients, 538 outpatients; mean 62.5 years (range 18–99 years); male n = 1245	New pulmonary infiltrate on chest radiograph + one of: fever cough purulent sputum focal chest signs dyspnoea pleuritic pain	Acquisition pneumonia after hospital admission; severe immunosuppression; expected terminal event of severe chronic comorbidity; alternative diagnosis	30-day mortality
Bont et al, 20088	Primary care, outpatients	Prospective consecutive cohort	n = 314; mean 77.3 years; male n = 145	One or more features of: new localising signs present during chest examination new infiltrates on chest radiograph strong suspicion because of severe dyspnoea in a very ill patient	One or more of: lung cancer haematologic malignant neoplasm HIV+ immunosuppressive medication use hospitalised during 2 weeks prior to diagnosis nursing home residents	30-day mortality
Buising et al, 200713	Emergency department	Prospective consecutive cohort	n = 722; median 73 years (range 18–98 years); male n = 411	New respiratory symptom + chest X-ray infiltrate	One or more of: age <18 years immunosuppression chronic lung disease nosocomial pneumonia >24 hours in emergency department	Death +/− requirement for ventilatory support
Capelastegui et al, 200614	Emergency department, substratified, inpatients, outpatients	Retrospective application of a prospective consecutive cohort	n = 1776; mean 61.8 years (standard deviation [SD] 20.5 years); male n = 1124	Pulmonary infiltrates on chest radiograph + symptoms consistent with pneumonia: cough dypsnoea fever pleuritic chest pain	One or more of: HIV chronic immunosuppression hospitalised within previous 14 days	30-day mortality
Chalmers et al, 200815	Hospitalised inpatients	Prospective observational cohort	n = 1007; mean 66 years (range 50–78 years); male n = 500	New infiltrate on chest radiograph >3 or more of: cough sputum production breathlessness pleuritic chest pain haemoptysis fever headache signs on chest auscultation	One or more of: hospital-acquired pneumonia active malignancy immunosuppression pulmonary embolism palliative care patients	30-day mortality
Ewig et al, 200922	CAP performance programme, German BQS	Consecutive retrospective cohort	n = 388 406	Age ≥18 years; ICD-10-GM-codes for pneumonia	Severe immunosuppression Acute bronchitis Exacerbations of COPD	In-hospital mortality
Kruger et al, 200816	CAPNETZ centres, inpatients, outpatients	Consecutive prospective cohort	n = 1404: 545 outpatients, 1001 inpatients; mean 61 years (range 18–98 year); male n = 772	Age >18 years, new pulmonary infiltrate on chest radiograph + one of: fever cough purulent sputum focal chest signs dyspnoea pleuritic pain	One or more of: systemic immune deficiency active TB hospitalisation <4 weeks prior to infection	28-day mortality
Man et al, 200717	Hospitalised inpatients admitted through emergency department	Consecutive prospective cohort	n = 1404; mean 72 years (range 17–103 years); male n = 583	Acute infection pulmonary parenchyma + symptoms of acute infection + acute infiltrate on chest radiograph	Age <17 years Immunosuppression Pulmonary TB Hospitalised <14 days prior to presentation Diagnosis other than CAP after admission	30-day mortality
Menendez et al, 200918	Hospitalised patients in two tertiary care centres	Consecutive prospective cohort	n = 453; mean 67 years (SD 17.1 years); male n = 282	New radiographic infiltrate; at least two days of compatible symptoms	Admitted in the previous 15 days Immunosupressive or steroid treatment Leudopenia or neutropenia Do not resuscitate	30-day mortality
Myint et al, 200623	Hospitalised inpatients	Retrospective analysis of two prospective observational cohorts	n = 192; median 77 years (range 17–96 years); male n = 111	Clinical features of pneumonia + new chest radiograph shadow; 2nd cohort age ≥65 years	Aspiration pneumonia Clinical diagnosis of CAP without new chest X-ray shadow TB Active malignancy	6-week mortality
Schaaf et al, 200719	Hospitalised inpatients	Consecutive prospective cohort	n= 105; mean 64.9 years (range 24–96 years); male n = 60	Isolation of S. pneumoniae or positive urinary antigen test + one or more of: fever respiratory symptoms auscultatory findings new infiltrate on chest radiograph	Neoplasia Immunosuppression HIV+ Immunoglobulin deficiency	40-day mortality
Schuetz et al, 200820	Emergency department	Pooled data of two randomised controlled studies	n = 372; median 73 years (interquartile range 59–82 years); male n = 244	Presence of a new infiltrate on chest radiograph + one of: cough sputum production dyspnoea fever >38.0°C auscultatory findings leucocytotis or leucopenia	Lower respiratory infections other than CAP: cystic fibrosis active pulmonary TB hospital-acquired pneumonia severe immunosuppression	30-day mortality
Zuberi and Khan, 200821	Hospitalised inpatients	Longitudinal cohort	n = 137; mean 60.4 years (range 16–95 years); male n = 74	Age ≥16 years + required admission for management of CAP	Pneumonia not primary cause for admission Post-obstructive pneumonia TB Bronchiectasis Malignancy HIV	30-day mortality

BQS = Bundesgeschäftsstelle Qualitatssicherung gGmbH. CAP = community-acquired pneumonia.

Figure 3.

Summary diagram of the quality assessment of the included studies.

Descriptive statistics of CRB-65 risk strata

Of the 397 875 patients included in the studies, 66 827 (17%) were classified as low risk (score = 0), 283 902 (71 %) intermediate risk (score = 1 or 2), and 47 146 (12%) high risk (score = 3 or 4) (Table 2). However, the percentage of patients in the three risk strata varied substantially depending on the setting of care, with community-based individuals being categorised into lower-risk categories. Observed events were much less frequent in community-based settings (Table 2).

Table 2.

Descriptive statistics for CRB-65 risk categories and observed risk for hospital and community-based subjects.

	Hospital based			Community based			Total
Risk category	n	%	Observed events, %	n	%	Observed events, %	n	%
Low	65 802	16.6	2.4	1025	54.4	0.0	66 827	16.8
Intermediate	283 137	71.5	13.3	765	43.6	1.6	283 902	71.4
High	47 119	11.9	34.3	27	1.9	18.5	47 146	11.8

Validation of CRB-65 risk strata

For lower-risk individuals, there are no events in community-based patients with consequent over-prediction; in low-risk hospital-based patients there is relative under-prediction, but the 95% CI around this estimate is compatible with accurate prediction (Figure 4). In intermediate-risk groups, over-prediction persists in community-based subjects, and observed 30-day mortality more closely matches predicted mortality for hospital-based patients, with a tighter CI (RR 0.91, 95% 95% CI 0.71 to 1.17) (Figure 5). For high-risk patients, the magnitude of over-prediction attenuates in community-based subjects, while observed mortality matches that predicted by CRB-65 in hospital-based patients (RR 1.01, 95% CI = 0.87 to 1.16) (Figure 6).

Figure 4.

Low-risk group (score = 0) predicted and observed deaths in community and hospital settings.

Figure 5.

Intermediate-risk group (score = 1 or 2) predicted and observed deaths in community and hospital settings.

Figure 6.

High-risk group (score = 3 or 4) predicted and observed deaths in community and hospital settings.

Sensitivity analysis

The study by Ewig et al has a significantly larger number of included participants than all others, and the results of that study dominate the meta-analysis of the hospital-based patients.²² Sensitivity analysis performed excluding this study shows similar results but with broader CIs, with a low-risk-group RR of 1.25 indicating over-prediction of CRB-65 but with a wider CI (0.60 to 2.59). The intermediate-risk group and high-risk group RR estimates remain similar, being RR 0.98, (95% CI = 0.79 to 1.22) and RR 1.03 (95% CI = 0.87 to 1.21) respectively.

DISCUSSION

Summary of main findings

This systematic review shows that application of CRB-65 accurately predicts 30-day mortality in hospitalised patients across three strata of risk. In community settings, CRB-65 appears to over-predict the probability of 30-day mortality across all strata of predicted risk (Figures 4–6).

Strengths and limitations of the study

Pooling of data across 14 separate validation studies enables assessment of the performance of CRB-65 in different clinical settings, addressing validity, generalisability, and precision of the estimates across the three different strata of risk. Several limitations of this study should be considered when interpreting the results. The low event rate, particularly in communitybased studies, makes precise estimates about CRB-65 performance less certain. Observed events (Table 2) are considerably lower than estimated in the original derivation study (Figure 1) on which clinical guidelines are based.^3,4 Individual patient data could not be obtained, so it was only possible to estimate predicted risk by applying the original risk estimates to the strata in each of the validation studies, an approach adopted in a community-based validation study.⁸ However, as CRB-65 relies on unequivocal clinical characteristics, it is unlikely that misclassification of patients occurred. The hospital-based patients category is dominated by one large study that contributed 98% of the total patient dataset;²² however, sensitivity analysis indicates that results are similar when this study is excluded from the meta-analysis. One study only measured patients who suffered from pneumococcal pneumonia,¹⁹ but rather than exclude this study, it was included albeit that the event rate is low and is likely to produce an under-prediction of risk. While more studies validating CRB-65 in primary care are needed, it is likely that a large number of patients will need to be recruited to enable precise estimates of performance to be made.⁹ The outcome of interest – 30-day mortality – is the final clinical endpoint but more frequently occurring intermediate outcomes might be more relevant to clinical care in a primary care setting, such as admission to hospital, clinical deterioration, or re-consultation. Alternatively, quality of life scores or duration of illness may be more indicative of morbidity associated with CAP, and are more likely to be outcomes of more immediate relevance to patients. Lastly, some studies assessed hospital mortality rather than 30-day mortality and this could account for differences in the performance of the rule between different studies.

Comparison with existing literature

CRB-65 is one of several clinical prediction rules that assess the severity of CAP. Other clinical prediction rules that address complications of CAP include the Pneumonia Severity Index (PSI) and CURB-65. The PSI was the first clinical prediction rule derived and validated as a prognostic indicator for CAP; however, its use of 20 different clinical variables makes calculation complex.²⁴ In addition, the inclusion of laboratory values limits its use in primary care settings, since many GPs wil not have immediate access to such investigations. CURB-65 includes the measurement of urea, and this also limits is timeliness and value in primary care.²⁵ The PSI has been compared to the CURB-65 rule in a number of studies,^{17,29,26–29} and the CPRs have been found to perform similarly.²⁵ These preliminary results indicate that the greater number of variables in the PSI does not significantly strengthen the power of the rule, and therefore their inclusion may only stand as a hindrance to its use.

A number of other clinical prediction rules have been derived with promising initial results, including SCAP,²⁸ SOAR,²³ and CORB,¹³ along with one clinical prediction rule derived entirely in primary care.^30,31 The hospital-derived CPRs all contain some or all of confusion, tachypnoea, hypotension, and age in their scoring, and therefore have similarities to CRB-65; however, these rules differ by including other criteria (such as oxygenation) and by the outcomes they predict: development of severe CAP,²⁸ requirement of ventilatory or inotropic support,¹³ or death.^13,23 The primary care-derived clinical prediction rule has different criteria to CRB-65 (diagnosis, age, comorbid conditions, medications, and recent hospitalisation), and the additional outcome measure of hospitalisation along with death. Although preliminary results are promising, these clinical prediction rules have yet to be validated to a great extent, and therefore their use in clinical practice is more uncertain.⁹ The use of generic sepsis and early-warning scores has also been proposed but preliminary results indicate that they are less effective than pneumonia-specific rules.¹¹

In terms of comparative prognostic value when comparing CURB-65 with CRB-65, it appears that additional urea measurement does not substantially improve the predictive value of CURB-65.^{12–15,17,20,23} This, along with the simplicity of CRB-65, indicates that it is emerging as potentially the most useful severity score for CAP.

Implications for future research and clinical practice

The inclusion and exclusion criteria for this meta-analysis were broad and reflect those used in the validation studies themselves but also do prove to be a limitation in the analysis. Significant common comorbid illnesses are known to have an impact on severity;^2,31–34 their impact was not assessed in the validation studies. Other known predictors of severity such as nursing home residency, recent hospitalisations, and recent antibiotics were also not addressed.^2,31,32,34 However, the derivation cohort for CRB-65 specifically included patients with these predictors, and the clinical prediction rule was developed adjusting for these prognostic markers. If included separately in a severity assessment alongside CRB-65, these factors may improve prediction; however, their presence in both the derivation and validation cohorts indicates that the components of CRB-65 have independent predictive characteristics.

Another additional way in which prediction may be improved is with the use of near-patient tests such as C-reactive protein or procalcitonin. These tests have been shown to be an adjunct in the diagnosis of CAP, and preliminary results indicate they may also have predictive value for severity.^16,18,35,36 A role for these tests may be found as an additional predictor of severity alongside CRB-65, but it is more likely they could be used as a further discriminator for the appropriate site of care after application of CRB-65.

In the context of clinical management strategies, over half of patients in community settings were classified as low risk, with over 40% classified as intermediate risk (Table 2). Within this group, CRB-65 was found to over-predict the risk of death. However, the event rate level is low and the extent of over-prediction by CRB-65 remains uncertain. It is possible that investigation of other clinical predictors or near-patient tests may improve prediction, but this requires further study. The addition of other risk factors such as comorbidity would also enable different thresholds to be set in community settings, reflecting the lower absolute number of events compared with hospital-based settings (Table 2).

In hospital-based patients, the severity scores were more evenly distributed, with two-thirds in the intermediate-risk group and one-fifth in the high-risk group. As a prognostic tool, CRB-65 appears to be more clinically useful in hospital settings. As clinical medicine is being driven by evidence-based standards of care,³⁷ prognostic rules are likely to become the benchmark against which acceptable standards of care are judged. In this context, as this systematic review shows, it is important that caution is used when extrapolating evidence from studies that have recruited from hospital-based rather than community-based settings. Accurate risk stratification of likely prognosis requires confirmatory validation in similar populations of patients and settings of care.⁷

Finally, future work with the CRB-65 rule will allow for more robust determination of its usefulness in both hospital and community-based practice. Assessment of the impact of the rule on antibiotic prescribing, especialy in the context of a low-risk patient, will be worthwhile. Also, determining an appropriate threshold for referral from primary care to hospital may be established, since the community-based results indicate that the rule may over-predict mortality in this setting, particularly in the low- and intermediate-risk groups (Figures 4 and 5).

In conclusion, CRB-65 has been found to accurately predict severity of pneumonia and resultant deaths in a hospital setting across al strata of risk. CRB-65 has not been validated sufficiently in primary care settings and preliminary findings suggest over-prediction, so its value as a prognostic indicator in the community remains uncertain.

Appendix 1. Statistical tests for lack of ‘small-study effects’ or publication bias (funnel plot results)

Egger's test for small-study effects: regress standard normal deviate of intervention effect estimate against its standard error
• Number of studies = 15
• Root MSE = 2.029
Std_Eff |	Coefficient	Standard error	t	P>|t|	95% CI
slope |	0.259	0.271	0.950	0.357	−0.327 to 0.844
bias |	−1.549	1.277	−1.210	0.247	−4.309 to 1.211
Begg's test for small-study effects: rank correlation between standardised intervention effect and its standard error
• adjusted Kendall's Score (P − Q) = −17
• Standard deviation of score = 20.21
• Number of studies = 15
• z = 0.79 (continuity corrected)
• Pr > |z| = 0.428 (continuity corrected)
Peter's test for small-study effects: regress intervention effect estimate on 1/Ntot, with weights S × F/Ntot
• Number of studies = 15
• Root MSE = 0.937
Std_Eff |	Coefficient	Standard error	t	P>|t|	95% CI
bias |	78.904	349.655	0.230	0.825	−676.479 to 834.287
constant |	−0.440	0.359	−1.230	0.242	−1.215 to 0.336
Harbord's modified test for small-study effects: regress Z/√(V) on √(V) where Z is efficient score and V is score variance
• Number of studies = 15
• Root MSE = 2.225
Z√(V)	Coefficient	Standard error	t	P>|t|	95% CI
√(V)	0.293	0.331	0.890	0.392	−0.422 to 1.008
bias |	−1.871	1.595	−1.170	0.262	−5.317 to 1.576

MSE = mean squared error.

Appendix 1.

Funnel plot with 95% confidence limits for existence of publication bias in the meta-analysis.

Funding body

Maggie McNally was supported as an RCSI Research Studentship, Kirsty O'Brien and Borislav D Dimitrov are supported by the HRB Centre for Primary Care Research under grant HRC/2007/1. The funders had no input into the study design; in the collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit the article for publication.

Competing interests

The authors have stated that there are none.

Discuss this article

Contribute and read comments about this article on the Discussion Forum: http://www.rcgp.org.uk/bjgp-discuss