Abstract
PURPOSE
To reduce inappropriate antibiotic prescribing, we sought to develop a clinical decision rule for the diagnosis of acute rhinosinusitis and acute bacterial rhinosinusitis.
METHODS
Multivariate analysis and classification and regression tree (CART) analysis were used to develop clinical decision rules for the diagnosis of acute rhinosinusitis, defined using 3 different reference standards (purulent antral puncture fluid or abnormal finding on a computed tomographic (CT) scan; for acute bacterial rhinosinusitis, we used a positive bacterial culture of antral fluid). Signs, symptoms, C-reactive protein (CRP), and reference standard tests were prospectively recorded in 175 Danish patients aged 18 to 65 years seeking care for suspected acute rhinosinusitis. For each reference standard, we developed 2 clinical decision rules: a point score based on a logistic regression model and an algorithm based on a CART model. We identified low-, moderate-, and high-risk groups for acute rhinosinusitis or acute bacterial rhinosinusitis for each clinical decision rule.
RESULTS
The point scores each had between 5 and 6 predictors, and an area under the receiver operating characteristic curve (AUROCC) between 0.721 and 0.767. For positive bacterial culture as the reference standard, low-, moderate-, and high-risk groups had a 16%, 49%, and 73% likelihood of acute bacterial rhinosinusitis, respectively. CART models had an AUROCC ranging from 0.783 to 0.827. For positive bacterial culture as the reference standard, low-, moderate-, and high-risk groups had a likelihood of acute bacterial rhinosinusitis of 6%, 31%, and 59% respectively.
CONCLUSIONS
We have developed a series of clinical decision rules integrating signs, symptoms, and CRP to diagnose acute rhinosinusitis and acute bacterial rhinosinusitis with good accuracy. They now require prospective validation and an assessment of their effect on clinical and process outcomes.
Keywords: sinusitis, rhinosinusitis, primary care, respiratory tract infections, clinical decision making, clinical decision rule, point score
INTRODUCTION
Practice guidelines recommend the use of antibiotics only for patients with prolonged, severe, or worsening symptoms of acute rhinosinusitis, when the likelihood of a bacterial cause is thought to be higher.1,2 It is common practice, however, for patients with a diagnosis of acute rhinosinusitis to be prescribed an antibiotic regardless of the duration of symptoms or their severity.3 One strategy to reduce inappropriate prescribing is to give physicians tools that can help them more confidently diagnose or rule out acute bacterial rhinosinusitis.
Previous studies have shown that individual signs and symptoms are of limited value for the diagnosis of acute bacterial rhinosinusitis.4,5 Point-of-care tests, such as C-reactive protein (CRP) and erythrocyte sedimentation rate (ESR) are promising, but by themselves they are not adequate to diagnose or rule out acute bacterial rhinosinusitis.6,7 Clinical decision rules have been proposed but have not been prospectively validated.7–11 Also, many previous studies of individual tests or clinical decision rules have used radiography or computed tomographic (CT) scans as the reference standard, tests that themselves have limited accuracy.12 Although a CT scan is highly sensitive for the detection of fluid in the sinuses, this fluid may also be caused by a viral infection, so the test lacks specificity and is, therefore, a suboptimal reference standard. Antral puncture can detect purulent secretions, which are more strongly associated with bacterial infection. Bacterial culture of these secretions is most specific for diagnosis of acute bacterial rhinosinusitis, but it may be less sensitive, because bacteria (even if present in the sinus) may not always grow in vitro.
Hansen and colleagues performed a study of 175 adults with suspected acute maxillary sinusitis, each of whom had a CT scan.6,13–15 If fluid was seen, antral puncture was performed to confirm the presence of purulent secretions, and the fluid was cultured for the presence of pathogenic bacteria. Thus, abnormal CT finding, the presence of purulent fluid, or a positive bacterial culture could each be used as a reference standard. In the current study, we will use these data to develop new clinical decision rules using both logistic regression and classification and regression tree (CART) approaches for the diagnosis of acute rhinosinusitis and acute bacterial rhinosinusitis using each reference standard.
METHODS
General Approach
We used 2 different approaches to develop clinical decision rules to diagnose acute rhinosinusitis or acute bacterial rhinosinusitis: point scores based on a logistic regression models, and algorithms based on classification and regression trees (CARTs). A point score and CART algorithm were developed for each of the 3 reference standards available to us: (1) abnormal finding on a CT scan, (2) the presence of purulent or mucopurulent fluid from an antral puncture of the maxillary sinus to diagnose acute rhinosinusitis, and (3) positive bacterial culture of antral puncture fluid to diagnose acute bacterial rhinosinusitis. For the remainder of the article, we will refer to these reference standards as abnormal CT finding, abnormal antral puncture finding, and positive bacterial culture. For each of the 6 resulting clinical decision rules, we identified groups at low, moderate, and high risk for the diagnosis of acute rhinosinusitis or acute bacterial rhinosinusitis.
Data Collection
The original data collection procedures have been described in several previous publications.6,13,14 In all, 282 patients were eligible for the study. Of these, 77 patients were excluded, mainly because they declined participation. After inclusion, an additional 31 patients changed their mind and withdrew. These patients showed no significant differences from the patients who completed the study with regard to sex, age, symptoms, signs, and laboratory tests. Ultimately, 175 adults between the ages of 18 and 65 years visited 1 of 8 general practices in Denmark. Patients were included if they were suspected of having acute maxillary sinusitis. Data on nonparticipants were not available, but it was the impression of the principal investigator (J.G.H.) that most patients approached agreed to participate. After a structured assessment of demographics, signs, and symptoms, blood was drawn for ESR and CRP testing. All patients then underwent a CT scan of the sinuses within 24 hours. If there was any evidence of mucosal thickening or fluid on the CT scan (n = 120), antral puncture was performed. An attempt was made to aspirate fluid, and all patients also underwent lavage with sterile saline. Any patient with aspirated or lavage fluid that was judged to be purulent or mucopurulent was classified as having acute rhinosinusitis (n = 91). This fluid was cultured, and any patient with a positive culture for a suspected bacterial pathogen was classified as having acute bacterial rhinosinusitis (n = 61). Variables with 3 values (eg, none, unilateral, or bilateral) were recoded to be dichotomous (eg, unilateral vs none or bilateral).
Point Scores Based on Logistic Regression
The most common approach to the development of clinical decision rules is to perform a logistic regression, and then assign points based on the β-coefficients of the final model.16 Patients with a greater number of points are more likely to have the diagnosis of interest, and scores are often stratified into low-, moderate-, and high-risk groups that correspond to the decisions to rule out, gather more information about, or rule in the diagnosis. An advantage of this approach is that the score is generally easy to use and has good face validity. For the current study, we first performed a univariate logistic regression analysis to identify signs, symptoms, and blood tests associated with an abnormal antral puncture, defined as an odds ratio of >1.5 or <0.5. We then assessed the predictor variables for multicollinearity.
We developed 3 logistic regression models, 1 for each of the 3 reference standards (abnormal CT finding, abnormal antral puncture finding, or positive bacterial culture). The reference standard was the dependent variable, and signs, symptoms, and CRP were the independent variables. We performed a stepwise model selection guided by the Akaike information criterion,17 with variables added until the Akaike information criterion increased significantly. A point score was developed based on the values of the β-coefficients, identifying the variable with the lowest β-coefficient, assigning it 1 point, and assigning points to other variables based on multiples of that β-coefficient. Low-, moderate-, and high-risk groups were created based on visual inspection of the point score distribution. All analyses were performed using Stata 13.3 (StataCorp LP).
CART Models
CARTs18 have previously been applied to a range of diagnostic and prognostic problems in medicine.19–22 Briefly, the CART algorithm identifies the predictor variable that best discriminates between patients with and without the outcome of interest. This process is repeated for each of the resulting groups, until a pre-specified minimum group size is reached or the process is halted by the investigator. The result is a tree: each terminus of the tree is sometimes called a leaf, and each leaf has a probability of the outcome of interest (for example, a 3% probability of acute rhinosinusitis or a 72% probability of acute rhinosinusitis). Leaves with similar probabilities of disease can be grouped to form low-, moderate-, and high-risk groups.
In our analysis, we used the classification tree procedure of SAS JMP 12.1 (SAS Institute Inc). We specified a minimum leaf size of 15 persons to avoid extremely unstable estimates with a wide confidence interval. We developed 3 CART models, 1 for each of the reference standards. Predictor variables included all dichotomous sign and symptom variables, as well as CRP as a continuous variable. The leaves of the resulting CART models were examined, and low-, moderate-, and high-risk groups were created for each model.
Assessment of Model Performance
For each of the models, the overall ability to discriminate between patients with and without acute rhinosinusitis (using the reference standards of abnormal CT finding or abnormal antral puncture finding) or acute bacterial rhinosinusitis (using the reference standard of a positive bacterial culture) was evaluated using the area under the receiver operating characteristic curve (AUROCC). Calibration was assessed using calibration curves for the point scores, and the Hosmer-Lemeshow statistic was calculated. The ability of each model to correctly classify patients as low, moderate, or high risk was evaluated based on the probability of acute rhinosinusitis or acute bacterial rhinosinusitis in each risk group and the number of patients classified in each group. Because of the small size of our data set, we did not attempt internal validation using a split sample.
RESULTS
Point Scores Based on Logistic Regression
The results of a univariate logistic regression analysis using antral puncture finding as the reference standard are summarized in Table 1. The strongest individual predictors of acute rhinosinusitis and acute bacterial rhinosinusitis are the CRP and the ESR. The following signs and symptoms had an adjusted odds ratio of >1.5 or <0.5 for the diagnosis of acute rhinosinusitis as defined by the presence of purulent or mucopurulent antral puncture fluid: preceding upper respiratory tract infection, previous diagnosis of sinusitis, maxillary toothache, unilateral maxillary pain, purulent nasal discharge, and any or unilateral tenderness of the maxillary sinus.
Table 1.
Univariate Logistic Regression of the Association Between Signs, Symptoms, C-Reactive Protein (CRP), and Erythrocyte Sedimentation Rate (ESR) With Antral Puncture Revealing Purulent or Mucopurulent Fluid
| Finding | OR (95% CI) | P Value |
|---|---|---|
| Symptoms | ||
| Preceding upper respiratory tract infectiona,b | 2.09 (0.90–4.86) | .088 |
| Maxillary toothachea | 1.99 (1.06–3.72) | .031 |
| Maxillary pain | ||
| Any | 0.42 (0.08–2.22) | .307 |
| Unilaterala,b | 1.66 (0.91–3.03) | .099 |
| Bilateral | 0.53 (0.29–0.97) | .040 |
| Cacosmia | 1.37 (0.75–2.49) | .309 |
| Anosmiab | 1.23 (0.67–2.27) | .500 |
| Cough | 1.23 (0.66–2.30) | .516 |
| Nasal congestion | 1.03 (0.50–2.14) | .929 |
| Pain bending forward | 0.86 (0.43–1.73) | .681 |
| Previous diagnosis of sinusitis | 0.43 (0.22–0.84) | .014 |
| Signs | ||
| Purulent nasal dischargea,b | 1.52 (0.77–2.99) | .226 |
| Tenderness of maxillary sinus | ||
| Anya,b | 1.93 (0.97–3.85) | .063 |
| Unilaterala,b | 2.19 (0.18–4.08) | .013 |
| Bilateral | 0.76 (0.40–1.43) | .391 |
| Tender tapping on teeth | 1.30 (0.69–2.44) | .415 |
| Purulent pharyngeal discharge | 1.30 (0.61–2.73) | .497 |
| Swollen inflamed turbinate | 1.01 (0.54–1.91) | .966 |
| Edema over maxillary sinus | ||
| Any | 0.64 (0.34–1.20) | .165 |
| Unilateral | 0.78 (0.39–1.57) | .486 |
| Bilateral | 0.48 (0.15–1.49) | .203 |
| Laboratory tests | ||
| C-reactive protein | ||
| >10 mg/L | 4.29 (2.27–8.11) | <.001 |
| >15 mg/La,b | 4.75 (2.50–9.02) | <.001 |
| >20 mg/L | 3.92 (2.02–7.61) | <.001 |
| Erythrocyte sedimentation rate | ||
| >10 mm/h | 3.30 (1.77–6.15) | <.001 |
| >20 mm/h | 3.81 (1.92–7.53) | <.001 |
OR=odds ratio.
Note: This analysis was also performed for abnormal CT finding and positive bacterial cultures as the reference standard; data available on request from the author.
Included in initial models for abnormal finding on antral puncture and positive bacterial culture as the reference standard.
Included in the initial model for abnormal computed tomographic finding as the reference standard.
The same analysis was repeated for abnormal CT finding and positive bacterial culture as the reference standards, with generally similar results (not shown). For the model using positive bacterial culture as the reference standard, the same variables were selected for the initial model. For abnormal CT finding as the reference standard, the same variables were also selected for the initial model other than the addition of anosmia and omission of maxillary toothache. These variables were entered into the initial logistic regression model for each reference standard, and the final model was selected based on stepwise addition of predictor variables until there was a significant increase in the Akaikie information criterion. The final models, the proposed point scores, and the AUROCC for each model are shown in Table 2. Calibration curves for each model are shown in Supplemental Appendices 1 and 2 (available at http://www.annfammed.org/content/15/4/347/suppl/DC1/), and the receiver operating characteristic curves are shown in Figure 1A, 1B, and 1C.
Table 2.
Final Logistic Regression Models to Predict the Likelihood of Sinusitis as Defined by 3 Different Diagnostic Reference Standards
| Independent Variable | Abnormal CT Finding | Antral Puncture Finding | Positive Bacterial Culture | |||
|---|---|---|---|---|---|---|
|
| ||||||
| β-Coefficient | Score Point | β-Coefficient | Score Point | β-Coefficient | Score Point | |
| Preceding URTI | 0.598 | 2 | 0.461 | 1 | 0.415 | 1 |
| Preceding sinusitis | −0.824 | −2 | −0.828 | −2 | −0.621 | −1 |
| Tender maxillary sinusitis (unilateral) | 0.584 | 2 | 0.470 | 1 | 0.746 | 2 |
| Maxillary toothache | … | … | 0.636 | 1 | 0.741 | 2 |
| Purulent nasal discharge | … | … | … | … | 0.559 | 1 |
| Anosmia | 0.363 | 1 | … | … | … | … |
| CRP >15 mg/L | 1.602 | 4 | 1.467 | 3 | 0.754 | 2 |
| Constant | −0.277 | … | −1.087 | … | −1.936 | … |
| AUROCC | 0.767 | … | 0.748 | … | 0.721 | … |
| Hosmer-Lemeshow χ2 test | 4.74a | … | 10.88b | … | 8.22c | … |
AUROCC = area under the receiver operating characteristic curve; CRP = C-reactive protein; CT = computed tomography; URTI = upper respiratory tract infection.
P =.79.
P =.14.
P =.41.
Figure 1.
Receiver operating characteristic (ROC) curves for logistic regression models using (A) abnormal bacterial culture, (B) abnormal finding on computed tomography, and (C) antral puncture revealing purulent fluid as reference standards.
CART Models
Separate CART models were developed for each of the 3 reference standards. The probability of sinusitis in each leaf (terminal node) was calculated, and similar leaves were grouped together to create low-, moderate-, and high-risk groups. The AUROCC was higher than that for the logistic regression models, ranging from 0.731 to 0.795. The classification accuracy of each CART model is summarized in Table 3, and the CART algorithm for positive bacterial culture as the reference standard is shown in Figure 2. The CART algorithms for abnormal antral puncture and abnormal CT finding as the reference standards are shown in Supplemental Appendix 2.
Table 3.
Accuracy of Point Scores Based on Logistic Regression Models for the Diagnosis of Acute Rhinosinusitis and Acute Bacterial Rhinosinusitis Using 3 Different Reference Standards
| Reference Standard | Score Points | Sinusitis/Total No. (%) | Likelihood Ratio |
|---|---|---|---|
| Abnormal CT finding | |||
| Low risk | −2 to 1 | 17/43 (39.5) | 0.29 |
| Moderate risk | 2 to 4 | 32/50 (64.0) | 0.80 |
| High risk | 5 to 9 | 69/78 (88.5) | 3.40 |
| Total | 118/171 (69.0) | ||
| Classified as low or high risk | 121/171 (70.8) | ||
| Abnormal antral puncture finding | |||
| Low risk | −2 to 0 | 11/44 (25.0) | 0.30 |
| Moderate risk | 1 to 4 | 53/99 (53.5) | 1.10 |
| High risk | 5 to 6 | 23/23 (100.0) | 42.00 |
| Total | 87/166 (52.4) | ||
| Classified as low or high risk | 67/166 (40.4) | ||
| Positive bacterial culture | |||
| Low risk | −1 to 3 | 13/80 (16.3) | 0.35 |
| Moderate risk | 4 to 6 | 35/71 (49.3) | 1.80 |
| High risk | 7 to 8 | 11/15 (73.3) | 5.00 |
| Total | 59/166 (35.5) | ||
| Classified as low or high risk | 95/166 (57.2) | ||
CT=computed tomography.
Note: Number with sinusitis and total are different from values for full data set, as cases with missing data were omitted.
Figure 2.
Classification and regression tree model for positive bacterial culture as the reference standard.
Models Without Preceding Sinusitis as a Predictor
Patients indicating that they had a previous episode of sinusitis had a lower likelihood of acute rhinosinusitis or acute bacterial rhinosinusitis. As the likelihood of having sinusitis diagnosed may depend on the person’s age, their physician, or cultural factors, we developed models that did not include this predictor. They are summarized in Supplemental Appendix 1.
DISCUSSION
This study is the first to systematically develop clinical decision rules for acute rhinosinusitis using different reference standards and statistical approaches. We believe that the most appropriate reference standard is a positive bacterial culture of antral puncture fluid, and the point score using this reference standard successfully identified groups with a low (16%), moderate (49%), and high (73%) likelihood of acute bacterial rhinosinusitis. Following this rule will likely lead to more conservative use of antibiotics, which is consistent with current practice guidelines. For example, current practice is that 72% of patients with a clinical diagnosis of acute rhinosinusitis receive an antibiotic.3 Using our rule and assuming that all high-risk and one-half of intermediate-risk patients receive an antibiotic would reduce that practice to 34% (60 of 175) of patients. This percentage is roughly consistent with the estimate that only 27% of episodes of sinusitis should be treated with an antibiotic.3 Even treating all patients who are at high or intermediate risk (52%) would be an improvement over current practice.
The CART model (Figure 2) using bacterial culture as the reference standard provides a good alternative for clinicians wanting a more visual, algorithmic approach. It has an AUROCC similar to the point score, and classifies a similar number of patients as low or high risk (Table 4). Although the models using a CT scan as the reference standard are presented, we do not recommend them for clinical use, as they place relatively few patients in the low-risk group.
Table 4.
Performance of CART Models for Outcomes of Abnormal CT Finding, Abnormal Antral Puncture Fluid Finding, and Abnormal Bacterial Culture
| Reference Standard | Sinusitis/Total No. (%) | Likelihood Ratio |
|---|---|---|
| Abnormal CT finding | ||
| Low risk (−2 to 1) | 13/42 (31) | 0.20 |
| Moderate risk (2 to 4) | 42/60 (70) | 1.07 |
| High risk (5 to 9) | 65/73 (89) | 3.72 |
| Total | 120/175 (69) | |
| AUROCC | 0.795 | |
| Classified as low or high risk | 115/175 | |
| Abnormal antral puncture finding | ||
| Low risk (−2 to 0) | 13/56 (23) | 0.28 |
| Moderate risk (1 to 4) | 36/67 (54) | 1.07 |
| High risk (5 to 6) | 42/52 (81) | 3.88 |
| Total | 91/175 (52) | |
| AUROCC | 0.772 | |
| Classified as low or high risk | 108/175 | |
| Positive bacterial culture | ||
| Low risk (−1 to 3) | 13/77 (17) | 0.38 |
| Moderate risk (4 to 6) | 33/75 (44) | 1.47 |
| High risk (7 to 8) | 15/23 (65) | 3.50 |
| Total | 61/175 (35) | |
| AUROCC | 0.731 | |
| Classified as low or high risk | 100/175 |
AUROCC = area under the receiver operating characteristic curve; CART = classification and regression tree; CT = computed tomography.
Perhaps the greatest strength of this type of “sinus score” and the algorithm in Figure 2 is that almost one-half of patients were classified as low risk, allowing clinicians to rule out acute bacterial rhinosinusitis in these patients and treat them symptomatically without prescribing antibiotics. Clinicians may choose to offer antibiotics to patients at high risk. For patients at intermediate risk, they could gather additional data to inform their treatment decision, such as the duration and severity of illness, as well as signs and symptoms that are not part of the decision rule, for example, double-sickening or transilluminiation. This approach is consistent with the threshold model of decision making.23 Other strengths of our study include recruitment of patients with clinically suspected sinusitis in the primary care setting, rather than a referral setting, and the ability to compare models created using different reference standards. Ours is the only proposed clinical decision rule that uses bacterial culture as the reference standard, which is highly specific for acute bacterial rhinosinusitis and avoids overtreatment of viral rhinosinusitis. There was also good consistency across the prediction models in terms of the predictor variables, and good discrimination with AUROCC of at least 0.72 for all models.
Our study has several limitations. First, we did not have complete data regarding fever, and we did not collect data regarding several variables that other studies have found were associated with radiographic sinusitis (double-sickening, transillumination, and a longer duration of symptoms).1,4,9 Second, our findings are limited to adults, and children may complain of acute bacterial rhinosinusitis differently. Most importantly, prospective validation in a different population should be pursued, although doing so is challenging given the difficulty of obtaining sinus puncture fluid in a broad sample of patients with clinically suspected sinusitis. Ultimately, whereas there is some evidence that antibiotics are effective for acute rhinosinusitis, it would be most helpful to know which subgroups are most likely to benefit from antibiotics. One systematic review concluded that patients with purulent drainage visible on physical examination were more likely to benefit, a sign incorporated in the “sinus score.”24,25
Clinical decision rules have been developed and validated to help primary care clinicians more accurately diagnose a variety of acute infections, including streptococcal pharyngitis, pneumonia, and urinary tract infection.26–28 One benefit of a more accurate diagnosis of acute infections is the potential to reduce inappropriate antibiotic prescribing and better target antibiotics at patients most likely to benefit. A recent report29 identified acute rhinosinusitis as the most common reason for antibiotics in the ambulatory setting and confirmed that they are prescribed for more than 70% of patients.3 Studies have shown, however, that only about 30% of patients have a bacterial cause based on culture of sinus fluid.6,30
This “sinus score” incorporates signs, symptoms, and the CRP blood test. The latter is increasingly available at the point of care. Studies have shown that use of point-of-care CRP testing can reduce antibiotic prescribing for acute respiratory tract infections,31–33 and our clinical decision rule strengthens the case for making this test more broadly available in primary care, as well as for studying it in the care of other conditions. Point-of-care CRP testing is widely used in some European countries and is currently available in the United States in laboratories certified to perform moderate complexity tests (QuikRead go CRP, Orion Diagnostica Inc; Alere Afinion CRP, Alere Inc). It takes less than 5 minutes and has been shown to be acceptable and cost-effective when used in patients with acute cough.29 The end-user cost is approximately $3.50 per test (excluding control costs, equipment, or labor) and Medicare reimbursement is $7.10 using CPT code 86140. It may also be possible to incorporate hand-held ultrasound examination, which is relatively sensitive for sinus fluid,12 into a decision algorithm that also includes signs, symptoms, and CRP, perhaps limiting its use to patients with an intermediate risk of acute bacterial rhinosinusitis. Even if CRP testing is not available, however, the CART algorithm (Figure 2) can still be useful, as patients without unilateral maxillary sinus tenderness who also do not have maxillary toothache would be classified as low risk for acute bacterial rhinosinusitis regardless of the CRP test result.
In conclusion, we have developed a point score and a CART-derived algorithm for diagnosing acute bacterial rhinosinusitis. Use of such a score can help patients and physicians more confidently avoid antibiotics if patients are at low risk for a bacterial infection, reducing overuse of antibiotics.
Footnotes
Conflicts of interest: authors report none.
Supplementary materials: Available at http://www.AnnFamMed.org/content/15/4/347/suppl/DC1/.
References
- 1.Wald ER, Applegate KE, Bordley C, et al. ; American Academy of Pediatrics. Clinical practice guideline for the diagnosis and management of acute bacterial sinusitis in children aged 1 to 18 years. Pediatrics. 2013;132(1):e262–e280. [DOI] [PubMed] [Google Scholar]
- 2.Chow AW, Benninger MS, Brook I, et al. ; Infectious Diseases Society of America. IDSA clinical practice guideline for acute bacterial rhinosinusitis in children and adults. Clin Infect Dis. 2012;54(8):e72–e112. [DOI] [PubMed] [Google Scholar]
- 3.Fleming-Dutra KE, Hersh AL, Shapiro DJ, et al. Prevalence of Inappropriate Antibiotic Prescriptions Among US Ambulatory Care Visits, 2010–2011. JAMA. 2016;315(17):1864–1873. [DOI] [PubMed] [Google Scholar]
- 4.Williams JW, Jr, Simel DL. Does this patient have sinusitis? Diagnosing acute sinusitis by history and physical examination. JAMA. 1993;270(10):1242–1246. [PubMed] [Google Scholar]
- 5.Glas AS, Lijmer JG, Prins MH, Bonsel GJ, Bossuyt PM. The diagnostic odds ratio: a single indicator of test performance. J Clin Epidemiol. 2003;56(11):1129–1135. [DOI] [PubMed] [Google Scholar]
- 6.Hansen JG, Schmidt H, Rosborg J, Lund E. Predicting acute maxillary sinusitis in a general practice population. BMJ. 1995;311(6999): 233–236. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Lindbaek M, Hjortdahl P, Johnsen UL. Use of symptoms, signs, and blood tests to diagnose acute sinus infections in primary care: comparison with computed tomography. Fam Med. 1996;28(3):183–188. [PubMed] [Google Scholar]
- 8.Huang SW, Small PA., Jr Rapid diagnosis of bacterial sinusitis in patients using a simple test of nasal secretions. Allergy Asthma Proc. 2008;29(6):640–643. [DOI] [PubMed] [Google Scholar]
- 9.Williams JW, Jr, Simel DL, Roberts L, Samsa GP. Clinical evaluation for sinusitis. Making the diagnosis by history and physical examination. Ann Intern Med. 1992;117(9):705–710. [DOI] [PubMed] [Google Scholar]
- 10.van Duijn NP, Brouwer HJ, Lamberts H. Use of symptoms and signs to diagnose maxillary sinusitis in general practice: comparison with ultrasonography. BMJ. 1992;305(6855):684–687. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Berg O, Carenfelt C. Analysis of symptoms and clinical signs in the maxillary sinus empyema. Acta Otolaryngol. 1988;105(3–4):343–349. [DOI] [PubMed] [Google Scholar]
- 12.Ebell MH, Mckay B, Guilbault R, Ermias Y. Diagnosis of acute rhinosinusitis in primary care: a systematic review of test accuracy. Br J Gen Pract. 2016;66(650):e612–632. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Hansen JG, Lund E. The association between paranasal computerized tomography scans and symptoms and signs in a general practice population with acute maxillary sinusitis. APMIS. 2011;119(1):44–48. [DOI] [PubMed] [Google Scholar]
- 14.Hansen JG, Højbjerg T, Rosborg J. Symptoms and signs in culture-proven acute maxillary sinusitis in a general practice population. APMIS. 2009;117(10):724–729. [DOI] [PubMed] [Google Scholar]
- 15.Hansen JG. Acute rhinosinusitis (ARS). Diagnosis and treatment of adults in general practice. Dan Med J. 2014;61(2):B4801. [PubMed] [Google Scholar]
- 16.Steyerberg EW, Vickers AJ, Cook NR, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128–138. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Akaike H. A new look at the statistical model identification. IEEE Trans Auto Control. 1974;19(6):716–723. [Google Scholar]
- 18.Marshall RJ. The use of classification and regression trees in clinical epidemiology. J Clin Epidemiol. 2001;54(6):603–609. [DOI] [PubMed] [Google Scholar]
- 19.Ebell MH, Afonso AM, Geocadin RG; American Heart Association’s Get With the Guidelines-Resuscitation (formerly National Registry of Cardiopulmonary Resuscitation) Investigators. Prediction of survival to discharge following cardiopulmonary resuscitation using classification and regression trees. Crit Care Med. 2013;41(12):2688–2697. [DOI] [PubMed] [Google Scholar]
- 20.Afonso AM, Ebell MH, Gonzales R, Stein J, Genton B, Senn N. The use of classification and regression trees to predict the likelihood of seasonal influenza. Fam Pract. 2012;29(6):671–677. [DOI] [PubMed] [Google Scholar]
- 21.Steurer J, Held U, Spaar A, et al. A decision aid to rule out pneumonia and reduce unnecessary prescriptions of antibiotics in primary care patients with cough and fever. BMC Med. 2011;9:56. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Fonarow GC, Adams KF, Jr, Abraham WT, Yancy CW, Boscardin WJ; ADHERE Scientific Advisory Committee, Study Group, and Investigators. Risk stratification for in-hospital mortality in acutely decompensated heart failure: classification and regression tree analysis. JAMA. 2005;293(5):572–580. [DOI] [PubMed] [Google Scholar]
- 23.Pauker SG, Kassirer JP. The threshold approach to clinical decision making. N Engl J Med. 1980;302(20):1109–1117. [DOI] [PubMed] [Google Scholar]
- 24.Young J, De Sutter A, Merenstein D, et al. Antibiotics for adults with clinically diagnosed acute rhinosinusitis: a meta-analysis of individual patient data. Lancet. 2008;371(9616):908–914. [DOI] [PubMed] [Google Scholar]
- 25.Hansen JG, Schmidt H, Grinsted P. [Penicillin treatment of acute maxillary sinusitis in adults. A randomized, double-blind, placebo-controlled trial from general practice]. Ugeskr Laeger. 2000;162(40):5351–5353. [PubMed] [Google Scholar]
- 26.McIsaac WJ, Goel V, To T, Low DE. The validity of a sore throat score in family practice. CMAJ. 2000;163(7):811–815. [PMC free article] [PubMed] [Google Scholar]
- 27.Little P, Turner S, Rumsby K, et al. Validating the prediction of lower urinary tract infection in primary care: sensitivity and specificity of urinary dipsticks and clinical scores in women. Br J Gen Pract. 2010;60(576):495–500. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.van Vugt SF, Broekhuizen BD, Lammens C, et al. ; GRACE consortium. Use of serum C reactive protein and procalcitonin concentrations in addition to symptoms and signs to predict pneumonia in patients presenting to primary care with acute cough: diagnostic study. BMJ. 2013;346:f2450. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Oppong R, Jit M, Smith RD, et al. Cost-effectiveness of point-of-care C-reactive protein testing to inform antibiotic prescribing decisions. Br J Gen Pract. 2013;63(612):e465–e471. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.van Buchem L, Peeters M, Beaumont J, Knottnerus JA. Acute maxillary sinusitis in general practice: the relation between clinical picture and objective findings. Eur J Gen Pract. 1995;1(4):155–160. [Google Scholar]
- 31.Cals JW, Schot MJ, de Jong SA, Dinant GJ, Hopstaken RM. Point-of-care C-reactive protein testing and antibiotic prescribing for respiratory tract infections: a randomized controlled trial. Ann Fam Med. 2010;8(2):124–133. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Jakobsen KA, Melbye H, Kelly MJ, et al. Influence of CRP testing and clinical findings on antibiotic prescribing in adults presenting with acute cough in primary care. Scand J Prim Health Care. 2010;28(4):229–236. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Aabenhus R, Jensen JU, Jørgensen KJ, Hróbjartsson A, Bjerrum L. Biomarkers as point-of-care tests to guide prescription of antibiotics in patients with acute respiratory infections in primary care. Cochrane Database Syst Rev. 2014;11(11):CD010130. [DOI] [PubMed] [Google Scholar]