Abstract
Background
Numerous decision aids have been developed recently, but the value they add above that of the initial clinical assessment is not well known.
Aim
To quantify whethera formal decision aid for chronic obstructive pulmonary disease (COPD) adds diagnostic information, above the physician's clinical assessment.
Design and setting
Subanalysis of a diagnostic study in Dutch primary care.
Method
Sixty-five primary care physicians included 357 patients who attended for persistent cough and were not known to have COPD. The physicians estimated the probability of COPD after short history taking and physical examination. After this, the presence or absence of COPD was determined using results of extensive diagnostic work-up. The extent to which an 8-item decision aid for COPD, which included only symptoms and signs, added diagnostic value above the physician's estimation was quantified by the increase of the area under the receiver operating characteristic curve (ROC area), and the improvement in diagnostic risk classification across two classes: ‘low probability of COPD’ (<20%) and ‘possible COPD’ (≥20%).
Results
One hundred and four patients (29%) had COPD. Adding the decision aid to the clinical assessment increased the ROC area from 0.75 (95% confidence interval [CI] = 0.70 to 0.81) to 0.84 (95% CI = 0.80 to 0.89) (P<0.005), and improved the diagnostic risk classification of the patients, such that 35 fewer patients needed spirometry testing and eight fewer COPD cases were missed.
Conclusion
A short decision aid for COPD added diagnostic value to the physician's clinical assessment.
Keywords: COPD, cough, diagnosis, diagnostic accuracy, primary care
INTRODUCTION
The diagnostic process for chronic obstructive pulmonary disease (COPD) in primary care starts with history taking and physical examination, and if results remain suggestive of COPD, spirometry is performed to diagnose or exclude COPD. Diagnosing or excluding COPD, especially the early cases, is not straightforward, as there is no single gold standard' available.1,2 Spirometry is very important, but results can only be interpreted in combination with symptoms and signs. There is substantial underdiagnosis of COPD in primary care,3,4 and awareness of possible COPD in symptomatic patients is recommended.1 However to perform spirometry in all patients with respiratory complaints is neither efficient nor feasible.2 Recently, the authors developed a decision aid for COPD, using only symptoms and signs, to assist physicians in estimating the probability of COPD and enhance efficient use of spirometry. The aid, developed from the data of 400 patients consulting their primary care physician with persistent cough, showed good ability to exclude COPD (negative predictive value >90%) (BDL Broekhuizen et al, unpublished data, 2010).
However, when a patient presents with a general symptom like persistent cough, the diagnostic process in practice usually does not start with obtaining the result of a formal decision aid. Rather, physicians first explore the problem using intuitive short history taking and physical examination. This results in an initial assessment of the probability of a possible underlying disorder After this, a decision aid is more likely to be used to obtain more diagnostic certainty. Therefore, to determine whether an 8-item decision aid for COPD is potentially useful for practice, its diagnostic accuracy should be determined in addition to the (initial) physician's assessment.
Despite the large number of decision aids published each year,5,6 little is known about the added value of decision aids above the physician's assessment. A few studies, of disorders other than COPD, compared the physician's assessment with a decision aid. In some the aid was superior,7–9 and in others the physician's assessment was.10,11 However the added value of the decision aid was not quantified.
The aim of the present analysis was to determine whether the recently developed 8-item decision aid for COPD added diagnostic value to the physician's clinical assessment, in middle-aged and older primary care patients presenting with persistent cough.
METHOD
Design and study population
This was a subanalysis of a large Dutch diagnostic study on COPD performed from 2006 to 2009. The study protocol has been described in detail elsewhere.4 Briefly, inclusion criteria for patients were being aged over 50 years and consulting their primary care physician for complaints of cough for 14 days or longer. Exclusion criteria were known COPD, suspected pneumonia, and terminal illness.
How this fits in
Numerous decision aids have been developed in recent decades. Decision aids for general complaints like cough are usually applied only after the clinical assessment of the physician. Whether decision aids have added diagnostic value to this clinical assessment was unknown. This study found that a short decision aid for chronic obstructive pulmonary disease (COPD) that included only symptoms and signs added relevant diagnostic value to the physician's clinical assessment, in 357 patients presenting with persistent cough. These results suggest that implementation of diagnostic decision tools in the diagnostic pathway for COPD could enhance efficient use of spirometry.
Physician's assessment and a decision aid for COPD
On the day the patient presented with persistent cough, the physician, who was the GP of the patient, recorded the results of short standardised history taking and physical examination, and, directly thereafter, their estimation of the probability that the patient had COPD.
All patients then underwent extensive further diagnostic work-up, including repeated spirometry by the GP, and secondary-care lung function testing (body plethysmography and carbon monoxide diffusing capacity measurement). Finally, an expert panel determined the presence or absence of COPD (reference test), according to international guidelines,1 using the results of all documented study tests, including the results of history taking and physical examination, except the GP-estimated probability of COPD. The expert panel also determined the severity of COPD according to the criteria of the Global Initiative for Chronic Obstructive Lung Disease (GOLD),1 and possible other diagnoses, like asthma.
Previously, the authors developed a decision aid for COPD including the independently contributing items of history taking and physical examination (BDL Broekhuizen et al, unpublished data, 2010). This aid was developed from the same study patients as included for the present analysis, using multivariable logistic regression analysis, with the expert panel diagnosis (COPD or no COPD) as diagnostic outcome. The final 8-item decision aid was internally validated by bootstrapping techniques to adjust the regression coefficients for overfitting and is presented in Box 1.
Box 1. Decision aid for COPD that was developed in 400 patients with persistent cough3
Risk of COPD presence = 1/(1 + exp−[−5.6 + 0.2 × every 5 years above age of 50 + 0.6 × male sex + 1.0 × current smoking + 0.7 × >20 pack years + 0.6 × cardiovascular disease + 0.6 × complaints of wheezing + 0.7 × diminished breath sounds])
Analysis
In the original diagnostic study, 400 patients with persistent cough were included. In the present analysis, patients were included if the physician's clinical assessment and the panel diagnosis were available, resulting in 357 participants, included (and assessed) by 65 GPs (Figure 1). The physician's estimation of the probability of COPD was entered as a continuous variable in a multivariable logistic regression analysis, with COPD as diagnostic outcome. The estimated probability for COPD by the previously developed decision aid was added as a continuous variable. To quantify the added discriminative value of the decision aid above the physician, the area under the receiver operating characteristic curve (ROC area) of the physician's estimate and of the combination of the physician's estimate and the decision aid, were compared.
Figure 1.
Flow chart of the study and participants.
Next, the extent to which the addition of the decision aid resulted in improvement in diagnostic risk classification across clinically relevant risk groups was quantified. In daily practice, the main aim of history taking and physical examination in cases of possible COPD, is to distinguish patients in whom COPD can safely be excluded (low estimated probability), from those who should undergo spirometry.1,2 Therefore, two predefined risk classes were used: ‘low probability of COPD’ (<20%) and ‘possible COPD’ (≥20 %). As there are no generally accepted thresholds for these risk classes, a relatively high threshold of 20% was chosen, because missed COPD is not life threatening, and could eventually be managed later, if symptoms persist. As a sensitivity analysis around the predefined risk threshold, this analysis was repeated using a threshold of 10% and 30%. Changes in diagnostic risk classification were expressed in number of spirometries needed and missed COPD cases. Analyses were performed using SPSS (version 17.0).
RESULTS
The mean age of the 357 participants was 63 years, 46% (n = 163) were male, and 28% (n = 102) were current smokers (Table 1). One hundred and four participants had COPD (prevalence 29%), of which 73 were mild (forced expiratory volume in 1 second [FEV1] >80% predicted), 28 moderate (FEV1 50-80% predicted), and three severe (FEV1 <50% predicted). Other diagnoses were asthma in 50 participants (in 16 participants this was known before the study), of which 19 had both COPD and asthma, four had heart failure, and one had lung cancer Characteristics of the analysed and excluded (n = 43) subjects did not substantially differ.
Table 1.
Demographic and clinical characteristics of the 357 study patients
| Characteristic | |
|---|---|
| Demographics (day 1) | |
| Mean age, years (SD) | 63 (9) |
| Male sex, % | 46 |
| History taking (day 1) | |
| Current smoker, % | 28 |
| Complaints of wheezing, % | 50 |
| Mean duration of cough, days (median) | 102 (40) |
| Physical examination (day 1) | |
| Pulmonary wheezing, % | 14 |
| Diminished breath sounds, % | 12 |
| Clinical view physician (day 1) | |
| Estimated probability of COPD, mean (SD) | 0. 41 (0.27) |
| Lung function tests (day 90) | |
| Post-dilator FEV1, % predicted mean (SD) | 100 (19) |
| Post-dilator FEV1/FVC, mean (SD) | 73 (10) |
| Panel diagnosis | |
| COPD present, % | 29 |
FEV1 = forced expiratory volume in 1 second. FVC = forced vital capacity. SD = standard deviation.
The ROC area of a regression model including only the physician's estimate was 0.75 (95% confidence interval [CI] = 0.70 to 0.81). Adding the decision aid significantly increased the ROC area (Figure 2) to 0.84 (95% CI = 0.80 to 0.89) (P<0.005).
Figure 2.
Receiver operating characteristic curves of the physician's assessment and the physician's assessment plus the decision aid's estimate combined.
Diagnostic classification by the physician is shown in Figure 3: of the 138 subjects estimated with low probability, 119 actually had no COPD, resulting in a negative predictive value (NPV) of low estimated probability of 86%. Of the 219 patients with a high (≥20%) estimated probability (possible COPD), 85 really had COPD, rendering a positive predictive value (PPV) of 39% for a high estimated probability. Diagnostic classification by the combination of the physician's assessment and the decision aid resulted in a NPV of 94% and a PPV of 51% for, respectively, low and high estimated probability, and implied that 35 fewer patients underwent spirometry (184 instead of 219) and eight fewer COPD cases were missed (11 instead of 19) (Figure 3). Using a threshold of 10% and 30% for low probability (to exclude COPD), resulted in, respectively, 37 fewer patients undergoing spirometry (253 instead of 290) and three fewer missed COPD cases (two instead of five), and 27 fewer patients undergoing spirometry (147 instead of 174) and nine fewer missed cases (20 instead of 29) (results not shown).
Figure 3.
Diagnostic risk stratification by physicians compared to physician's estimation and decision aid combined.
DISCUSSION
Summary
The diagnostic value of a aid including 8 items of history taking and physical examination in addition to the physician's assessment, was studied in patients presenting with persistent cough. The aid added diagnostic value (ROC area increased from 0.75 [95% CI = 0.70 to 0.81] to 0.84 [95% CI = 0.80 to 0.89]) and improved the diagnostic risk classification of the patients, such that 35 fewer patients needed spirometry testing and eight fewer COPD cases were missed.
Strengths and limitations
To appreciate the present results, a few potential strengths and weaknesses of the study should be addressed. First, a strength of the study was the valid measurement of the physician's initial clinical assessment, directly after short standard history taking and physical examination, without knowledge on spirometry results or the diagnostic probability estimated by the aid. Moreover, all patients underwent the reference test for COPD, irrespective of the physician's or aid's estimation, eliminating verification bias.
A limitation is that incorporation bias was possibly induced by presenting the results of history taking and physical examination to the expert panel to decide on the presence or absence of COPD (the reference test). These items were included in the reference test to prevent misclassification of COPD. Nevertheless, the effect of incorporation bias was probably limited because the expert panel decision was based on many more items, of which, for example, the lung functions tests probably had much more influence on the outcome of the reference test. A second limitation is that the applied decision aid had been internally validated, but not externally, that is, in new patients, which would have resulted in a more generalisable decision aid.12 Finally, in the present study, physicians were requested to report their estimated probability of the presence of COPD in their patients visiting with cough, while it is possible that they might otherwise not even have considered this diagnosis. Therefore, the physician's estimation in the study may have been more accurate than the initial diagnostic assessment of COPD in daily practice.
Comparison with existing literature
Others have studied the diagnostic value of clinical assessment for COPD,13–17 and reported sensitivities ranging from 50% to 64% and specificities from 64% to 93%. Considering an estimated probability of <20% as ‘COPD excluded’ in the present study, would result in a sensitivity of 82% and a specificity of 47%. However, the other studies did not include primary care patients, and COPD severity was mostly severe; therefore results are difficult to compare with the present ones. To the authors' knowledge, no studies have quantified the diagnostic value added to the physician's clinical assessment by a decision aid for COPD.
In this study, the clinical assessment remained independently associated with COPD after addition of the decision aid's estimate. Hence, the clinical assessment incorporated information not included in the variables of the decision aid. Other studies have also reported, for other disorders than COPD, that the clinical assessment was an independent diagnostic tool.18,19 This finding is sometimes referred to as a physician's ‘gut feeling’, which is obviously hard to define or to measure.20 Underlying factors of this gut feeling could be awareness of other risk factors for COPD, knowledge about prior respiratory complaints, or noticing other suggestive clinical findings during history taking and physical examination that were not registered in the present study.
Implications for practice and research
The implications of the added value of a decision aid — fewer spirometries needed and more detected cases (that is, fewer missed cases) of COPD — are relevant for daily practice. Cough is one of the most frequent complaints in primary care, which underlines the need for efficient use of spirometry. Spirometry is safe, but time consuming and costly, because it must be performed by trained professionals.
A debatable issue is the willingness of physicians to implement decision aids in daily practice.21,22 Most of the numerous decision aids or models published in recent decades, have never been implemented in daily practice. In the present study, results of the clinical assessment and the decision aid were combined, assuming that the aid was always used, and that reclassification of diagnostic risk was always applied by physicians to decide on whether or not to perform spirometry. However, physicians are usually more inclined to use a decision aid if they are not confident enough about the presence or absence of the target disorder after short history taking and physical examination.23 Moreover, according to the widely acknowledged cognitive mechanisms of ‘conservatism’ and ‘anchoring’, physicians who suspect (the absence) of a certain disorder, especially a severe disorder, are less willing to alter their diagnostic risk assessment, despite controversial new information, than those who do not have this strong suspicion.23–25 An implementation study could provide evidence on how clinical decision making is really altered by the use of a decision aid, in patients with possible COPD.22
Acknowledgments
We gratefully acknowledge all participating patients, GPs, and other healthcare professionals who contributed to the study.
Funding body
This work was funded by The Netherlands Organisation for Health Research and Development (ZonMW/NWO), grant number 945—04–015. The study sponsor had no involvement in the study design, the collection, analysis and interpretation of the data, the writing of the report or in the decision to submit the paper for publication.
Ethics committee
All participants gave written informed consent and the ethics committee of the University Medical Center Utrecht approved the study. The ethics committee of the University Medical Centre Utrecht approved the study (protocol number 05/142).
Provenance
Freely submitted; externally peer reviewed.
Competing interests
Alfred PE Sachs has been speaker on congresses, sponsored by GlaxoSmithKline, Boehringer-Ingelheim, Pfizer, or AstraZeneca. Theo JM Verheij participates in studies that have received grants from GlaxoSmithKline and Wyeth. All other authors have declared no competing interests.
Discuss this article
Contribute and read comments about this article on the Discussion Forum: http://www.rcgp.org.uk/bjgp-discuss
REFERENCES
- 1.Rabe KF, Hurd S, Anzueto A, et al. Global strategy for the diagnosis, management, and prevention of chronic obstructive pulmonary disease: GOLD executive summary. Am J Respir Crit Care Med. 2007;176(6):532–555. doi: 10.1164/rccm.200703-456SO. [DOI] [PubMed] [Google Scholar]
- 2.Levy ML, Quanjer PH, Booker R, et al. Diagnostic spirometry in primary care: Proposed standards for general practice compliant with American Thoracic Society and European Respiratory Society recommendations. Prim Care Respir J. 2009;18(3):130–147. doi: 10.4104/pcrj.2009.00054. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Bednarek M, Maciejewski J, Wozniak M, et al. Prevalence, severity and underdiagnosis of COPD in the primary care setting. Thorax. 2008;63(5):402–407. doi: 10.1136/thx.2007.085456. [DOI] [PubMed] [Google Scholar]
- 4.Broekhuizen BD, Sachs AP, Hoes AW, et al. Undetected chronic obstructive pulmonary disease and asthma in people over 50 years with persistent cough. Br J Gen Pract. 2010;60(576):489–494. doi: 10.3399/bjgp10X514738. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Royston P, Moons KG, Altman DG, Vergouwe Y. Prognosis and prognostic research: Developing a prognostic model. BMJ. 2009;338:b604. doi: 10.1136/bmj.b604. [DOI] [PubMed] [Google Scholar]
- 6.Laupacis A, Sekar N, Stiell IG. Clinical prediction rules. A review and suggested modifications of methodological standards. JAMA. 1997;277(6):488–494. [PubMed] [Google Scholar]
- 7.Palchak MJ, Holmes JF, Kuppermann N. Clinician judgment versus a decision rule for identifying children at risk of traumatic brain injury on computed tomography after blunt head trauma. Pediatr Emerg Care. 2009;25(2):61–65. doi: 10.1097/PEC.0b013e318196e9c0. [DOI] [PubMed] [Google Scholar]
- 8.Cooper J, Kapur N, Kway-Jones K. A comparison between clinicians' assessment and the Manchester Self-Harm Rule: a cohort study. Emerg Med J. 2007;24(10):720–721. doi: 10.1136/emj.2007.048983. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Perez RS, Burm PE, Zuurmond WW, et al. Physicians' assessments versus measured symptoms of complex regional pain syndrome type 1: presence and severity. Clin J Pain. 2005;21(3):272–276. doi: 10.1097/00002508-200505000-00010. [DOI] [PubMed] [Google Scholar]
- 10.Glas AS, Pijnenburg BA, Lijmer JG, et al. Comparison of diagnostic decision rules and structured data collection in assessment of acute ankle injury. CMAJ. 2002;166(6):727–733. [PMC free article] [PubMed] [Google Scholar]
- 11.Sinuff T, Adhikari NK, Cook DJ, et al. Mortality predictions in the intensive care unit: comparing physicians with scoring systems. Crit Care Med. 2006;34(3):878–885. doi: 10.1097/01.CCM.0000201881.58644.41. [DOI] [PubMed] [Google Scholar]
- 12.Justice AC, Covinsky KE, Berlin JA. Assessing the generalizability of prognostic information. Ann Intern Med. 1999;130(6):515–524. doi: 10.7326/0003-4819-130-6-199903160-00016. [DOI] [PubMed] [Google Scholar]
- 13.Garcia-Pachon E. Paradoxical movement of the lateral rib margin (Hoover sign) for detecting obstructive airway disease. Chest. 2002;122(2):651–655. doi: 10.1378/chest.122.2.651. [DOI] [PubMed] [Google Scholar]
- 14.Badgett RG, Tanaka DJ, Hunt DK, et al. The clinical evaluation for diagnosing obstructive airways disease in high-risk patients. Chest. 1994;106(5):1427–1431. doi: 10.1378/chest.106.5.1427. [DOI] [PubMed] [Google Scholar]
- 15.McAlister FA, Straus SE, Sackett DL. Why we need large, simple studies of the clinical examination: the problem and a proposed solution. CARE-COAD1 group. Clinical Assessment of the Reliability of the Examination-Chronic Obstructive Airways Disease Group. Lancet. 1999;354(9191):1721–1724. doi: 10.1016/s0140-6736(99)01174-5. [DOI] [PubMed] [Google Scholar]
- 16.Holleman DR, Jr, Simel DL, Goldberg JS. Diagnosis of obstructive airways disease from the clinical examination. J Gen Intern Med. 1993;8(2):63–68. doi: 10.1007/BF02599985. [DOI] [PubMed] [Google Scholar]
- 17.Fletcher CM. The clinical diagnosis of pulmonary emphysema; an experimental study. Proc R Soc Med. 1952;45(9):577–584. [PubMed] [Google Scholar]
- 18.Tierney WM, Fitzgerald J, McHenry R, et al. Physicians' estimates of the probability of myocardial infarction in emergency room patients with chest pain. Med Decis Making. 1986;6(1):12–17. doi: 10.1177/0272989X8600600103. [DOI] [PubMed] [Google Scholar]
- 19.Stolper E, Van RP, Van de WM, et al. Consensus on gut feelings in general practice. BMC Fam Pract. 2009;10:66. doi: 10.1186/1471-2296-10-66. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Greenhalgh T. Intuition and evidence-uneasy bedfellows? Br J Gen Pract. 2002;52(478):395–400. [PMC free article] [PubMed] [Google Scholar]
- 21.Moons KG, Altman DG, Vergouwe Y, Royston P. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ. 2009;338:b606. doi: 10.1136/bmj.b606. [DOI] [PubMed] [Google Scholar]
- 22.Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med. 2006;144(3):201–209. doi: 10.7326/0003-4819-144-3-200602070-00009. [DOI] [PubMed] [Google Scholar]
- 23.Friedman CP, Gatti GG, Franz TM, et al. Do physicians know when their diagnoses are correct? Implications for decision support and error reduction. J Gen Intern Med. 2005;20(4):334–339. doi: 10.1111/j.1525-1497.2005.30145.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Tversky A, Kahneman D. Judgment under uncertainty: heuristics and biases. Science. 1974;185(4157):1124–1131. doi: 10.1126/science.185.4157.1124. [DOI] [PubMed] [Google Scholar]
- 25.Elstein AS, Schwartz A. Clinical problem solving and diagnostic decision making: selective review of the cognitive literature. BMJ. 2002;324(7339):729–732. doi: 10.1136/bmj.324.7339.729. [DOI] [PMC free article] [PubMed] [Google Scholar]