Cognitive reflection and antibiotic prescribing for acute respiratory infections

Abstract

Background.

Variation in clinical decision-making could be explained by clinicians’ tendency to make ‘snap-decisions’ versus making more reflective decisions. One common clinical decision with unexplained variation is the prescription of antibiotics for acute respiratory infections (ARIs).

Objective.

We hypothesized that clinicians who tended toward greater cognitive reflection would be less likely to prescribe antibiotics for ARIs.

Methods.

The Cognitive Reflection Test (CRT) is a psychological test with three questions with intuitive but incorrect answers that respondents reach if they do not consider the question carefully. The CRT is scored from 0 to 3, representing the number of correct answers. A higher score indicates greater cognitive reflection. We administered the CRT to 187 clinicians in 50 primary care practices. From billing and electronic health record data, we calculated clinician-level antibiotic prescribing rates for ARIs in 3 categories: all ARIs, antibiotic-appropriate ARIs and non-antibiotic-appropriate ARIs.

Results.

A total of 57 clinicians (31%) scored 0 points on the CRT; 38 (20%) scored 1; 51 (27%) scored 2; and 41 (22%) scored 3. We found a roughly U-shaped association between cognitive reflection and antibiotic prescribing. The antibiotic prescribing rate for CRT scores of 0, 1, 2 and 3 for all ARIs (n = 37080 visits) was 32%, 26%, 25% and 30% (P = 0.10); for antibiotic-appropriate ARIs (n = 11220 visits) was 60%, 55%, 54% and 58% (P = 0.63); and for non-antibiotic-appropriate ARIs (n = 25860 visits) was 21%, 17%, 13% and 18%, respectively (P = 0.03).

Conclusions.

In contrast to our hypothesis, there appears to be a ‘sweet-spot’ of cognitive reflection for antibiotic prescribing for non-antibiotic-appropriate ARIs. Differences in clinicians’ cognitive reflection may be associated with other variations in care.

Introduction

Clinicians who are prone to automatic, unreflective thinking may make poor clinical decisions. One frequently criticized decision is the prescription of unnecessary antibiotics for acute respiratory infections (ARIs). A clinician who is less reflective may fail to consider the potential harms of unnecessary antibiotics—increased costs, adverse drug events and antibiotic-resistant bacteria—and prescribe antibiotics for the sake of expediency and patient demand (1,2).

The Cognitive Reflection Test (CRT) consists of three simple math problems, each of which suggests an intuitive but wrong answer that respondents tend to reach if they do not carefully audit their answer (i.e. answers that arise from automatic thought without further reflection) (3). Increasing CRT scores have been associated with increased patience, ability to delay gratification, academic achievement and cognitive ability (3). To our knowledge, performance on the CRT has not been compared to actual medical decision-making.

We conducted a cross-sectional survey and analysis of antibiotic prescribing among primary care clinicians. We hypothesized that clinicians with higher CRT scores—those tending towards greater cognitive reflection—would be less likely to prescribe antibiotics for ARIs.

Methods

As part of a larger study, we invited 245 clinicians from 50 primary care practices in Boston and Los Angeles by e-mail to complete a web-based survey in June 2013 and January 2014, respectively (4,5). The full survey, of which the CRT was a part, generally took between 20 and 30 minutes to complete. The rest of the survey included 32 questions about electronic health record use, performance measurement, the office environment and antibiotic prescribing in a variety of formats including Likert scales, dichotomous choices, multiple choice and brief open responses (like the CRT). For completing the survey, clinicians or their practices received $100.

The three questions of the CRT are considered correct or incorrect and the test is scored from 0 to 3 for the number of correct answers (3) (Appendix). A higher score indicates greater cognitive reflection. The CRT measures a construct related to rational thinking (6,7), distinct from intuitive thinking (8) or numeracy (9), and may have a physiological basis (10,11).

We used billing and electronic health record prescribing data from the larger study to identify ARI visits and antibiotic prescribing in the ~1 year prior to administering the CRT. We excluded ARI visits by patients with chronic lung disease, cancer, diabetes, immunosuppression or concomitant acute diagnoses (e.g. cellulitis) for which antibiotics would be indicated. We calculated clinician-level antibiotic prescribing rates for all ARIs, non-antibiotic-appropriate diagnoses and antibiotic-appropriate diagnoses. We considered acute bronchitis [international classification of diseases-9 (ICD-9) 466, 490], non-specific upper respiratory infection (ICD-9 460, 464, 465), influenza (ICD-9, 487) and non-streptococcal pharyngitis (ICD-9 462, 463) non-antibiotic-appropriate diagnoses. We considered streptococcal pharyngitis (ICD-9 034.0), otitis media (ICD-9 381, 382), sinusitis (ICD-9 461, 473) and pneumonia (ICD-9 481–486) antibiotic-appropriate diagnoses. If a visit had both types of diagnoses, we considered the visit to have an antibiotic-appropriate diagnosis. We use the term ‘antibiotic-appropriate diagnoses’, although antibiotics are not necessary for all visits with these diagnoses.

We used analysis of variance to compare respondents’ CRT scores with their antibiotic prescribing rates and pairwise t-tests to compare prescribing rates between CRT groups. We examined use of shrinkage estimators to account for varying within-clinician ARI visit numbers, but the results did not change substantively. We also examined confounding by clinician sociodemographic factors (including differences between physicians and mid-level clinicians) and the results did not change substantively. We did not examine confounding by patient characteristics as this was a study of clinician decisions that should be, especially given the visit exclusion criteria, independent of patient characteristics. We used Stata/MP (Version 13.1, College Station, TX) and considered P-values < 0.05 statistically significant. Institutional review boards approved the study protocol for each of the participating sites.

Results

The survey response rate was 76% (187/245). Among respondents, the mean age was 50 years; 64% were women; and 81% were physicians (19% were nurse practitioners or physician assistants). During the ~1 year prior to completing the CRT, the 187 respondents saw 37080 ARI visits (mean visits per clinician, 198; median visits per clinician, 127), including 25860 non-antibiotic-appropriate visits and 11220 antibiotic-appropriate visits.

The antibiotic prescribing rate was 29% for all ARIs, 57% for antibiotic-appropriate ARIs and 17% for non-antibiotic-appropriate ARIs. Increasing CRT scores were associated with a roughly U-shaped antibiotic-prescribing rate, which was only statistically significant for non-antibiotic-appropriate ARIs (Fig. 1).

Figure 1.

Antibiotic prescribing by CRT score. The 187 clinicians saw 37080 acute respiratory infection visits (mean visits per clinician, 198; median visits per clinician, 127); 25860 non-antibiotic-appropriate visits; and 11220 antibiotic-appropriate visits. P-values for the association between CRT score and antibiotic prescribing for antibiotic-appropriate diagnoses, all acute respiratory infection diagnoses and non-antibiotic-appropriate diagnoses were calculated by analysis of variance. P-values for each score compared to 0 were calculated by pairwise t-tests.

Conclusions

The CRT generally shows linear relationships (3). We hypothesized that clinicians who demonstrated less cognitive reflection would have higher inappropriate prescribing rates. However, we found a roughly U-shaped relationship between level of reflection and inappropriate prescribing. The finding was only statistically significant for non-antibiotic-appropriate diagnoses, but the same trend was apparent across all the results.

One explanation for our findings is that lower-scoring clinicians may tend to make unreflective antibiotic-prescribing decisions, acceding to the perceived desire of patients to receive antibiotics. In contrast, higher-scoring clinicians may tend to overthink patients’ presentation, alternative diagnoses or potential complications so that they rationalize why antibiotics might be indicated. In medical decision-making, reflective reasoning is useful, but has the potential to lead to loss of efficiency (12,13). For making diagnostic decisions, evidence is conflicting between the relative benefits and balance between intuitive or reflective clinical decisions (14,15).

Our study has several limitations. First, we had a small sample size, limiting power to detect differences between groups. Second, respondents may not be representative of the broader universe of primary care clinicians. Third, the CRT was part of a longer survey, which could influence clinicians’ responses on the CRT. Fourth, clinicians may actually be more or less reflective when delivering clinical care.

Future research will need to investigate the robustness and psychological mechanisms underlying this pattern of results. While tentative, our results suggest there may be a ‘sweet-spot’ of reflection for some clinical decisions. Differences in clinicians’ cognitive reflection may be associated with other variations in care.

Declarations

Funding: National Institutes of Health (RC4 AG039115).

Ethical approval: the University of Southern California Institutional Review Board, the Partners Human Research Committee and the Western Institutional Review Board approved the study protocol and the method for obtaining consent to participate. Clinicians provided electronic consent at the beginning of the web-based enrolment survey.

Conflict of interest: none.

Appendix

Please cover the answers and write down your response to each question before looking at the answers below.

Cognitive Reflection Test

• (1) A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?

• (2) If it takes 5 machines 5 minutes to make 5 widgets, how long would it take 100 machines to make 100 widgets?

• (3) In a lake, there is a patch of lily pads. Every day, the patch doubles in size. If it takes 48 days for the patch to cover the entire lake, how long would it take for the patch to cover half the lake?

CRT intuitive, wrong answers and correct answers

(1)	Intuitive, wrong answer, $0.10.	Correct answer, $0.05.
(2)	Intuitive, wrong answer, 100 minutes.	Correct answer, 5 minutes.
(3)	Intuitive, wrong answer. 24 days.	Correct answer, 47 days.