Confidence and Information Access in Clinical Decision-Making: An Examination of the Cognitive Processes that affect the Information-seeking Behavior of Physicians

Abstract

Clinical decision-making involves the interplay between cognitive processes and physicians’ perceptions of confidence in the context of their information-seeking behavior. The objectives of the study are: to examine how these concepts interact, to determine whether physician confidence, defined in relation to information need, affects clinical decision-making, and if information access improves decision accuracy. We analyzed previously collected data about resident physicians’ perceptions of information need from a study comparing abstracts and full-text articles in clinical decision accuracy. We found that there is a significant relation between confidence and accuracy (φ=0.164, p<0.01). We also found various differences in the alignment of confidence and accuracy, demonstrating the concepts of underconfidence and overconfidence across years of clinical experience. Access to online literature also has a significant effect on accuracy (p<0.001). These results highlight possible CDSS strategies to reduce medical errors.

Introduction

Physicians are faced with many clinical decisions of varying complexity in their everyday practice. Most of these decisions revolve around questions on the diagnosis and therapeutic management of patients. The frequency of clinical questions varies from 0.16 to 1.27 per patient¹, depending on specialty, and up to 5 questions in the inpatient setting². In the process of decision-making, physicians generally use their ‘accumulated clinical knowledge’, defined as a physician’s personal knowledge base accumulated through years of formal education, medical training, research and clinical experience,^{3, 4} to answer clinical questions. Accumulated clinical knowledge is traditionally used by physicians as it is the most convenient resource of information in the healthcare setting. This is especially highlighted in critical situations, such as the emergency department where there may be an immediate need for a diagnosis and management. However, reliance on this knowledge alone may lead to medical errors when these clinical questions remain unanswered or unsupported by more recent medical literature.

More physicians are integrating evidence-based medicine in their clinical practice. The conscientious, explicit and judicious use of current best evidence in clinical decision-making is the fundamental principle in the practice of evidence-based medicine (EBM)⁵. However, physicians are faced with numerous challenges that prevent the utilization of clinical decision support systems (CDSS), which are information systems designed to assist in clinical decisions by providing alerts, notifications, and links to important information personalized to patients, and the wider use of online evidence-based literature. Lack of time is seen to be one of major reasons that prevent physicians from accessing online literature to answer their clinical questions. In a previous study, 60% of physicians expressed that time was an issue in seeking information while in another study, 40% of clinical questions remained unresolved because of time constraints^{6, 7}.

Another hindrance that may be intrinsic to the medical profession is confidence in their own clinical knowledge. Physicians who do not perceive the need to access information when faced with a clinical question, may err in their clinical decision. In some cases, a physician may be unaware that there is a gap in their knowledge, thus making no attempt in remedying it⁸. Moreover, previous research demonstrated how some physicians did not even attempt to resolve clinical questions that they acknowledged might have been important in the management of their patients^9–11. Some researchers proposed that the sociology of being a medical professional, which holds physicians that have expert knowledge and clinical competence in high regard, might explain this behavior^{12, 13}. The act and perception of needing to seek knowledge may reflect negatively on a physician’s competence and professionalism^{14, 15}.

In examining the process of how physicians make clinical decisions, an understanding of the cognitive processes of physicians must be considered. Clinical reasoning, which is a necessary cognitive process used to evaluate and manage a patient’s medical problem¹⁶, is a major process that defines clinical competence. This competency allows physicians to arrive at their diagnoses and treatment plans¹⁷.

In our understanding of cognitive processes that complicate how physicians arrive at their decisions, we can see the role of confidence in clinical reasoning and information seeking behavior. Physicians who believe that their accumulated clinical knowledge is all they need to reach a correct decision will not be motivated to access external resources and may possibly be inflexible to change a planned management course when information in the EBM literature that may be contrary to their belief is presented¹⁸. Friedman, et al., who first attempted to examine the relationship between confidence and clinical diagnosis accuracy, proposed four states of concordance or discordance that depend on the alignment of confidence and correctness^{18, 19}.

In states of concordance, physicians who are confident about their decision are indeed proven correct in their patient management. Conversely, those who are diffident, or are not confident, and are incorrect are also in a state of concordance. Physicians who fall in either of these two states of concordance exhibit appropriate confidence and their beliefs align with reality. The ideal situation would be physicians who are appropriately confident, while those who are appropriately diffident are likely to be receptive to accessing external sources of information. In contrast, states of discordance can be understood as underconfidence and overconfidence. Physicians who are not confident but are correct in their clinical decisions are deemed underconfident. Physicians in this state are more likely to seek information to confirm their decisions, however, it is still possible that doing so may steer them away from their originally correct answer²⁰. The state of discordance that is of greatest concern is overconfidence. Physicians in this category are confident in their clinical decisions but in reality are incorrect and are therefore prone to committing medical errors. These physicians are less likely to pursue additional information that could correct their flawed decision.

Motivated by the complex interactions of confidence and decision-making in clinical reasoning, we aimed to determine whether there was a significant relationship between physicians’ confidence in their accumulated knowledge in answering clinical cases and the accuracy of these decisions. This study attempted to address the following questions:

1. Is there a relationship between residency year level and clinical decision accuracy?

2. Is there a relationship between confidence and clinical decision accuracy?

3. Does the relationship between confidence and clinical decision accuracy depend per specialty or residency year levels?

4. Does accessing online literature affect clinical decision accuracy?

5. Does the effect of accessing online literature in clinical decision accuracy depend on specialty and residency year level?

Methods

Previous study

With permission from the authors, we reexamined the data collected by Marcelo et al. in their study where they compared the effects of using full-text articles against journal abstracts alone in clinical decision accuracy³. In that study, an attending physician from four different specialty departments (Family Medicine, Internal Medicine, Emergency Medicine, and Surgery) each prepared five clinical scenarios, specific to their specialty, and with varying complexities. The clinical cases were based on a PubMed search of recent journal articles that contained relevant clinical scenarios with their appropriate diagnostic and treatment recommendations. Resident physicians from each department answered five written clinical cases prepared by their respective attending physicians without access to online resources. After answering the case simulations, each resident was asked to indicate the clinical cases they thought could be accurately answered by accumulated knowledge alone and if they considered that a literature search was needed to arrive at the correct answer. After answering the written cases, each resident answered an online version of the same set of clinical cases, but was given the option to access relevant information either through journal abstracts or full-text articles. The attending physicians who prepared the cases evaluated the accuracy of each resident’s answers in both the written (pre-intervention) and online (post-intervention) versions of the clinical cases.

Study design

In this study, we looked at the data on the resident physicians’ perceptions on their need for information and their accuracy in answering clinical cases. In our analysis, the participants were additionally stratified based on specialty and year level. We examined the relationship between the residents’ confidence and clinical decision accuracy. The data reviewed consisted of 322 cases. Excluded from this review were items where the resident did not indicate their initial perception (n=63).

Responses wherein the resident indicated that their accumulated knowledge was adequate to make a correct clinical decision were categorized to be confident. Responses where the residents expressed the need for accessing online PubMed sources were considered to be diffident. The residents’ judgments of their confidence were completed without feedback on whether they were correct or incorrect. Clinical decision accuracy of each case was coded as correct or incorrect.

Data Analysis

In this study, each case paired a resident’s subjective assessment of their confidence based on their need for an online literature search with an objective evaluation of their clinical decision accuracy based on correctness on the clinical cases. Chi-square analysis was done to determine if there is a relationship between confidence and clinical decision accuracy across all cases, and stratified based on specialty and residency year level. Cramer’s V (V), Phi (φ) and Kendall’s Tau-c (τ_c) values were used as symmetric measures to examine the strength of the association when the chi-square values were found to be statistically significant. McNemar’s test was done to determine marginal homogeneity in matched pairs that examines the effect of online literature access by comparing clinical decision accuracy at pre- and post-intervention stages. Statistical analyses were performed using IBM SPSS Statistics Software Version 22 software.

Results

Irrespective of physician confidence, we found a small but significant relationship between residency year level and clinical decision accuracy (χ2=8.077, df=3, p<0.05, τ_c=0.111). First year residents had more incorrect answers (63%) than their more experienced colleagues (45%, 53%, and 50% of 2^nd, 3^rd, and 4^th year residents, respectively). In 228 of 322 cases (71%), residents expressed the need for a literature search to arrive at the correct clinical decision.

We found a small but statistically significant relationship between confidence and clinical decision accuracy in the pre-intervention scores across all cases (χ2=8.697, df=1, p<0.01, φ=0.164,). Resident physicians were more incorrect (55%) than correct (45%), and more diffident (71%) than confident (29%) regardless of confidence. In total, 60% of the cases demonstrated appropriate confidence or diffidence where the residents’ perceived need for information access was aligned with the accuracy of their answers. Calculating the risk ratios of physician confidence and clinical decision accuracy, a diffident physician was 1.42 times more likely to be incorrect than a confident physician. Resident physicians were more often underconfident (28%) than overconfident (12%). Residents were also more often appropriately diffident (43%) than they were appropriately confident (17%). (Table 1) shows a summary of findings.

Table 1.

Contingency table on the Alignment of Confidence and Clinical Decision Accuracy

Physician Confidence	Clinical Decision		Total
	Correct	Incorrect
Confident	17% Appropriately Confident	12% Overconfident	29% Confident
Diffident	28% Underconfident	43% Appropriately Diffident	71% Diffident
Total	45% Correct	55% Incorrect	100%

A separate analysis stratifying cases in their respective specialties found no statistically significant relationship between confidence and clinical decision accuracy.

When stratified based on residency year level, physician confidence had a small effect on clinical decision accuracy among first years (χ2=4.97, df=1, p<0.05, V=0.199) and a medium effect among fourth years (χ2=4.196, df=1, P<0.05, V=0.418). We could see a trend of increasing confidence as resident physicians progress in year levels. First year residents were the least confident (23%) compared to fourth year residents who were confident (46%) on the adequacy of their accumulated knowledge in answering the clinical cases.

Fourth year residents were more appropriately confident (33%) than first year residents (13%). First year residents were more appropriately diffident (52%) than their more senior residents (32%, 40%, and 37% for 2^nd, 3^rd, and 4^th year residents, respectively). Underconfidence decreases when residents reach their 4^th year (25, 38, 28, and 17% for 1^st, 2^nd, 3^rd, and 4^th year residents respectively) while overconfidence (10,12, 15, and 13% of 1^st, 2^nd, 3^rd, and 4^th year residents, respectively) remained consistent.

Association between online literature access and clinical decision accuracy

McNemar’s test demonstrated a significant effect of online literature access in clinical decision accuracy (p<0.001) across all cases. Clinical decision accuracy increased from 45% in the pre-test to 72% in the post-test. When clustered by specialty, the effect is statistically significant at p<0.001 except in the Emergency Medicine specialty (p=0.108). Separately for each residency year level, literature access is seen to have a statistically significant effect on clinical decision accuracy except among forth year residents (p=0.180).

Discussion

Overall, we found a small but statistically significant relationship between residency year level and clinical decision accuracy. First year residents had more incorrect clinical decisions than their seniors. This may imply that CDSS might be beneficial when introduced earlier in medical training and education. This is consistent with Friedman’s 2001 and 2004 papers that discussed how medical students had more incorrect diagnosis than residents and attending physicians.

In this study, a small but significant relationship between physicians’ confidence and their accuracy in clinical decisions was found. Residents were more incorrect (55%) than correct (45%). This highlights the problem of how physicians underestimate their own error rates²⁵. When made more aware of these errors, physicians may default to practicing defensive medicine, wherein their lack of confidence leads to unnecessary tests and studies, which are additional burdens to patients^{21, 22}. Furthermore, as there was a larger number of instances where residents were not confident, which makes them 1.42 times more likely to be incorrect than they were if they were confident, strategies that increase their clinical competency and in effect, increase their confidence is needed.

Resident physicians were generally more concordant (60%), a setting where their confidence is aligned with correctness, than discordant (40%). In clinical practice, a concordance of confidence and clinical accuracy or correctness is favored more than in cases where there is misalignment. Within concordant cases, residents were also more often appropriately diffident (72%) than confident (28%). This demonstrates that there is a greater likelihood to be incorrect when lacking confidence compared to a marginal chance of being correct when feeling confident. Within cases of discordance, residents were more often underconfident (70%) than overconfident (30%). This finding is similar to the previous study’s results that also demonstrated a tendency for clinicians towards underconfidence³². This has optimistic implications in the use of CDSS as these physicians are likely to confirm their decisions using external resources.

Additional findings demonstrate that the relationship of confidence and decision accuracy does not depend on specialty. Specialties were equally prone to errors, irrespective of confidence. When clustered by residency year level, a significant relationship was seen among first and fourth year residents, with the latter group having a stronger effect. First year residents were less confident than fourth years. This may be due to a perceived lack of clinical experience, or because they are overmatched by the difficulty of managing patients they see. This is further demonstrated by the fact that first years are more appropriately diffident than fourth years. Fourth year residents on the other hand were more appropriately confident than first year residents. Fourth year residents were also less underconfident compared to the younger residency years. Senior residents may perceive that they have higher clinical competency, and that their clinical experience is correctly matched to the cases compared to their juniors. This evidence of clinical experience bridging the gap between their confidence and accuracy of clinical decisions is similar to previous findings that suggest how higher levels of experience leads to an increased awareness of one’s capabilities³², which then improves the alignment of confidence and accuracy. Overconfidence remains consistent among year levels, which exhibits the role of CDSS in all levels of clinical experience.

Consistent with the findings from the previous study where data in this study was derived from, access to online literature search significantly affects clinical decision accuracy (from 45% to 72% accuracy). When stratified based on specialty, this effect was also significant, except among Emergency Medicine residents. In terms of residency year level, the effect was significant except among 4^th year residents. These findings stress how providing access to literature search through CDSS affords statistically significant improvements in the accuracy of making evidence-based clinical decisions in a physician’s everyday clinical practice²⁰.

Limitations

The same limitations from the previous study also apply. The sample size limits generalizability, while the use of clinical cases can only approximate actual clinical encounters. The limited number of cases answered per resident may have also reduced the possible variations in the study. These limitations may have contributed to the small effect sizes observed in the data analysis. In future studies, a larger sample size, pool of cases, and use of quantifiable measures for confidence may help address these limitations.

Conclusion

In this study, we discovered how physicians’ confidence based on perceptions for needing access to online literature, plays a vital role in clinical cognition and decision-making. These results demonstrated how misalignments in confidence and clinical decisions due to cognitive processes may affect the accuracy of health care delivery to patients. We also showed further that online literature access improves clinical decision-making. The results indicate that clinical information should probably be ‘pushed’ to physicians who are overconfident and unlikely to seek information. Furthermore, easy and convenient access to information through different strategies such as practice guidelines, smartphones and mHealth at the point of care will likely benefit underconfident and diffident physicians. Previous studies of have shown the importance of convenient access to evidence where it’s needed. These findings have diverse implications in design and implementation strategies of CDSS to support evidence-based practice.