Japanese Internists' Most Memorable Diagnostic Error Cases: A Self-reflection Survey

Abstract

Objective

The etiologies of diagnostic errors among internal medicine physicians are unclear. To understand the causes and characteristics of diagnostic errors through reflection by those involved in them.

Methods

We conducted a cross-sectional study using a web-based questionnaire in Japan in January 2019. Over a 10-day period, a total of 2,220 participants agreed to participate in the study, of whom 687 internists were included in the final analysis. Participants were asked about their most memorable diagnostic error cases, in which the time course, situational factors, and psychosocial context could be most vividly recalled and where the participant provided care. We categorized diagnostic errors and identified contributing factors (i.e., situational factors, data collection/interpretation factors, and cognitive biases).

Results

Two-thirds of the identified diagnostic errors occurred in the clinic or emergency department. Errors were most frequently categorized as wrong diagnoses, followed by delayed and missed diagnoses. Errors most often involved diagnoses related to malignancy, circulatory system disorders, or infectious diseases. Situational factors were the most cited error cause, followed by data collection factors and cognitive bias. Common situational factors included limited consultation during office hours and weekends and barriers that prevented consultation with a supervisor or another department.

Conclusion

Internists reported situational factors as a significant cause of diagnostic errors. Other factors, such as cognitive biases, were also evident, although the difference in clinical settings may have influenced the proportions of the etiologies of the errors that were observed. Furthermore, wrong, delayed, and missed diagnoses may have distinctive associated cognitive biases.

Introduction

Diagnostic errors represent critical patient safety challenges for internists (1-7). The harm related to diagnostic errors is significant, as they occur in at least approximately 0.7% of all hospital admissions and cause 1 of every 10 emergency room visits (8). Furthermore, diagnostic errors are associated with significant mortality. In the United States, for example, 1 out of every 1,000 diagnostic errors are thought to be fatal, and diagnostic errors are estimated to be responsible for 40,000 to 80,000 deaths annually.

The causes of diagnostic errors have been studied from various perspectives and with various explanatory frameworks, including the dual-process model (9), studies of cognitive biases (10-15), and the analysis of system errors (3,4,16). Previous studies have hypothesized that cognitive biases contribute more significantly to errors than physician memory and experience (11,17-23). However, determining the cause of internists' diagnostic errors remains challenging for several reasons (4,6,24-26). First, some factors contributing to errors are intrinsic to the provider (e.g., medical knowledge and experience, psychological or work challenges, degree of fatigue) and are difficult to deduce from common study approaches that often rely on a third-party assessment of data. Further differences in health care systems, safety culture, and national differences may also have an impact on errors (27,28). Second, eliminating cognitive bias is challenging without reflection by those involved in the error (6,26). Third, recall bias may lead to a lack of crucial and detailed information about the diagnostic error if it is epidemiologically defined in the time after its occurrence (e.g., error within one week) (25). Therefore, honest reflection and analysis of diagnostic error cases are necessary for physicians to extract the most detailed information at the time of the error (6). Therefore, this study aimed to identify diverse causes and characteristics of diagnostic errors as identified by self-reflection among Japanese internists.

Materials and Methods

We performed a nationwide cross-sectional survey using a web-based tool. To minimize recall bias, participants were asked about their most memorable diagnostic error case (MDEC). MDEC was defined as a diagnostic error in which the participant had direct involvement and could most vividly recall the time course, situational factors, and psychosocial context in which the error occurred.

Participants

The internists participating in the study were recruited over 10 days (21-31 January 2019) through the member page of ‘Nikkei Medical Online', the most prominent physician membership site in Japan. We summarized the study and briefly described 10 cognitive biases that are relatively well known in Japan (Supplementary material 1) and then obtained consent to participate in the study (6). Registration was limited to one case per participant. In total, 2,220 internists consented to participate. A total of 687 cases were included in the analysis after exclusion criteria were applied (Supplementary material 2).

Sample size

According to the 2018 Ministry of Health, Labour, and Welfare survey, internists account for 39.8% of all physicians in Japan (n=124,235/311,963). Therefore, of the approximately 110,000 registered members of Nikkei (as of 1 January 2019), we assumed that 43,780, or approximately 40%, were internal medicine physicians, with a margin error of 5% and a confidence interval (CI) of 95%. Thus, the required sample size of internal medicine physicians was 385.

Inclusion and exclusion criteria

The inclusion criteria for selecting participants were that their primary specialty was internal medicine and that they had completed their initial mandatory training in Japan (three years or more of postgraduate training). Cases were excluded if the primary specialty of the physician was not internal medicine, the final diagnosis was unclear, answers to participant characteristics (e.g., age, affiliation, clinical setting) were insufficient, intentionally inappropriate answers were provided (e.g., reported no diagnostic errors, provided five or more identical answers consecutively), self-reflection about the cause of error was insufficient, or the clinical outcome could not be accurately ascertained. Based on these criteria, 1,533 cases were excluded, leaving 687 cases for the final analysis (Supplementary material 2).

Variables and definition

MDEC was defined as 1) a case in which the internists could most vividly recall the time course of the case, circumstances, and psychological context in which the diagnostic error occurred and 2) a case in which the internists themselves were primarily involved in the error. Diagnostic error was defined as a missed, wrong, or delayed diagnosis, a taxonomy proposed by Graber (29,30). An English translation of the survey tool is available in the supplementary materials (Supplementary material 3). Internists' demographic characteristics (age, years since medical school graduation, current institutional affiliation) were self-reported. In addition, we collected information such as time taken to recognize the diagnostic error, place, time, and day of the week on which the error occurred, and the final diagnosis based on a prior study (6). Internist respondents were also asked to analyze the causes associated with the MDEC. Finally, the causes were classified into three categories of factors (i.e., situational, data gathering/interpretation, and cognitive bias), and participants were asked to assign points regarding the weight of the contribution of each factor to the error. Participants had a total of 10 points to allocate (e.g., 2/10 situational, 3/10 data gathering/interpretation, 5/10 cognitive bias) (6,31). Furthermore, for cognitive bias, we proposed 10 representative cognitive biases from a previous study conducted in Japan and added a brief example and explanation (Supplementary material 1) (6).

Outcome measures

The primary aim of this study was to assess the relative involvement of the three diagnostic error categories (wrong, missed, delayed) (29,30). The secondary aim was to use a multivariate analysis to determine which background and cognitive bias factors were associated with each types of diagnostic error. The characteristics of the MDEC were also identified descriptively.

Statistical analysis

All analyses were performed using Stata statistical software (Stata 17 Base Reference Manual; Stata, College Station, USA). All tests were two-tailed, with p values of ＜0.05 considered to indicate statistical significance. Standard descriptive statistics were number, percentage, mean, median, and interquartile range (IQR) for each piece of data. The chi-square test or Fisher's exact test was used for categorical data comparisons. For continuous variables, t tests or Wilcoxon rank sum tests were employed as appropriate.

Additionally, multivariate linear and multivariate logistic regression analyses were performed to examine factors associated with cognitive bias. Seven situational factors, four data gathering/interpretation factors, and ten cognitive-bias factors were selected based on clinical relevance and prior research (24). The criteria used for the explanatory variables were: 1) factors associated with cognitive bias in previous studies; and 2) factors that were statistically significant (p＜0.05) based on univariate screening and univariate regression. Specifically, the number of years after graduation at the time and location of the error, whether the error occurred at night, and whether the error occurred on weekends/holidays were selected for the multiple linear regression analysis of factors associated with cognitive bias and multiple logistic regression analysis of factors associated with the presence of cognitive bias.

Finally, we also performed a sensitivity analysis incorporating several factors into the multivariate logistic equation for each of the three diagnostic error categories, night or day, location of errors, holidays, and internists' age.

Ethical considerations

The authors obtained approval from the Medical Research Ethics Committee, Shimane University Faculty of Medicine (No. 20181017-1). All participants provided informed consent before taking part in the study.

Results

Baseline characteristics of survey respondents

A total of 687 MDECs were analyzed. Participants had a median age of 50 years (IQR 40-58), and the median number of years since graduation was 25 (IQR 14-31). At the time of the survey, 269 (39.2%) participants were affiliated with large hospitals (＞300 beds), 203 (29.6%) with small- and medium-sized hospitals (20-299 beds), 207 (30.1%) with clinics, and 8 (1.2%) with other settings (public health, government facilities). Table 1 provides background information on the diagnostic error cases.

Table 1.

Characteristics of the Most Memorable Diagnostic Error Case among Japanese Internists (n=687).

Years after graduation when encountering the case	4 (IQR 2-10)
The place encountered the case
General outpatient office	241 (35.1%)
Emergency room	213 (31.0%)
Ward	158 (23.0%)
Specialist outpatient office	38 (5.5%)
Procedure room (dialysis, endoscope, etc.)	20 (2.9%)
Other	17 (2.5%)
Working hours when encountering the case
Morning (08:30-12:00)	250 (36.39%)
Afternoon (12:00-17:00)	148 (21.5%)
Night shift/on duty (17:00-08:30)	220 (32.0%)
Other	69 (10.0%)
Day of the week when encountering the case
Monday-Friday	571 (83.1%)
Saturday	42 (6.1%)
Sunday/holiday	74 (10.8%)
Time taken to detect the diagnostic error
Within a few hours	64 (9.3%)
Within a few days	347 (53.2%)
Within a few weeks	119 (18.3%)
Within a few months	89 (13.7%)
Within a few years	68 (10.2%)

IQR: interquartile range

Errors were mostly classified as wrong diagnoses, followed by delayed diagnoses and missed diagnoses (Fig. 1). In total, 13.2% of cases were classified into multiple categories, 52.0% of errors had a wrong diagnosis component, 35.4% had a delayed diagnosis component, and 29.5% had a missed diagnosis component. The diagnoses most recalled in the reflection of MDECs included those related to malignancy, the cardiovascular system (e.g., acute coronary syndrome), and infectious diseases, which accounted for 54.1% of all cases. The next most common diagnoses were stroke, peritonitis/intestinal obstruction, acute appendicitis, and endocrine diseases (Fig. 2).

Figure 1.

Diagnostic error classification (wrong, delayed, missed) of the most memorable diagnostic errors by internists (n=687). A wrong diagnosis was found in half of all cases, while delayed and missed diagnoses were found in approximately 30% of the cases.

Figure 2.

Pareto chart and ranking of confirmed final diagnosis of the most memorable diagnostic error cases among Japanese internists.

Weighting of contributing factors to diagnostic errors provided by self-reflection

Internists cited situational factors as most commonly being present in their case, followed by data gathering/interpretation factors and cognitive bias factors. When evaluating the correlation between these factors and each diagnostic error category (missed, wrong, and delayed), data gathering/interpretation factors were significantly correlated with missed diagnosis (Table 2). Otherwise, there were no significant correlations between the three diagnostic error categories and the three factors.

Table 2.

Score out of 10 Points for All Contributing Factors in Wrong, Delayed, and Missed Diagnoses, as Evaluated by the Internists Involved in the Cases.

	Wrong diagnosis n=281
	+			-
	Mean	SE	95% CI	Mean	SE	95% CI	p value
Situational factors	3.87	0.16	3.55-4.19	3.51	0.10	3.32-3.70	0.239
Data gathering/interpretation factors	3.04	0.12	2.81-3.27	3.44	0.08	3.28-3.61	0.088
Cognitive bias factors	3.09	0.15	2.80-3.37	3.04	0.09	2.87-3.22	0.951
	Delayed diagnosis n=178
	+			-
Situational factors	3.50	0.11	3.28-3.71	3.75	0.13	3.50-4.00	0.762
Data gathering/interpretation factors	3.45	0.10	3.26-3.64	3.19	0.10	2.99-3.39	0.376
Cognitive bias factors	3.05	0.10	2.85-3.25	3.06	0.11	2.84-3.28	0.584
	Missed diagnosis n=137
	+			-
Situational factors	3.55	0.14	3.28-3.83	3.65	0.10	3.45-3.86	0.139
Data gathering/interpretation factors	3.44	0.12	3.20-3.68	3.26	0.08	3.10-3.43	0.046
Cognitive bias factors	3.01	0.13	2.76-3.27	3.08	0.09	2.90-3.26	0.979

SE: standard error, CI: confidence interval,+: present, -: absent

Contributions of cognitive bias, data gathering/interpretation, and situational factors to MDEC cases

The contributions of cognitive bias, data gathering/interpretation, and situational factors to MDEC cases are presented in Fig. 3.

Figure 3.

Results of the subcategorization of the three major factors of the most memorable diagnostic error cases among Japanese internists.

Cognitive bias factors

Of all MDEC cases, in 480 cases (69.8%), cognitive bias factors contributed to the error. Of the 10 representative biases presented, the most frequently cited were overconfidence bias (23.4%) and confirmation bias (18.3%), followed by availability bias (10.6%), premature closure (9.0%), and anchoring bias (8.2%). In contrast, the prevalence of rule bias and Maslow's law were negligible.

Data gathering/interpretation factors

Similarly, data gathering and interpretation factors were involved in 579 cases (84.3%). Participants frequently indicated that information gathered during the diagnostic process was insufficient. Faulty data gathering involved a lack of physical examination or laboratory testing (54.3%) most commonly, followed by inadequate history taking (23.4%), misinterpretation of information on examination (23.1%), and misinterpretation of information from the patient and family (13.0%).

Situational factors

Situational factors contributed to 520 (75.7%) of all cases. The most common environmental or situational causes were restricted examinations (26.9%), such as time of day or weekends; barriers at night or during the day that prevented consultation with supervisors or other departments (24.3%); and daily fatigue (24.3%). This was followed by patient overcrowding (20.4%), external stress (16.7%), unreasonable work schedules and shift problems (15.6%), and poor staffing and facilities in the hospital at all times (12.4%).

The results of the univariate and multivariate logistic analyses of the factors associated with each case of wrong diagnosis, delayed diagnosis, and missed diagnosis are presented in Table 3. The univariate analysis comparing 23 items for each of the three diagnostic error categories demonstrated that confirmation bias was significant among those making a wrong diagnosis due to misinterpretation of patient information. For a delayed diagnosis, only the base rate neglect was significant. For missed diagnoses, overcrowding, overconfidence, and hassle bias made significant contributions. The results adjusted for the other 23 items were not significant.

Table 3.

Factors Associated with Wrong, Missed, and Delayed Diagnoses.

	Univariate logistic analysis for each diagnostic error category				Multiple logistic analysis for each diagnostic error category
	OR	95% CI	p value		Adjusted OR	95% CI	p value
Wrong diagnosis, n=357 (52.0%)
Misinterpretation of the patient's information	1.67	1.05-2.65	0.028		1.59	0.95-2.65	0.078
Rule bias	3.08	0.99-9.54	0.051		3.67	1.14-11.8	0.029
Confirmation bias	1.51	1.02-2.24	0.039		1.53	1.01-2.32	0.046
Delayed diagnosis, n=243 (35.4%)
Base rate neglect	2.10	1.24-3.54	0.006		1.83	1.04-3.20	0.036
Anchoring bias	1.10	0.63-1.95	0.728		2.37	1.02-5.50	0.044
Missed diagnosis, n=203 (29.5%)
Overcrowding	1.68	1.14-2.47	0.009		1.53	0.99-2.36	0.052
Overconfidence bias	1.65	1.14-2.40	0.008		1.70	1.13-2.58	0.012
Hassle bias	2.07	1.20-3.58	0.009		3.30	1.51-7.22	0.003

To adjust for potential confounders of clinically significant associated factors for the three diagnostic error categories, the following variables were incorporated in the multivariate analysis: situational factors (7 items excluding others), data gathering/interpretation factors (4 items excluding others), cognitive bias factors (9 items excluding others and Maslow’s law), location (ER/outpatient), time (evening), and day of the week when the diagnostic error occurred (holidays).

OR: odds ratio, CI: confidence interval

In the multivariate analysis, rule bias [adjusted odds ratio (aOR) 3.67; 95% CI 1.14-11.8, p=0.029] and confirmation bias (aOR 1.53; 95% CI 1.01-2.32, p=0.046) were associated with obtaining a wrong diagnosis. Overconfidence bias (aOR 1.70; 95% CI 1.13-2.58, p=0.012) and hassle bias (aOR 3.30; 95% CI 1.51-7.22, p=0.003) were significant for missed diagnoses. No statistically significant associations were identified in the sensitivity analysis.

Discussion

In this cross-sectional survey in which Japanese internists self-reflected on 687 cases of most memorable diagnostic errors, wrong diagnosis was the most common category of error, followed by delayed and missed diagnoses. A total of 13% of cases had a combination of wrong, delayed, and missed diagnoses. Notably, Japanese internists were most likely to report that situational factors, rather than cognitive factors, were the cause of each type of diagnostic error in the study. This may result from features of the medical system and the working environment in each clinical setting, and may call into question the application in Japan of conventional theories of cognitive biases as a primary cause of diagnostic error (20,29,31-37). Situational and cognitive bias factors were found to be correlated with each category of diagnostic error. Based on this assumption, according to the Pareto chart (Fig. 2), which shows Japanese internists' diagnostic errors, malignancies, cardiovascular diseases including acute coronary syndrome, and infectious diseases accounted for more than half of the cases in this study. This was similar to the rankings in US studies in which diagnostic errors leading to death or serious sequelae were classified as the Big Three: malignancy, vascular disease, and infectious disease (38,39). Thus, our data are consistent with the trends in epidemiological studies of the Big Three (4,38-40).

Why are situational factors high in japanese studies?

Studies have emphasized the importance of information integration factors, or cognitive process issues, as the most significant contributors to diagnostic error (12,16-22,31). Cognitive bias, which can lead to internists making errors in judgment, has been particularly recognized as a significant problem (29,41). Our results revealed that situational factors were prominent contributors to error.

We hypothesize that the cause of diagnostic errors may be the adverse effects of issues in the Japanese medical system and problems in the practice environment of internists. Japan has a universal health insurance system that guarantees free access to almost all standard medical care (42). To understand the potential implications of universal access, it is helpful to compare access to care and access to providers in Japan and the US. Japan has the highest number of hospital beds per 1,000 population globally, more than four times that of the US (12.6 vs. 2.8 beds/thousand persons) (43). Furthermore, the number of outpatient visits per capita in Japan is also among the highest worldwide (43). However, the number of physicians per capita is the same in Japan and in the US (2.6) (43). This indicates that a single physician in Japan is responsible for a considerable number of outpatients and ward patients. A systematic review of the time spent per patient in primary care settings found that US physicians spend more than 20 minutes per patient, whereas Japanese physicians spend only approximately 10 minutes (44). Such challenging work environments lead to physician burnout and have become a medical safety issue and a critical problem in Japan (4,7,45).

From a diagnostics process perspective, although Japan has considerably more computed tomography (CT; 112/million) and magnetic resonance imaging (MRI; 55/million) equipment per population than the US (CT 43/million, MRI 38/million), the number of CT and MRI examinations per population is slightly higher or approximately the same in the US (43). This is due to process limitations, such as a lack of systems for examinations at night and on holidays and a lack of radiologists to interpret images, a task that often falls upon internists (46). Thus, we consider it highly likely that fundamental differences in clinical settings underlie differences in the etiologies of the errors that were observed in Japan relative to the US. Research on diagnostic errors in many countries outside the United States is scarce (47), and contradictory findings regarding the causes of diagnostic errors due to differences among countries and health care systems cannot be ignored. Therefore, situational factors and relevant system errors must be considered to be causes of diagnostic errors, and their contributing factors should be explored to improve the working environment in Japanese medical settings.

Strengths

This study had three main strengths. First, this study utilized reflection on the most memorable diagnostic errors, from which details could be extracted. This allowed for a unique lens and understanding of the causes of diagnostic errors (6). As a result, we were able to substantially/effectively reduce the arbitrary manipulation of causes of error that often occurs when using third-party data (26,27,47-49). Second, this is the first large-scale nationwide diagnostic error study conducted in Japan. Although small-scale pilot studies have been published, there have been no studies with a sufficiently large sample size (6). Third, this is one of the few studies demonstrating situational factors as the etiology of internists' diagnostic errors. As mentioned previously, cognitive bias has been considered the most significant cause of diagnostic errors. However, the results of the error analysis by internists themselves may differ from those by a third party, and different sociocultural backgrounds may produce different ideas about the causes of errors.

Limitations

This study was associated with several limitations. First, the study was not designed to investigate the epidemiology of actual diagnostic errors. The study design did not allow us to determine how often diagnostic errors occur. However, reflecting on the most memorable diagnostic errors allowed internists to analyze the situation in greater detail. Second, these data were obtained from internet-based Nikkei Medical Online registrants, and hence, sampling bias was inevitable. Third, external validity may be a concern, although it may be influenced by Japan's medical system. The results need to be interpreted in light of these critical differences. Fourth, the selection of cognitive biases may be a limitation. Although we used previous studies to list 10 cognitive biases that are well known in Japan, proving the representativeness and validity of these cognitive biases is difficult. This is because more than 100 cognitive biases are already known, and it was difficult to reflect on all of them in the study. Furthermore, because cognitive biases often overlap, they may be difficult for internists to categorize rigorously. Despite these inevitable limitations, the present study approaches the more fundamental causes of diagnostic errors by allowing internists themselves to reflect on their errors.

Conclusion

The results of self-reflection on diagnostic error cases by Japanese internists suggest that situational factors may be an important cause of such errors. In contrast to the previously held belief that cognitive bias is the most important cause of diagnostic errors, our results suggest that differences in clinical settings might change the degree of contribution of different factors to the causes of diagnostic errors as considered by the internists themselves. We also found that distinct cognitive biases may be associated with wrong, delayed, and missed diagnoses. Further research should be conducted to explore the reasons for the high frequency of situational and data gathering/interpretation factors among Japanese internists.

The authors state that they have no Conflict of Interest (COI).

Financial Support

This work was supported by national academic research grant funds [JSPS KA-KENHI: 17K15745, 20H03913].

Supplementary Material

Supplementary File 1.

Ten representative cognitive biases shared with participants.

Supplementary File 2.

Sampling process flow chart

Supplementary File 3.

Questionnaire