Abstract
Objectives
Despite their significance, little is known about diagnostic errors in general pediatric practice other than data from malpractice claims. We surveyed pediatricians to elicit their perceptions about frequency, contributing factors and potential systems and provider-based solutions to address diagnostic errors.
Methods
Academic, community, and trainee pediatricians at three tertiary care institutions and affiliated 109 clinics (N=1,362) were invited to complete the survey anonymously via an Internet survey administration service between November 2008 and May 2009.
Results
Overall response rate was 53% (N=726). Over half (54%) of respondents reported that they made a diagnostic error at least once or twice per month; this frequency was markedly higher (77%) among trainees. Almost half (45%) reported diagnostic errors that harmed patients at least once or twice per year. Failure to gather information through history, physical examination or chart-review was the most common reported process breakdown, whereas inadequate care coordination and teamwork was the most common system factor reported. Viral illnesses being diagnosed as a bacterial illness was the most commonly reported diagnostic error followed by misdiagnosis of medication side-effects, psychiatric disorders, and appendicitis. Physicians ranked access to electronic health records and close follow-up of patients as strategies most likely to be effective in preventing diagnostic errors.
Conclusions
Pediatricians reported making diagnostic errors relatively frequently and patient harm from these errors was not uncommon. Our survey provides new data about the types of diagnostic errors and their causes and lays the groundwork for a concerted, multifaceted approach to reduce these errors in children.
Keywords: diagnostic errors, children, patient safety, missed and delayed diagnosis, misdiagnosis, malpractice
Introduction
Errors in diagnosis constitute a sizable proportion of medical errors in the United States and are responsible for significant costs and harm.1–10 For instance, in the Utah-Colorado study diagnostic breakdowns were the most common type (36%) of ambulatory preventable adverse events that led to hospital admission.10 Although medical errors related to treatment (e.g., medications, surgery) have received much-needed recent attention,11 errors in the diagnostic process remain relatively understudied.12 Data about diagnostic errors from pediatric practice settings are especially limited. To date, knowledge of these errors is limited mostly to events that resulted in malpractice claims. Error in diagnosis is the most commonly identified reason in closed pediatric malpractice claims (32%), with the highest median indemnity payment and defense expenses.13 However, diagnostic errors that result in claims may only represent a small proportion of all diagnostic errors; given their low frequency and high severity, they may not be representative of all the types of diagnostic errors that occur in routine practice.2,14 Furthermore, underlying contributory factors for litigated diagnostic errors might differ systematically from errors not litigated.14 In short, malpractice claims studies have provided useful data to understand diagnostic errors but may not represent the entire spectrum of diagnostic errors.
Errors of diagnosis occur when diagnosis is unintentionally delayed (sufficient information was available earlier), wrong (another diagnosis was made before the correct one), or missed (no diagnosis was ever made), as judged by the eventual appreciation of more definitive information. 4 Although diagnostic errors may occur when symptoms and signs of a disease are atypical or absent, they are largely related to cognitive (e.g., faulty data gathering or clinical reasoning) and systems-related factors (e.g., issues with policies, processes and procedures, organizational issues).2,4,15–28 This understanding of diagnostic errors is largely based on non-pediatric settings and populations.
The diagnostic process in children is unlikely to be entirely analogous to adults. The most prevalent types of diseases and their associated diagnostic processes are different (e.g., certain infections are more common whereas cancer is more uncommon in children). Other unique factors in pediatrics include systems issues, patient/caregiver issues, training of physicians, and practice workflow, all of which are likely to influence diagnostic errors. In order to enhance our understanding about the relative prevalence of diagnostic errors, contributory factors, and potential preventive strategies, we designed an anonymous survey to study pediatricians’ experiences with diagnostic errors. We also sought data on specific disease conditions commonly associated with diagnostic errors in pediatrics.
Methods
Setting and Participants
To obtain diverse perspectives and solutions, we sampled three types of pediatricians: academic (general pediatricians and subspecialists), trainees (residents and fellows), and non-academic community-based pediatricians. Academic pediatricians and trainees belonged to the University of Texas Medical School, Baylor College of Medicine, and Cincinnati Children’s Medical Center. Although we invited participation from all trainees and general pediatricians at the three study sites, we randomly invited only 50% of pediatric subspecialists at Baylor College of Medicine and Cincinnati Children’s Medical Center due to the large number of subspecialists at those sites. Community pediatricians were selected from large practice groups affiliated with the teaching hospitals of the latter two institutions and represented 109 practice locations. Institutional Review Board approval was obtained at all three study sites.
Survey Development
We were unable to find an existing survey that addressed our study questions. Building on prior empirical and theoretical research on diagnostic errors, we developed a comprehensive survey following an exhaustive review of both adult and pediatric literature. A psychometrician guided the survey development process, which included item writing and refinement using subject matter expert input, clinician review, and pilot testing. Content of survey items was informed by published studies, input from experts in patient safety and diagnostic errors, and feedback from a sample of pediatricians from our study sites. The survey was developed for Web-based administration utilizing the resources of a user-friendly commercial Internet survey administration service (www.SurveyMonkey.com).
After initial refinements of survey items, we convened four focus groups of local pediatricians to review the survey for readability, clarity, and ease of completion in a Web-based environment. The survey was then pilot-tested with 8 general pediatricians. To understand the burden of diagnostic error in relation to other types of error, we enquired about frequency and potential to harm for errors related to medications, surgery and other clinical activities in pediatrics. In addition, the 23 item final survey (see Appendix) assessed physicians’ perceptions of 1) the most common process breakdowns associated with diagnostic errors (e.g. issues with history and examination, test ordering, performance or interpretation, and follow-up issues as gleaned from recent literature2,4,7,8,26,29–31); 2) the most common contributory factors that lead to diagnostic errors; 3) the most common types of medical conditions associated with diagnostic errors; and 4) the most useful strategies for prevention of diagnostic errors. Therefore, for many questions we used response scales with forced-choice ranking of top three response choices rather than Likert-style responses. The survey also assessed demographic information, self-reported frequency of diagnostic errors, attitudes toward discussing diagnostic errors, and previous training about diagnostic error. Estimated completion time was 15–20 minutes based on pilot testing.
Survey Administration
We identified potential participants from medical staff offices at each institution and sent each physician an e-mail invitation to complete the survey, followed by up to two e-mail reminders. As an incentive to participate, we provided a web link to a $10 gift certificate in the initial invitation. Participants accessed and completed the items anonymously; only the study site and pediatrician category of each participant were identified.
Data Analysis
Data were downloaded from the Internet survey administration service and analyzed using SAS version 9.2. We generated descriptive statistics about the nature and frequency of diagnostic errors in aggregate and then separately for academic pediatricians, community pediatricians, and trainees. Many items asked respondents to rank a first, second, and third choice from a list of possible outcomes. To calculate these rankings, we used weighted averages computed as follows: if a respondent ranked a particular choice first, that choice was given 3 points; 2 points were given for a second choice and 1 point for a third choice (all other choices were scored as 0). We then computed the average of all these values over all respondents. To assess for non-randomness of ranking, we used Friedman’s chi-squared test to test for significant differences among choices marked by respondents, i.e. whether a certain item was ranked above or below others after removal of variation among respondents. We tested the items in pairs with the significance level adjusted for multiple comparisons. We tested for differences in response frequencies between pediatrician type and between study sites using chi-square tests. We used linear regression to examine the effect of gender, race, practice site, type of pediatrician, training related to diagnostic errors in medical school and residency and years of clinical experience post-residency on the outcome “self-reported rate” of diagnostic errors. We used a scale consisting of seven possible frequencies; the seven values were treated as a continuous variable. The potential covariates were examined individually for inclusion in the final model.
Results
Between November 2008 and May 2009, a total of 1,362 survey invitations were sent to physicians at the three study sites: 194 to the University of Texas at Houston, 569 to Baylor College of Medicine, and 599 to the University of Cincinnati. Invitees included 516 academic physicians, 430 trainees, and 416 community-based physicians. The overall response rate was 53%; groupwise rates were 52% for academic physicians, 39% for community-based physicians, and 67% for trainees. However, not all respondents completed the survey in its entirety; the overall survey completion rate among respondents was 86%, groupwise completion rates were 87%, 88%, and 85% respectively. Chi-squared tests showed that there were no significant differences between physician types or between sites in response rate.
Academic, community, and trainee groups did not differ in racial/ethnic distribution (Table 1). About a quarter (27%) of respondents had been in practice for more than 10 years, and two-thirds (67%) performed mostly clinical duties. Approximately half (48%) of respondents received training about diagnostic errors in medical school and a somewhat higher number (59%) in residency; however, this training was largely informal in nature. Majority (81%) of respondents reported discussing the diagnostic errors they had made either some, most, or all of the time with colleagues.
Table 1.
Baseline Characteristics of Survey Respondents
| Trainee Pediatrician (n=291) | Academic Pediatrician (n=289) | Community Pediatrician (n=165) | Total (n=727) | |
|---|---|---|---|---|
| n (%) | n (%) | n (%) | n (%) | |
| Study Site (p value <0.0001) | ||||
| Texas Children’s & Affiliates | 118 (41) | 137 (51) | 102 (62) | 357 (49) |
| University of Texas Houston | 44 (15) | 59 (22) | 0 (0)* | 103 (14) |
| Cincinnati Children’s Hospital Medical Center | 129 (44) | 75 (28) | 63 (38) | 267 (37) |
| Gender (p value 0.045) | ||||
| Male | 103 (35) | 123 (46) | 67 (41) | 293 (40) |
| Female | 188 (65) | 146 (54) | 98 (59) | 432 (59) |
| Missing | 0 | 2 (1) | 0 | 2 (0) |
| Race/Ethnicity (p value NS) | ||||
| American Indian of Alaska native | 1 (0) | 1 (0) | 0 (0) | 2 (0) |
| Asian | 47 (16) | 39 (14) | 19 (12) | 105 (14) |
| Black or African American | 24 (8) | 10 (4) | 15 (9) | 49 (7) |
| Hispanic/Latino | 23 (8) | 24 (9) | 9 (5) | 56 (8) |
| Native Hawaiian or Pacific Islander | 0 (0) | 2 (1) | 0 (0) | 2 (0) |
| White | 186 (64) | 183 (68) | 116 (70) | 485 (67) |
| Other | 10 (3) | 10 (4) | 5 (3) | 25 (3) |
| Missing | 0 | 2 (1) | 1 (0) | 3 (0) |
| Training in Medical School1 | ||||
| Yes, in informal training | 114 (39) | 77 (28) | 52 (32) | 243 (33) |
| Yes, in formal curriculum | 81 (28) | 24 (9) | 10 (6) | 115 (16) |
| No | 62 (21) | 136 (50) | 84 (51) | 282 (39) |
| Missing | 43 (15) | 36 (13) | 20 (12) | 99 (14) |
| Training in Residency1 | ||||
| Yes, in informal training | 106 (36) | 111 (41) | 68 (41) | 285 (39) |
| Yes, in formal curriculum | 115 (40) | 27 (10) | 18 (11) | 160 (22) |
| No | 37 (13) | 99 (37) | 62 (38) | 198 (27) |
| Missing | 43 (15) | 36 (13) | 20 (20) | 99 (14) |
| Years in Practice (p value <0.0001) | ||||
| 0–5 years | 12 (4) | 85 (31) | 31 (19) | 128 (18) |
| 6–10 years | 0 (0) | 36 (13) | 26 (16) | 62 (8) |
| 11–15 years | 0 (0) | 40 (15) | 21 (13) | 61 (8) |
| 16–20 years | 0 (0) | 26 (10) | 17 (10) | 43 (6) |
| Greater than 20 years | 0 (0) | 44 (16) | 49 (30) | 93 (13) |
| Not Applicable | 236 (81) | 4 (1) | 1 (1) | 241 (33) |
| Missing | 43 (15) | 36 (13) | 20 (12) | 99 (14) |
| Allocated Percentage of Clinical Time (p value <0.0001) | ||||
| 0–25% | 6 (2) | 49 (18) | 0 (0) | 55 (8) |
| 26–50% | 11 (4) | 56 (21) | 2 (1) | 69 (9) |
| 51–75% | 26 (9) | 52 (19) | 8 (5) | 86 (12) |
| 76–100% | 192 (66) | 72 (26) | 134 (81) | 398 (55) |
| Not Applicable | 10 (3) | 0 (0) | 1 (1) | 11 (1) |
| Missing | 46 (16) | 42 (16) | 20 (12) | 108 (15) |
No community group at the University of Texas Houston site.
Percentages add up to more than 100 because few participants responded yes to both informal training and formal curriculum
Table 2 lists the average frequency rankings of clinical activities believed by pediatricians to be associated with error and potential for harm. The Friedman test showed significant differences among the ranked activities (p<0.0001). Overall, pediatricians endorsed errors in medication-related activities such as prescribing and administering as most prone to error and associated with the greatest potential for harm. Figure 1 shows the distributions (by pediatrician category) of self-reported frequency of error (regardless of harm) and frequency of error with harm. Over half (54%) of respondents reported that they made a diagnostic error at least once or twice per month. However, the frequency of self-reported errors that may cause harm was lower. Only 4% reported errors that harmed the patient at least once or twice per month, 18% reported that such errors occurred at least once or twice per quarter, and 45% reported they occurred at least once or twice per year.
Table 2.
Ranking of Clinical Activities Respondents Considered to be Associated with Frequency of Error and Potential to Harm
| n (%) | Ave. rank | |
|---|---|---|
| Frequency of Error | ||
| Medication-related activities, such as prescribing, dispensing or administering of medications | 617 (85) | 2.06 |
| Prevention-related activities, such as hand washing, injury prevention/nutrition counseling, or vaccination status | 398 (55) | 1.11 |
| Monitoring-related activities, such as follow-up on growth charts or close follow-up of acutely or chronically ill children | 419 (58) | 1.01 |
| Evaluation & diagnosis-related activities, such as history and physical, and/or obtaining diagnostic tests and consultations | 401 (55) | 0.99 |
| Surgery and anesthesia-related activities that occur in the operating room | 150 (21) | 0.36 |
| Non-surgical procedure-related activities, such as lumbar punctures or venipunctures* | 91 (13) | 0.17 |
| Potential to Harm | ||
| Medication-related activities, such as prescribing, dispensing or administering of medications | 653 (90) | 2.01 |
| Surgery and anesthesia-related activities that occur in the operating room | 514 (71) | 1.82 |
| Evaluation & diagnosis-related activities, such as history and physical, and/or obtaining diagnostic tests and consultations | 312 (43) | 0.72 |
| Prevention-related activities, such as hand washing, injury prevention/nutrition counseling, or vaccination status | 208 (29) | 0.43 |
| Monitoring-related activities, such as follow-up on growth charts or close follow-up of acutely or chronically ill children | 186 (26) | 0.36 |
| Non-surgical procedure-related activities, such as lumbar punctures or venipunctures* | 194 (27) | 0.35 |
n refers to the number of participants who selected the item and assigned a rank of first, second or third.
(%) refers to the percentage of participants who selected a particular item and ranked the item first, second or third.
errors could potentially include any type of error in procedure performance (wrong patient, wrong technique etc.)
Figure 1.
Figures 1A and 1B: Frequency of Diagnostic Errors Regardless of Patient Harm (1A) and Frequency of Diagnostic Errors that Caused Patient Harm (1B) Self-reported by Pediatricians
Note: Percentages reported in the text are higher because they reflect cumulative frequencies.
In the regression model, nonwhite pediatricians had significantly lower self-perceived error rate and trainees had significantly higher reported rates than academic and community pediatricians. Neither gender nor number of years of experience post-residency had a significant effect on the self-reported diagnostic error rate. Collectively, provider characteristics explained a relatively small proportion of the variance (R2 = 0.17) in self-reported frequency of diagnostic errors.
Diagnosis of viral illnesses as bacterial illnesses was the event most frequently associated with diagnostic error, followed by misdiagnosis of medication side-effects, psychiatric disorders, and appendicitis (see Table 3). The Friedman test showed significant differences among the ranked diseases (p<0.0001). Tables 4 and 5 display respondents’ rankings of process breakdowns and contributory factors perceived to be associated with diagnostic error. Among various types of process breakdowns associated with diagnostic error (Table 4), failure to gather available medical information had the highest ranking followed by care not sought in a timely manner by patient/caregiver. With regard to details of specific contributing factors (Table 5), inadequate care coordination, teamwork and/or communication across clinical settings or providers received the highest rating of all system-related factors. Of all cognitive factors, inadequate data gathering or work-up was ranked highest overall, though trainees ranked inadequate data assessment as a more frequent contributing factor than data gathering. Of miscellaneous factors that lead to diagnostic error, all groups assigned time/workload issues the highest average ranking. Again, the Friedman test showed significant differences among all the ranked variables (p<0.0001) in Table 4 and 5. We further explored the association of “care not sought in a timely manner” with “heath literacy” and found a strong positive correlation (p<0.0005).
Table 3.
Ranking of Conditions Respondents Considered Most Commonly Misdiagnosed in Pediatric Practice*
| Total (n=727) | Trainee (n=291) | Academic Pediatrician (n=289) | Community Pediatrician (n=165) | |||||
|---|---|---|---|---|---|---|---|---|
| n (%) | Ave. rank | n (%) | Ave. rank | n (%) | Ave. rank | n (%) | Ave. rank | |
| Viral Illness Diagnosed as Bacterial Illness | 379 (52) | 1.28 | 161 (55) | 1.32 | 131 (48) | 1.17 | 87 (53) | 1.41 |
| Medication Side Effects | 383 (53) | 1.12 | 164 (56) | 1.27 | 153 (56) | 1.16 | 66 (40) | 0.76 |
| Psychiatric Disorders | 232 (32) | 0.58 | 92 (32) | 0.59 | 86 (32) | 0.59 | 54 (33) | 0.53 |
| Appendicitis | 202 (28) | 0.53 | 79 (27) | 0.48 | 65 (24) | 0.48 | 58 (35) | 0.70 |
| Asthma | 121 (17) | 0.31 | 45 (15) | 0.30 | 53 (20) | 0.37 | 23 (14) | 0.22 |
| Otitis Media | 116 (16) | 0.31 | 39 (13) | 0.26 | 49 (18) | 0.37 | 28 (17) | 0.29 |
| Intussusception | 93 (13) | 0.24 | 35 (12) | 0.21 | 34 (13) | 0.23 | 24 (15) | 0.30 |
| Developmental Dysplasia of the hip | 101 (14) | 0.23 | 37 (13) | 0.18 | 31 (11) | 0.20 | 33 (20) | 0.38 |
| Pneumonia | 74 (10) | 0.16 | 31 (11) | 0.17 | 23 (8) | 0.13 | 20 (12) | 0.19 |
| Meningitis | 59 (8) | 0.15 | 21 (7) | 0.13 | 19 (7) | 0.14 | 19 (12) | 0.22 |
| Testicular Torsion | 37 (5) | 0.10 | 13 (4) | 0.07 | 13 (5) | 0.10 | 11 (7) | 0.14 |
| Bronchiolitis | 37 (5) | 0.07 | 24 (8) | 0.11 | 11 (4) | 0.06 | 2 (1) | 0.01 |
| Other | 30 (4) | 0.06 | 2 (1) | 0.01 | 26 (10) | 0.15 | 2 (1) | 0.01 |
| Fracture | 26 (4) | 0.06 | 7 (2) | 0.03 | 11 (4) | 0.06 | 8 (5) | 0.10 |
The Friedman test showed significant differences among the ranked options (p < 0.0001)
Table 4.
Respondents Ranking of Breakdowns in the Diagnostic Process Most Commonly Associated with Diagnostic Errors*
| Total (n=727) | Trainee (n=291) | Academic Pediatrician (n=289) | Community Pediatrician (n=165) | |||||
|---|---|---|---|---|---|---|---|---|
| n (%) | Ave. rank | n (%) | Ave. rank | n (%) | Ave. rank | n (%) | Ave. rank | |
| Failure to gather medical information through history & physical &/or review of charts | 346 (48) | 1.05 | 138 (47) | 1.08 | 132 (49) | 1.04 | 76 (46) | 1.00 |
| Care not sought in a timely manner by patient/caregiver | 305 (42) | 0.91 | 131 (45) | 0.91 | 99 (37) | 0.81 | 75 (45) | 1.07 |
| Failure to follow-up on abnormal diagnostic laboratory test results | 283 (39) | 0.75 | 106 (36) | 0.73 | 113 (42) | 0.77 | 64 (39) | 0.75 |
| Patient/caregiver non-adherence to provider recommended follow-up plan | 271 (37) | 0.73 | 109 (37) | 0.71 | 93 (34) | 0.69 | 69 (42) | 0.82 |
| Problems with ordering, performance or interpretation of diagnostic/laboratory tests | 258 (35) | 0.71 | 89 (31) | 0.67 | 112 (41) | 0.77 | 57 (35) | 0.65 |
| Inappropriate or inadequate follow-up plan by the provider after the child’s evaluation | 210 (29) | 0.52 | 91 (31) | 0.58 | 77 (28) | 0.53 | 42 (25) | 0.41 |
| Delays related to subspecialist/consultation process | 178 (24) | 0.46 | 68 (23) | 0.39 | 66 (24) | 0.50 | 44 (27) | 0.50 |
The Friedman test showed significant differences among the ranked options (p < 0.0001)
Table 5.
Respondents Ranking of Most Common Contributory Factors of Diagnostic Errors in Pediatrics*
| Total (n=727) | Trainee (n=291) | Academic Pediatrician (n=289) | Community Pediatrician (n=165) | |||||
|---|---|---|---|---|---|---|---|---|
| n (%) | Ave. rank | n (%) | Ave. rank | n (%) | Ave. rank | n (%) | Ave. rank | |
| Types of Factors Involved | ||||||||
| System-related errors, organizational issues or problems in communication | 573 (79) | 1.78 | 240 (82) | 1.91 | 213 (79) | 1.76 | 120 (73) | 1.59 |
| Interplay of cognitive and system-related factors | 562 (77) | 1.58 | 219 (75) | 1.58 | 219 (81) | 1.69 | 124 (75) | 1.39 |
| Cognitive errors (inadequate data gathering or faulty medical decision-making) | 524 (72) | 1.29 | 205 (70) | 1.20 | 195 (72) | 1.25 | 124 (75) | 1.51 |
| “No fault” errors (those involving rare disease or related to patient/caregiver nonadherence) | 321 (44) | 0.80 | 125 (43) | 0.73 | 105 (39) | 0.70 | 91 (55) | 1.07 |
| Types of System Factors Involved | ||||||||
| Inadequate care coordination, teamwork &/or communication across clinical sites/providers | 596 (82) | 2.17 | 237 (81) | 2.19 | 225 (83) | 2.23 | 134 (81) | 2.02 |
| Personnel issues (staffing, training/orientation) | 339 (47) | 0.87 | 141 (48) | 0.85 | 129 (48) | 0.90 | 69 (42) | 0.86 |
| Unavailability of resources (diagnostic testing) due to financial/insurance reasons | 219 (30) | 0.51 | 94 (32) | 0.58 | 68 (25) | 0.39 | 57 (35) | 0.56 |
| Inadequate information systems | 206 (28) | 0.49 | 77 (26) | 0.46 | 90 (33) | 0.57 | 39 (24) | 0.41 |
| Types of Cognitive Factors Involved | ||||||||
| Inadequate data gathering or work- up | 568 (78) | 1.70 | 214 (74) | 1.50 | 218 (80) | 1.78 | 136 (82) | 1.92 |
| Inadequate data assessment | 567 (78) | 1.63 | 228 (78) | 1.71 | 205 (76) | 1.57 | 134 (81) | 1.59 |
| Inadequate recognition of critical information previously documented in chart | 479 (66) | 1.22 | 180 (62) | 1.15 | 189 (70) | 1.27 | 110 (67) | 1.26 |
| Inadequate knowledge base | 312 (43) | 0.74 | 140 (48) | 0.87 | 102 (38) | 0.64 | 70 (42) | 0.68 |
| Types of Miscellaneous Factors Involved | ||||||||
| Time/workload issues | 457 (63) | 1.50 | 180 (62) | 1.51 | 169 (62) | 1.44 | 108 (65) | 1.61 |
| Inexperience of the provider | 348 (48) | 0.99 | 136 (47) | 0.92 | 136 (50) | 1.06 | 76 (46) | 0.98 |
| Language barriers | 289 (40) | 0.81 | 119 (41) | 0.85 | 115 (42) | 0.86 | 55 (33) | 0.65 |
| Health literacy of the patient/family | 243 (33) | 0.61 | 86 (30) | 0.55 | 91 (34) | 0.60 | 66 (40) | 0.73 |
The Friedman test showed significant differences among the ranked options (p < 0.0001)
We also inquired about specific heuristics (important cognitive shortcuts in the face of complex situations) and biases that could affect medical decision-making. Overall, the type of bias with the highest average frequency rating was being too focused on a diagnosis or treatment plan. Another bias with a relatively high frequency rating included being misled by a normal history, physical, laboratory or imaging study result.
Table 6 lists the ranking of selected solutions to reduce diagnostic errors. Of provider-based solutions, close follow-up of patients was ranked as most likely to be effective by all three subgroups of pediatricians, followed by improving teamwork, more time in clinical encounters, and empowering patients and families to be vigilant about the possibility of diagnostic errors. For systems-based solutions, access to electronic medical records which provide comprehensive clinical data was ranked highest again by all three groups of pediatricians, followed by availability of diagnostic decision support tools. Community-based physicians ranked increased access to and availability of consultants and experts second. The Friedman test showed significant differences among all the ranked solutions (p<0.0001).
Table 6.
Respondents Ranking of Likely Effective Provider-Based and System-Based Solutions*
| Total (n=727) | Trainee (n=291) | Academic Pediatrician (n=289) | Community Pediatrician (n=165) | |||||
|---|---|---|---|---|---|---|---|---|
| n (%) | Ave. rank | n (%) | Ave. rank | n (%) | Ave. rank | n (%) | Ave. rank | |
| “Provider Based” Solutions | ||||||||
| Close follow-up of patients of ensure that the diagnosis is correct (rapid follow-up) | 532 (73) | 1.63 | 202 (69) | 1.49 | 193 (71) | 1.59 | 137 (83) | 1.95 |
| Improving teamwork between members of the health care team | 424 (58) | 1.19 | 169 (58) | 1.23 | 167 (62) | 1.27 | 88 (53) | 0.99 |
| Increasing time spent in clinical encounters | 284 (39) | 0.79 | 127 (44) | 0.90 | 92 (34) | 0.67 | 65 (39) | 0.76 |
| Empowering patients and families to be vigilant about the possibility of diagnostic errors | 278 (38) | 0.66 | 101 (35) | 0.60 | 102 (38) | 0.64 | 75 (45) | 0.78 |
| Asking for informal second opinions from name specialty colleagues | 221 (30) | 0.55 | 87 (30) | 0.53 | 87 (32) | 0.61 | 47 (28) | 0.51 |
| More training in clinical reasoning skills | 136 (19) | 0.34 | 56 (19) | 0.36 | 59 (22) | 0.38 | 21 (13) | 0.25 |
| System Based Solutions | ||||||||
| Access to EMR which provides comprehensive patient data | 539 (74) | 1.75 | 224 (77) | 1.84 | 200 (74) | 1.76 | 115 (70) | 1.59 |
| Diagnostic decision support tools (Internet, EMR reference texts, electronic support tools) | 425 (58) | 1.17 | 171 (59) | 1.15 | 159 (59) | 1.22 | 95 (58) | 1.10 |
| Establishing feedback pathways to communicate changes in diagnoses to previous providers | 418 (57) | 1.10 | 176 (60) | 1.21 | 162 (60) | 1.11 | 80 (48) | 0.92 |
| Increased access to and availability of consultants and experts | 321 (44) | 0.73 | 104 (36) | 0.50 | 115 (42) | 0.70 | 102 (62) | 1.19 |
| Establishing a peer review process for randomly selected patient records | 181 (25) | 0.43 | 69 (24) | 0.42 | 69 (25) | 0.42 | 43 (26) | 0.47 |
The Friedman test showed significant differences among the ranked options (p < 0.0001)
Discussion
We surveyed academic, community-based, and trainee pediatricians at three study sites about diagnostic errors and found several new insights that may help understand and prevent such errors in the future. Out of 6 types of medical errors, errors in diagnosis were ranked fourth in frequency and third in potential for harm. However, pediatricians reported they made diagnostic errors rather frequently; over half (54%) reported they made a diagnostic error at least once or twice per month. Diagnostic errors that led to harm were also not infrequent; almost half (45%) reported diagnostic errors that harmed patients at least once or twice per year. The most frequent diagnostic error was viral illnesses being diagnosed as bacterial illness, followed by misdiagnosis of medication side-effects, psychiatric disorders, and appendicitis. Failures in data gathering (history, examination, chart review) and care delays by patients/caregivers were reported to be the most frequent process breakdowns. Of various interventions for preventing diagnostic errors, pediatricians gave highest rankings to close follow-up of patients and access to electronic medical records.
To our knowledge, this was the first study to assess diagnostic errors in any setting through a comprehensive survey. Our study builds on the knowledge available from malpractice literature to enhance the understanding of diagnostic errors that affect children, serving as a guide to development of strategies to prevent these errors. According to our findings, pediatricians believe that errors in diagnosis occur most commonly in a set of conditions that differ markedly from those mentioned in pediatric malpractice literature. For example, in pediatric malpractice literature, diagnoses of meningitis, appendicitis, pneumonia and testicular torsion are cited to be most often missed. 32,33 However, we found that most pediatricians believed that viral illness misdiagnosed as bacterial was the most common diagnostic error, followed by misdiagnosis of medication side-effects and psychiatric disease. None of these overlap with the top diagnostic errors found in claims files. Errors of diagnosis of medication related side-effects are relatively unexplored in contrast to medication errors related to prescribing and administering. Coincidentally, a recent analysis of 583 physician-reported diagnostic errors in adults also revealed drug reactions or overdose as second most common type of diagnostic error. 34 Similarly, the concept of potential misdiagnosis of psychiatric disease in children although not entirely new,35 has not been discussed in context of malpractice claims and needs to be revisited with more specificity in future work. Another advantage of using this methodology is that pediatricians provided rich details about the most common process breakdowns and contributory factors, some of which are much harder to obtain from reviewing malpractice claims or medical records.36
We also solicited providers’ rankings of proposed strategies to prevent diagnostic errors. Despite ongoing debate about benefits of electronic health records,37 all groups of pediatricians ranked electronic records as the best system-based solution. Other strategies, such as diagnostic decision support tools and techniques to ensure timely follow-up of certain patients,29,38 may need to be prioritized in research on preventive strategies. Pediatricians also believed that an important factor leading to diagnostic error was patients’ and caregivers’ failure to seek care in a timely manner. Strategies to empower caregivers and patients should be studied and implemented in the context of patient safety research. Physicians often do not know the outcomes of the patients whose diagnosis they miss, hence feedback pathways to relay changes in diagnosis back to the original physicians are likely to be useful.29
Our findings provide valuable data to inform ongoing patient safety and medical error training efforts. The lack of emphasis on formal training for diagnostic error provides future opportunities to redesign the curriculum for both future pediatricians and practicing physicians.39,40 For instance, both academic pediatricians and trainees believed that failure to gather available medical information through history and physical examination and/or review of previous charts was a common breakdown. It may be valuable to refocus education on this much neglected aspect of training through innovative techniques that use standardized or virtual patients and simulation.40–42 Training should also focus on the interplay of both systems and cognitive factors, which has thus far been underemphasized in the literature. Critical thinking skills and strategies to reduce cognitive biases should be taught in the context of teamwork and system-related factors (such as time pressure) that may affect diagnostic performance.43–47 Carefully designed forums for open discussions of diagnostic errors are needed and are likely to be well received; many residents self-reported a high number of diagnostic errors and most physicians reported that they discussed their own diagnostic errors with colleagues at least some of the time.
Our study had several limitations. Because we relied entirely on self-reported error data rather than actual errors and could not determine which reported errors caused patient harm, our findings may need to be validated through other data sources. However, obtaining physicians’ (and other caregivers’) perceptions about errors is important and widely accepted as a fundamental approach to understanding and improving safety.48,49 The content and design of our survey was based largely on adult literature, although we incorporated a significant amount of feedback from pediatricians prior to administration. Pediatricians also may not know they missed a diagnosis, so errors may be underreported. While our response rate was perhaps not high enough, we believe this does not jeopardize the quality or generalizability of our findings. Literature has shown that response rates from physicians have been lower than other participants and response rates have declined over time.50,51 Published surveys of physicians have a mean response rate of only 54%, versus 68% from non-physicians.50 Lastly, we only surveyed pediatricians through e-mail, which may have limited participation by those lacking reliable e-mail access; however, the trainees and academic pediatricians in our sample were expected to use institutional e-mail accounts, which would limit this bias for this sample.
In summary, pediatricians reported making diagnostic errors relatively frequently, and they endorsed inadequate data gathering, poor care coordination, and patient/caregiver-related delays as prominent contributing factors. Improved follow-up of patients and access to electronic health records were perceived as the most promising potential interventions. In contrast to previous literature, our findings may be more generalizable to routine practice and provide concrete targets for future training and interventions to prevent diagnostic errors in children.
Acknowledgments
Funding Source
The study was supported by an NIH K23 career development award (K23CA125585) to Dr. Singh, the Houston VA HSR&D Center of Excellence (HFP90-020) and in part by the Fulbright & Jaworski Educational Award.
These sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.
Footnotes
The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.
Data
All authors had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Reference List
- 1.Chandra A, Nundy S, Seabury SA. The growth of physician medical malpractice payments: evidence from the National Practitioner Data Bank. Health Aff (Millwood) 2005;(Suppl Web Exclusives):W5-240–W5-249. doi: 10.1377/hlthaff.w5.240. [DOI] [PubMed] [Google Scholar]
- 2.Gandhi TK, Kachalia A, Thomas EJ, et al. Missed and delayed diagnoses in the ambulatory setting: A study of closed malpractice claims. Ann Intern Med. 2006;145:488–496. doi: 10.7326/0003-4819-145-7-200610030-00006. [DOI] [PubMed] [Google Scholar]
- 3.Graber M. Diagnostic errors in medicine: a case of neglect. Jt Comm J Qual Patient Saf. 2005;31:106–113. doi: 10.1016/s1553-7250(05)31015-4. [DOI] [PubMed] [Google Scholar]
- 4.Graber ML, Franklin N, Gordon R. Diagnostic error in internal medicine. Arch Intern Med. 2005;165:1493–1499. doi: 10.1001/archinte.165.13.1493. [DOI] [PubMed] [Google Scholar]
- 5.Holohan TV, Colestro J, Grippi J, Converse J, Hughes M. Analysis of diagnostic error in paid malpractice claims with substandard care in a large healthcare system. South Med J. 2005;98:1083–1087. doi: 10.1097/01.smj.0000170729.51651.f7. [DOI] [PubMed] [Google Scholar]
- 6.Phillips RL, Jr, Bartholomew LA, Dovey SM, Fryer GE, Jr, Miyoshi TJ, Green LA. Learning from malpractice claims about negligent, adverse events in primary care in the United States. Qual Saf Health Care. 2004;13:121–126. doi: 10.1136/qshc.2003.008029. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Schiff GD, Kim S, Abrams R, Cosby K, Lambert B, Elstein AS. Diagnosing diagnosis errors: Lessons from a multi-institutional collaborative project. Advances in Patient Safety. 2005 [PubMed] [Google Scholar]
- 8.Singh H, Petersen LA, Thomas EJ. Understanding diagnostic errors in medicine: a lesson from aviation. Qual Saf Health Care. 2006;15:159–164. doi: 10.1136/qshc.2005.016444. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Singh H, Sethi S, Raber M, Petersen LA. Errors in cancer diagnosis: current understanding and future directions. J Clin Oncol. 2007;25:5009–5018. doi: 10.1200/JCO.2007.13.2142. [DOI] [PubMed] [Google Scholar]
- 10.Woods DM, Thomas EJ, Holl JL, Weiss KB, Brennan TA. Ambulatory care adverse events and preventable adverse events leading to a hospital admission. Quality and Safety in Healthcare. 2007 doi: 10.1136/qshc.2006.021147. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Institute of Medicine. To Err is Human: Building a Safer Health System. Washington, D.C: National Academies Press; 1999. [PubMed] [Google Scholar]
- 12.Newman-Toker DE, Pronovost PJ. Diagnostic Errors--The Next Frontier for Patient Safety. JAMA. 2009;301:1060–1062. doi: 10.1001/jama.2009.249. [DOI] [PubMed] [Google Scholar]
- 13.Carroll AE, Buddenbaum JL. Malpractice Claims Involving Pediatricians: Epidemiology and Etiology. Pediatrics. 2007;120:10–17. doi: 10.1542/peds.2006-3618. [DOI] [PubMed] [Google Scholar]
- 14.Thomas EJ, Petersen LA. Measuring errors and adverse events in health care. J Gen Intern Med. 2003;18:61–67. doi: 10.1046/j.1525-1497.2003.20147.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Croskerry P. AHRQ. Rockville, MD: 2005. Diagnostic failure: A cognitive and affective approach. Ref Type: Report. [PubMed] [Google Scholar]
- 16.Elstein AS. Heuristics and biases: selected errors in clinical reasoning. Acad Med. 1999;74:791–794. doi: 10.1097/00001888-199907000-00012. [DOI] [PubMed] [Google Scholar]
- 17.Hodder RJ, Ballal M, Selvachandran SN, Cade D. Variations in the evaluation of colorectal cancer risk. Colorectal Disease. 2005;7:254–262. doi: 10.1111/j.1463-1318.2005.00771.x. [DOI] [PubMed] [Google Scholar]
- 18.Kahneman D, Slovic P, Tversky A. Judgment under uncertainty: Heuristics and biases. Cambridge UK: Cambridge University Press; 1982. [Google Scholar]
- 19.Klein G. Sources of Power: How People Make Decisions. Cambridge, Massachusetts: The MIT Press; 1998. [Google Scholar]
- 20.McDonald CJ. Medical heuristics: the silent adjudicators of clinical practice. Ann Intern Med. 1996;124:56–62. doi: 10.7326/0003-4819-124-1_part_1-199601010-00009. [DOI] [PubMed] [Google Scholar]
- 21.Tanenbaum SJ. What physicians know. N Engl J Med. 1993;329:1268–1271. doi: 10.1056/NEJM199310213291713. [DOI] [PubMed] [Google Scholar]
- 22.Tversky A, Kahneman D. The framing of decisions and the psychology of choice. Science. 1981;211:453–458. doi: 10.1126/science.7455683. [DOI] [PubMed] [Google Scholar]
- 23.Voytovich AE, Rippey RM, Suffredini A. Premature conclusions in diagnostic reasoning. J Med Educ. 1985;60:302–307. doi: 10.1097/00001888-198504000-00004. [DOI] [PubMed] [Google Scholar]
- 24.Kassirer JP, Kopelman RI. Cognitive errors in diagnosis: instantiation, classification, and consequences. Am J Med. 1989;86:433–441. doi: 10.1016/0002-9343(89)90342-2. [DOI] [PubMed] [Google Scholar]
- 25.Sox HC, Blatt MA, Higgins MC, Marton KI. Medical Decision Making. Stoneham, MA: Butterworth-Heinemann; 1988. [Google Scholar]
- 26.Gandhi TK. Fumbled handoffs: one dropped ball after another. Ann Intern Med. 2005;142:352–358. doi: 10.7326/0003-4819-142-5-200503010-00010. [DOI] [PubMed] [Google Scholar]
- 27.Redelmeier DA. Improving patient care. The cognitive psychology of missed diagnoses. Ann Intern Med. 2005;142:115–120. doi: 10.7326/0003-4819-142-2-200501180-00010. [DOI] [PubMed] [Google Scholar]
- 28.Berner ES, Graber ML. Overconfidence as a Cause of Diagnostic Error in Medicine. The American Journal of Medicine. 2008;121:S2–S23. doi: 10.1016/j.amjmed.2008.01.001. [DOI] [PubMed] [Google Scholar]
- 29.Schiff GD. Minimizing Diagnostic Error: The Importance of Follow-up and Feedback. The American Journal of Medicine. 2008;121:S38–S42. doi: 10.1016/j.amjmed.2008.02.004. [DOI] [PubMed] [Google Scholar]
- 30.Singh H, Thomas E, Khan M, Petersen L. Identifying diagnostic errors in primary care using an electronic screening algorithm. Arch Intern Med. 2007;167:302–308. doi: 10.1001/archinte.167.3.302. [DOI] [PubMed] [Google Scholar]
- 31.Singh H, Weingart S. Diagnostic Errors in Ambulatory Care: Dimensions and Preventive Strategies. Advances in Health Sciences Education. doi: 10.1007/s10459-009-9177-z. In press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.McAbee GN, Donn SM, Mendelson RA, McDonnell WM, Gonzalez JL, Ake JK. Medical Diagnoses Commonly Associated With Pediatric Malpractice Lawsuits in the United States. Pediatrics. 2008;122:e1282–e1286. doi: 10.1542/peds.2008-1594. [DOI] [PubMed] [Google Scholar]
- 33.Selbst SM, Friedman MJ, Singh SB. Epidemiology and etiology of malpractice lawsuits involving children in US emergency departments and urgent care centers. Pediatr Emerg Care. 2005;21:165–169. [PubMed] [Google Scholar]
- 34.Schiff GD, Hasan O, Kim S, et al. Diagnostic Error in Medicine: Analysis of 583 Physician-Reported Errors. Arch Intern Med. 2009;169:1881–1887. doi: 10.1001/archinternmed.2009.333. [DOI] [PubMed] [Google Scholar]
- 35.Moreno C, Laje G, Blanco C, Jiang H, Schmidt AB, Olfson M. National Trends in the Outpatient Diagnosis and Treatment of Bipolar Disorder in Youth. Arch Gen Psychiatry. 2007;64:1032–1039. doi: 10.1001/archpsyc.64.9.1032. [DOI] [PubMed] [Google Scholar]
- 36.Singh H, Thomas EJ, Khan M, Petersen LA. Identifying diagnostic errors in primary care using an electronic screening algorithm. Arch Intern Med. 2007;167:302–308. doi: 10.1001/archinte.167.3.302. [DOI] [PubMed] [Google Scholar]
- 37.Hartzband P, Groopman J. Off the Record -- Avoiding the Pitfalls of Going Electronic. N Engl J Med. 2008;358:1656–1658. doi: 10.1056/NEJMp0802221. [DOI] [PubMed] [Google Scholar]
- 38.Graber ML. Taking Steps Towards a Safer Future: Measures to Promote Timely and Accurate Medical Diagnosis. The American Journal of Medicine. 2008;121:S43–S46. doi: 10.1016/j.amjmed.2008.02.006. [DOI] [PubMed] [Google Scholar]
- 39.Kuhn GJ. Diagnostic errors. Acad Emerg Med. 2002;9:740–750. doi: 10.1111/j.1553-2712.2002.tb02155.x. [DOI] [PubMed] [Google Scholar]
- 40.Wachter RM, Holmboe ES. Diagnostic Errors and Patient Safety. JAMA. 2009;302:258. doi: 10.1001/jama.2009.1006. [DOI] [PubMed] [Google Scholar]
- 41.Battles JB, Wilkinson SL, Lee SJ. Using standardised patients in an objective structured clinical examination as a patient safety tool. Qual Saf Health Care. 2004;13:i46–i50. doi: 10.1136/qshc.2004.009803. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Salas E, Wilson KA, Burke CS, Priest HA. Using simulation-based training to improve patient safety: what does it take? Jt Comm J Qual Patient Saf. 2005;31:363–371. doi: 10.1016/s1553-7250(05)31049-x. [DOI] [PubMed] [Google Scholar]
- 43.Croskerry P. Diagnostic failure: A cognitive and affective approach. Joint Commission on Accreditation of Healthcare Organizations. 5 A.D [Google Scholar]
- 44.Croskerry P, Wears RL, Binder LS. Setting the educational agenda and curriculum for error prevention in emergency medicine. Acad Emerg Med. 2000;7:1194–1200. doi: 10.1111/j.1553-2712.2000.tb00464.x. [DOI] [PubMed] [Google Scholar]
- 45.Croskerry P. The cognitive imperative: thinking about how we think. Acad Emerg Med. 2000;7:1223–1231. doi: 10.1111/j.1553-2712.2000.tb00467.x. [DOI] [PubMed] [Google Scholar]
- 46.Gaba DM, Howard SK, Fish KJ, Smith BE, Sowb YA. Simulation-based training in anesthesia crisis resource management (ACRM): a decade of experience. Simul Gaming. 2001;32:175–193. [Google Scholar]
- 47.Bond WF, Deitrick LM, Eberhardt M, et al. Cognitive versus technical debriefing after simulation training. Acad Emerg Med. 2006;13:276–283. doi: 10.1197/j.aem.2005.10.013. [DOI] [PubMed] [Google Scholar]
- 48.Sklar DP, Crandall CS, Zola T, Cunningham R. Emergency Physician Perceptions of Patient Safety Risks. Annals of Emergency Medicine. doi: 10.1016/j.annemergmed.2009.08.020. In Press, Corrected Proof. [DOI] [PubMed] [Google Scholar]
- 49.Vincent C. Patient safety. Edinburgh: Churchill Livingston; 2009. [Google Scholar]
- 50.Asch DA, Jedrziewski MK, Christakis NA. Response rates to mail surveys published in medical journals. J Clin Epidemiol. 1997;50:1129–1136. doi: 10.1016/s0895-4356(97)00126-1. [DOI] [PubMed] [Google Scholar]
- 51.McAvoy BR, Kaner EF. General practice postal surveys: a questionnaire too far? BMJ. 1996;313:732–733. doi: 10.1136/bmj.313.7059.732. [DOI] [PMC free article] [PubMed] [Google Scholar]