Primary care pediatricians’ interest in diagnostic error reduction

Abstract

Background

Diagnostic errors causing harm in children are understudied, resulting in a knowledge gap regarding pediatricians’ interest in reducing their incidence.

Methods

Electronic survey of general pediatricians focusing on diagnostic error incidence, errors they were interested in trying to improve, and errors reduced by their electronic health record (EHR).

Results

Of 300 contacted pediatricians, 77 (26%) responded, 58 (19%) served ambulatory patients, and 48 (16%) completed the entire questionnaire. Of these 48, 17 (35%) reported making a diagnostic error at least monthly, and 16 (33%) reported making a diagnostic error resulting in an adverse event at least annually. Pediatricians were “most” interested in “trying to improve” missed diagnosis of hypertension (17%), delayed diagnosis due to missed subspecialty referral (15%), and errors associated with delayed follow-up of abnormal laboratory values (13%). Among the 44 pediatricians with an EHR, 16 (36%) said it reduced the likelihood of missing obesity and 14 (32%) said it reduced the likelihood of missing hypertension. Also, 15 (34%) said it helped avoid delays in follow-up of abnormal laboratory values. A third (36%) reported no help in diagnostic error reduction from their EHR.

Conclusions

Pediatricians self-report an appreciable number of diagnostic errors and were most interested in preventing high frequency, non-life-threatening errors. There exists a need to leverage EHRs to support error reduction efforts.

Introduction

Although the 1999 Institute of Medicine (IOM) report “To Err is Human” [1] identified diagnostic errors as an important aspect of patient safety and quality improvement, they were examined closely in terms of prevalence, challenges and recommendations only recently in a 2015 IOM report [2]. Diagnostic errors cause substantial patient harm in the pediatric setting [3, 4]. In closed malpractice claims involving pediatricians, diagnostic errors were the most prevalent errors (32%) [3]. A 2010 survey of pediatricians and pediatric residents reported that 54% made a diagnostic error at least monthly and 45% made harmful errors at least annually [4]. Unfortunately, diagnostic errors remain understudied and cause appreciable patient harm [4–7].

Evidence-based approaches to diagnostic error reduction lag behind advances in other patient safety areas [8], and in pediatrics, the literature is scant. Of 109 studies included in a systematic review on interventions to reduce diagnostic errors, only eight focused on children [8].

It is unclear if lack of interest in diagnostic error reduction or lack of awareness of the magnitude of the problem drives this paucity of pediatric diagnostic error studies. Before designing an intervention-based research project to reduce ambulatory pediatric diagnostic errors, we investigated general pediatricians’ interest in participating in a diagnostic error reduction project, the types of errors they would be interested in “trying to improve” during such a project, the errors they were currently “trying to improve”, and the errors their electronic health record (EHR) “help[ed] to reduce”.

Materials and methods

We conducted an anonymous electronic survey using SurveyMonkey, of the American Academy of Pediatrics’ Quality Improvement Innovation Networks (QuIIN), a national group of pediatricians interested in improving the quality of care for children. When the questionnaire was administered in October 2013, QuIIN was comprised of 300 pediatricians in 46 states, practicing in diverse settings from single pediatrician private practices to large academic institutions with over 80 pediatricians. While the majority of QuIIN members are general primary care pediatricians (PCP), other pediatricians are able to join and receive listserve email messages. In this survey, pediatricians were excluded if they did not self-identify as “ambulatory physicians”, and were not asked any further questions. QuIIN directs over 39 current and completed pediatric quality improvement projects, providing an infrastructure for quality improvement through staff experienced in quality improvement methodology and organization. Interested practices sign up for projects that involve quality improvement collaboratives.

The questionnaire was developed by researchers with expertise in diagnostic errors and quality improvement, along with project managers from QuIIN. The questionnaire was pilot tested by five pediatricians on the QuIIN steering committee for readability, length, appropriateness of diagnostic errors of interest, and ease of understanding. Minimal changes were made to the questionnaire after pilot testing. The request to complete the questionnaire was emailed to QuIIN practitioners twice 7 days apart (see Supplemental Material for full questionnaire).

The questionnaire asked pediatricians 15 questions including (1) how often they made diagnostic errors, (2) if they were interested in participating in error reduction projects, (3) the errors if any, they would be interested in “trying to improve” during a QuIIN project, (4) the errors they were currently “trying to improve”, and (5) the errors their EHR “help[ed] to reduce.” For questions regarding frequency of errors, the questionnaire utilized exact questions from previously published work [4]. Similar to previously published work, we provided definitions of diagnostic errors (“delayed, wrong, or missed diagnosis”) and adverse events (“complication, injury, or harm to a patient resulting from medical management, and not from the patient’s underlying condition or disease”) to respondents [4]. Possible diagnostic errors of interest were chosen based on previously published reports of high frequency errors in claim and survey research [3, 4], and the feasibility to develop a quality improvement project related to the type of error. These errors included both process errors that did not involve a specific diagnosis but could lead to a diagnostic error (e.g. delayed follow-up of abnormal laboratory values), and specific errors of diagnosis (e.g. missed diagnosis of appendicitis). Diagnostic errors were presented as a list, and respondents were able to enter “other” errors they would be interested in “trying to improve” during a QuIIN project, and “other” diagnostic errors they were currently “trying to improve” (see Supplemental Material for full questionnaire). Data analysis was limited to descriptive statistics because of concerns that the small sample size would not allow credible hypothesis testing. This study was deemed exempt by the American Academy of Pediatrics’ Institutional Review Board.

Results

Of the 300 pediatricians on the QuIIN listserve, 77 (26%) responded. Of those, 58 (19%) provided care to ambulatory patients and 48 (16%) completed the entire questionnaire. Of the ten excluded pediatricians who completed part of the survey, all ten completed the practice setting question, six completed the EHR use question, three completed the interest in “participating in a QuIIN project to reduce diagnostic errors” question, and none completed the “most interested in trying to improve during a diagnostic errors QuIIN project” question. The 48 included pediatricians came from a variety of practice settings, with a wide range in the number of patients seen per year. Almost all practices (92%) used an EHR (Table 1).

Table 1.

Physician and practice characteristics.

Provider type (n = 48)
Ambulatory physician: private practice	33 (69%)
Ambulatory physician: academic medical center	14 (29%)
Ambulatory physician: private practice &   academic medical center	1 (2%)
Practice size (patients seen annually) (n = 40)a
Median	13,500
Interquartile range	6875, 25,000
Does practice use EHR (n = 48)
Yes	44 (92%)
No	4 (8%)
Level of interest in “participating in a QuIIN project to reduce pediatric diagnostic errors” (n = 48)
Very interested	26 (55%)
Interested	16 (33%)
Neutral	5 (10%)
Disinterested	1 (2%)
Very disinterested	0 (0%)

^aEight practices did not answer this question but did answer all others and were included in the analysis. EHR, electronic health record; QuIIN, quality improvement innovation networks.

Of the 48 respondents who provided ambulatory care to patients and completed the entire questionnaire, 17 (35%) reported making a diagnostic error at least monthly, regardless of harm to the patient, and 16 (33%) reported making a diagnostic error resulting in an adverse event at least annually. Twenty-six (55%) reported they would be “very interested” and 16 (33%) “somewhat interested” in “participating in a QuIIN project to reduce diagnostic errors”.

When pediatricians were asked to prioritize one of 15 acute and subacute errors they were “most interested in trying to improve” during a diagnostic errors QuIIN project, pediatricians most often selected errors related to the missed diagnosis of hypertension (17%), delayed diagnosis because of missed subspecialty referral (15%), or errors associated with delayed follow-up of abnormal laboratory values (13%). When able to express interest in any of the 15 errors, respondents selected “interested” or “very interested” in “trying to improve” errors involving missed diagnosis of adolescent depression (90%), missed diagnosis of developmental dysplasia of the hip (88%), and errors associated with delayed follow-up of abnormal laboratory values (87%) (Figure 1). Eleven respondents wrote in “other” diagnostic errors of interest, including two who wrote “missed diagnosis of developmental delay.”

Figure 1.

Diagnostic errors pediatricians were “very interested” or “interested” in “trying to improve”, currently “trying to improve” and ehrs “helps to reduce”.

EHR, electronic health record; Dx, diagnosis; DDH, developmental dysplasia of the hip; HTN, hypertension; Med, medication; +, positive; STI, sexually transmitted disease.

Most respondents (81%) reported that they were “currently trying to improve” at least one diagnostic error, with the most commonly cited error associated with delayed follow-up of abnormal laboratory values (33%). The errors with the greatest incongruence between “interest” in “trying to improve” versus “currently trying to improve” included missed developmental dysplasia of the hip (87% interested vs. 10% currently trying to improve), missed medication side effects (79% interested vs. 4% currently trying to improve), and missed adolescent depression (90% interested vs. 23% currently trying to improve). Nine respondents (19%) were not currently “trying to improve” any diagnostic errors.

Among 44 pediatricians with an EHR, 16 (36%) reported that their EHR “help[ed] to reduce” missed diagnosis of obesity, 15 (34%) reported their EHR “help[ed] to reduce” errors associated with delayed follow-up of abnormal laboratory values, and 14 (32%) reported their EHR “help[ed] to reduce” missed diagnosis of hypertension. The errors with the greatest incongruence between “interest” in “trying to improve” versus EHR “helps to reduce” included missed developmental dysplasia of the hip (87% interested vs. 5% EHR helps reduce), missed adolescent depression (90% interested vs. 9% EHR helps reduce), missed appendicitis (73% interested vs. 5% EHR helps reduce), and delayed diagnosis because of missed sub-specialty referral (81% interested vs. 14% EHR helps reduce). Sixteen pediatricians (36%) reported their EHR did not “help to reduce” any diagnostic errors.

Discussion

Pediatricians self-reported making harmful and non-harmful diagnostic errors at an appreciable frequency. Improvement projects on certain types of conditions, such as missed diagnosis of adolescent depression or hypertension, and failed processes, such as errors associated with delayed follow-up of abnormal laboratory values were of high interest for pediatricians. While the majority of pediatricians surveyed were “trying to improve” one or more diagnostic errors, their efforts were not congruent with the errors they were “interested in trying to improve”. Additionally, almost one fifth of respondents reported they were not “currently trying to improve” any diagnostic errors. These data likely overestimate the number of pediatricians involved with error reduction projects because the survey targeted members of a quality improvement network.

The percentage of pediatricians who reported making a diagnostic error regardless of harm at least monthly is similar to that of attending pediatricians in a previous study (35% in the present study vs. 39% in the previous study) [4]. This self-reported statistic, coupled with 33% of pediatricians reporting a diagnostic error resulting in an adverse event at least annually, suggests that many children are exposed to harmful diagnostic errors and provides an imperative to reduce these errors in primary care pediatrics. These data are even more concerning because research suggests physician overconfidence leads to underestimation of personal error rates [9].

In prior studies, meningitis was the most commonly cited diagnosis associated with diagnostic errors in pediatric malpractice claims [3], pediatricians ranked medication side effects as the second most commonly misdiagnosed [4], and pediatric-focused studies in a diagnostic error intervention systematic review [8] investigated acute pediatric complaints, pediatric trauma diagnoses, acute abdominal pain episodes, and cystic fibrosis screening. In our survey, which focused only on ambulatory pediatricians, a greater number were either “interested” or “very interested” in “trying to improve” higher frequency but sub-acute errors such as adolescent depression (90%) or hypertension (79%) rather than rarer but more acute errors such as missed meningitis (60%) or ankle fracture (52%). A recent multisite survey found that nearly half of pediatricians incorrectly diagnosed pre-hypertensive or hypertensive values in children as normal [10]. A prior study that asked pediatricians, including subspecialists, to rank conditions they believed were most commonly misdiagnosed also differed from the conditions general pediatricians were “interested” or “very interested” in “trying to improve” in this study. Highly ranked conditions in the prior study included viral illnesses diagnosed as bacterial illnesses, medication side effects, and psychiatric disorders [4]. This may suggest that in an outpatient setting, pediatricians are more concerned about and require more help with improving diagnostic errors related to sub-acute processes with potential for long-term harm. The authors hypothesize this may be due to the increased frequency of these errors in the primary care setting, the increased feasibility of conducting projects aimed at reducing subacute errors, and the additional checks which already exist in most health systems to prevent more acute errors.

This survey identified incongruence between pediatricians’ interests in diagnostic errors and 1) their current attempts to improve errors, and 2) their ability to reduce errors via EHR systems. Although the majority of pediatricians reported they were “currently trying to improve” one or more diagnostic errors, the percentage of pediatricians “currently trying to improve” each diagnostic error was lower than the percentage “interested” or “very interested” in “trying to improve [that specific>]… diagnostic error during a diagnostic error QuIIN project.” Furthermore, appreciably fewer pediatricians reported that their EHR systems “help to reduce” specific diagnostic errors than the number of pediatricians “interested” or “very interested” in “trying to improve [that specific]… diagnostic error during a diagnostic error QuIIN project”. This was especially evident for missed diagnosis of developmental dysplasia of the hip, missed diagnosis of adolescent depression, missed diagnosis of appendicitis, and delayed diagnosis because of missed sub-specialty referral. Pediatricians in a prior survey ranked EHR systems highly as “likely effective” to reduce diagnostic errors [4]. Taken together, these results suggest the pressing need for ambulatory diagnostic error reduction projects that correspond to pediatricians’ interests, and the potential benefit of leveraging EHR capabilities to reduce diagnostic errors.

Our study had several limitations. We surveyed QuIIN physicians, a network of pediatricians interested in quality improvement. It is possible the level of interest in “trying to improve” diagnostic errors and the percent of pediatricians already “trying to improve” diagnostic errors may be higher in this group than across all pediatricians. However, even among this motivated group, almost one fifth of respondents reported they were not “trying to improve” any diagnostic errors and one third reported making a diagnostic error resulting in an adverse event at least once a year. Additional limitations include reliance on self-reporting, which may have led to an underestimation of actual diagnostic error rates [9] and/or misclassification bias, because pediatricians may not have understood stated definitions of diagnostic errors or adverse events. The questionnaire’s introduction (see Supplemental Material) included results on self-reported diagnostic error frequencies from a prior survey [4], which may have created social desirability bias, with respondents feeling more comfortable reporting higher error rates. We calculated that our survey response rate was 26%, although it is unclear how many of the 300 email addresses on the QuIIN listserv were actually active and/or saw ambulatory patients. While the response rate could suggest a lack of pediatrician interest in diagnostic errors, 26% is consistent with the average response rate for email questionnaires [11–13]. Our sample consisted of pediatricians from several different settings across the United States, suggesting potential for generalizability. Finally, because there was not an option to describe potential errors due to EHRs in our questionnaire, we were unable to capture potential harms related to EHR use.

This survey complements previous research showing that pediatricians are making diagnostic errors at a high frequency. It suggests that pediatricians are interested in trying to prevent diagnostic errors, with a focus on subacute processes that may cause long-term harm. Further research is needed on what types of interventions may be useful in reducing harmful pediatric diagnostic errors.