Diagnostic Errors in Primary Care Pediatrics: Project RedDE

Abstract

Objective

Diagnostic errors (DEs), “the failure to establish an accurate and timely explanation of the patient’s health problem(s) or communicate that explanation,” cause appreciable morbidity, but are understudied in pediatrics. Pediatricians have expressed interest in reducing high-frequency/sub-acute DEs, but their epidemiology remains unknown. The objective of this study was to investigate the frequency of two high-frequency/sub-acute DEs and one missed opportunity for diagnosis (MOD) in primary care pediatrics.

Methods

As part of a national quality improvement collaborative, 25 primary care pediatric practices were randomized to collect five months of retrospective data on one DE or MOD: elevated blood pressure(BP) and abnormal laboratory values(DEs), or adolescent depression evaluation(MOD). Relationships between DE or MOD proportions and patient age, gender, and insurance status were explored with mixed-effects logistic regression models.

Results

DE or MOD rates in pediatric primary care were found to be 54% for patients with elevated BP(N=389), 11% for patients with abnormal laboratory values(N=381) and 62% for adolescents with an opportunity to evaluate for depression(N=400). When examining the number of times a pediatrician may have recognized an abnormal condition, but either knowingly or unknowingly did not act according to recommended guidelines: providers did not document recognition of an elevated BP in 51% of patients with elevated BP, and did not document recognition of an abnormal laboratory value without a delay in 9% of patients with abnormal laboratory values.

Conclusion

DEs and MODs occur at an appreciable frequency in pediatric primary care. These errors may contribute to care delays and patient harm.

Introduction

The Institute of Medicine (IOM) report, “Improving Diagnosis in Health Care, “highlights the significance of diagnostic errors (DE), and defines them as “the failure to establish an accurate and timely explanation of the patient’s health problem(s) or communicate that explanation to the patient.”¹ The report asserts that each of us will likely have a meaningful DE in our lifetime, with one estimate suggesting DEs affect 1 in 20 outpatient adults annually.² DEs are also responsible for approximately $34 billion dollars in annual United States malpractice payments.³ While studies on reducing ambulatory diagnostic breakdowns in adults have emerged,^{4, 5} little progress has been made to understand or reduce ambulatory pediatric DEs.⁶ In surveys, 35–54% of pediatricians reported a DE occurring at least monthly and 33–45% reported DEs that harmed a patient at least annually.^{7, 8} The true burden is likely higher, given that physicians generally underestimate their personal error rates.⁹ Research into pediatric ambulatory DEs remains in its infancy, additionally compounded by challenges in defining and measuring DEs.¹⁰ This study seeks to define and identify these errors across a broad range of pediatric ambulatory clinics.

When choosing which DEs to address, primary care pediatricians expressed more interest in working to reduce high-frequency/sub-acute DEs, such as missed hypertension diagnosis, versus low-frequency/acute DEs, such as missed appendicitis diagnosis.⁸ The epidemiology of these sub-acute DEs is unknown, but their high frequency and long term health effects may lead to increased morbidity and cost as compared to low-frequency/acute DEs. For example, adolescent depression affects 20% of adolescents before age 20, and 7.8% of adolescents attempt suicide.^{11– 16} Unfortunately, in 60 children with probable mental health diagnoses, only 15 (25%) were identified by pediatricians, and in only 14% did pediatricians consider a psychiatric referral.¹⁷ Similarly, 3–5% of children have hypertension.^{18, 19} The first step in diagnosing hypertension is recognizing when blood pressure (BP) is elevated. Recognition in both the hypertensive and pre-hypertensive range is important, as pre-hypertensive children are at greater risk for developing hypertension and have worse cardiovascular outcomes when compared to normotensive children.^{20, 21} In a single center study, 39% of pediatric visits included an elevated BP, but only 13% of these elevations were recognized by providers.²² Finally, 40% of ambulatory primary care visits include laboratory testing,²³ but 83% of physicians report at least one delay in reviewing laboratory results during the previous two months, and 40% report missing results despite a highly computerized health system. It is crucial to investigate the epidemiology of these high-frequency/sub-acute DEs across multiple practices to better describe the pervasiveness of pediatric DEs, as potentially many more patients are affected by this type of error. Rigorous epidemiologic multi-site studies can also increase generalizability of findings, demonstrate models for other DE measures, and create an imperative to reduce these errors.

To fill the knowledge gap surrounding the epidemiology of pediatric high-frequency/sub-acute DEs, pediatric primary care practices were randomly assigned to retrospectively investigate one DE or missed opportunities for diagnosis (MOD): elevated BP, and abnormal laboratory values (DEs) and adolescent depression evaluation (MOD). Our objective was to define these DEs and MOD and estimate their frequency in a multisite cohort, both foundational steps for reducing DE and MOD harm.

Methods

Study design

This analysis derives from a larger, ongoing quality improvement collaborative study, Project RedDE (Reducing Diagnostic Errors in Pediatric Primary Care), which aims to reduce DEs in primary care pediatric practices, in collaboration with the American Academy of Pediatrics’ (AAP) Quality Improvement Innovation Networks (QuIIN). Data presented in this study derive from Project RedDE’s baseline data collection before the start of intervention work. Pediatric practices recruited to participate in the error reduction collaborative were initially randomized to collect baseline data on one DE or MOD: elevated BP, abnormal laboratory values, or adolescent depression evaluation. Randomization allowed the creation of control groups during the intervention work, following the collection of baseline data described here.

Recruitment & randomization

In March 2015, we started recruiting practices through email list-serves, notices in quality improvement newsletters, and direct referrals. No compensation or inducement was provided to practices beyond the opportunity to participate in Project RedDE and improve their care delivery systems. Practices were stratified by two characteristics: university affiliation and presence of a self-reported prior record of working to reduce these DEs or MOD. Within the resulting stratum, practices were ranked by total annual visits per pediatric practitioner equivalents. Practices were then randomly assigned to one DE or MOD within each stratum. A total of 34 practices were randomized; 9 practices dropped out after randomization but before submitting data. Attrition of all 9 practices was related to the inability of these practices to collect necessary data to participate in the project. All 25 remaining practices submitted complete data for the project.

Data Collection Procedures

During July, August and September 2015, each practice collected five months of retrospective data from clinical encounters in February through June 2015, and entered data into an AAP online web portal. The Project RedDE conceptual model of preventing DEs was based on the premise that a medical diagnosis must first be entertained or “recognized” before the patient can be given that diagnosis.¹⁰ Therefore each DE has both a primary outcome (DE measure) and a secondary outcome(s) (recognition measure(s)). The MOD has a recognition measure only. Practices examined the first ten patients from each month who met inclusion criteria for their assigned DE or MOD (Table 1 and below). If practices had less than ten eligible patients in a given month, they entered all data available.

Table 1.

Inclusion Criteria and Outcomes for Diagnostic Errors and Missed Opportunity for Diagnosis of Interest

Diagnostic Error/Missed Opportunity for Diagnosis	Inclusion Criteria	Primary Outcome	Secondary Outcome(s)
Elevated Blood Pressure	≥3 years old at health supervision visit who had an elevated systolic or diastolic blood pressure	Patients without an appropriate action taken by the provider per 100 patients with elevated blood pressure	Patients without documentation that blood pressure was elevated, or appropriate action taken per 100 patients with elevated blood pressure Patients without documentation of blood pressure percentiles per 100 patients with elevated blood pressure
Abnormal Laboratory Values	Abnormal laboratory value*	Patients without an appropriate action taken without a delay per 100 patients with abnormal laboratory values	Patients without documentation that the laboratory value was abnormal, provided a diagnosis, or appropriate action taken per 100 patients with abnormal laboratory values
Adolescent Depression	≥11 years old at health supervision visit	Patients without documentation that provider pursued an evaluation for adolescent depression at visit per 100 adolescent health supervision visits	N/A

^±Appropriate actions included a) rechecking and documenting the blood pressure, b) noting a plan to recheck the blood pressure at a future visit, c) referring the patient to a specialist (e.g. nephrologist), or d) ordering laboratory or radiologic studies to further evaluate causes of elevated blood pressure (e.g. urine analysis, creatinine, echocardiogram, renal ultrasound, etc.).

^*See text for included abnormal laboratory values and for corresponding appropriate actions

Practices were educated on DE or MOD definitions in July 2015 during one hour webinars. They received slides and written definitions describing the measures, email listservs were available for questions, and the research team was available for clarifications. Practices were encouraged to examine notes, billing records, and problem lists for relevant documentation. Each practice selected one project administrator who was able to enter data into the central database. Chart review was performed by physicians, nurses and/or this administrator depending on the site. As noted below, charts were given a defined number of days after the patient’s visit before chart review was considered appropriate. For each eligible patient, practices recorded age, gender, and insurance status (public, private, self, unknown).

Diagnostic Error Definitions

DE measures must both measure DEs and be feasible for primary care pediatricians to collect. However, definitions and collection standards for DEs vary substantially in available literature, including DE definitional concepts such as “[diagnoses that are] missed, wrong or delayed,”²⁶ and “diagnostic process failures.”²⁷ We chose to use the recent DE definition put forth by the IOM which is necessarily broad,¹ encompasses all of these ideas, and recognizes the evolving, multifaceted nature of the diagnostic process. The concept of “missed opportunities for diagnosis (MOD)”²⁸ was adapted from previous work and defined to occur when evaluation for a diagnosis was not pursued despite a clear need to do so. The diagnostic concepts chosen involved failures at different stages of the diagnostic process: evaluation of symptoms (adolescent depression), evaluation of signs (elevated BP) and follow-up of diagnostic tests (abnormal laboratory values). Each DE or MOD chosen also met the following requirements: 1) they represent a prevalent underlying condition in children 2) evidence suggests the diagnoses are missed or delayed in primary care, 3) related DEs or MOD are harmful and can be prevented 4) primary care pediatricians are interested in reducing these DEs or MOD and 5) data collection is feasible. We anticipate that future studies will be able to refine our definitions, but as one of the first multicenter studies on ambulatory pediatric DEs or MOD, we hoped to gain an initial broad understanding of the scope of this problem using pragmatic measurement strategies.

DE or MOD rates identify patients who should have received a diagnosis or evaluation, respectively, but did not receive it. These data are presented as “percent of positive diagnoses or evaluations not made,” as opposed to “percent of total diagnoses or evaluations not made appropriately” (e.g. the number of children with elevated BP not correctly diagnosed per total number of children with elevated BP, as opposed to per total number of children seen with BP taken). This convention allows a focus on failures, a high reliability organization behavior which facilitates teams’ work to reduce errors following the baseline data collection period. Finally, definitions of “appropriate actions” indicating a diagnosis was made were necessarily broad as the study relied on front-line clinicians to collect data, and robust research-team led chart review was beyond the scope of this work.

Elevated blood pressure

Inclusion criteria for the elevated BP DE were patients ≥ 3 years old who had an elevated systolic or diastolic BP recorded at their health supervision visit. Elevated BPs were defined as ≥ 90^th percentile for age, height, and gender or ≥ 120 mmHg systolic or 80 mmHg diastolic at any age.²⁹ As described in The Fourth Report on the Diagnosis, Evaluation, and Treatment of High Blood Pressure in Children and Adolescents, “Adolescents with BP levels ≥120/80 mm Hg should be considered prehypertensive.”²⁹ The primary outcome measure was the number of patients with elevated BP, who did not have an appropriate action taken by the provider per 100 patients with elevated BP. This provider “appropriate action” confirms a diagnosis was made, as not all providers document a diagnosis specifically. Appropriate actions were purposefully broad, including a) rechecking and documenting the BP, b) noting a plan to recheck the BP at a future visit, c) referring the patient to a specialist, or d) ordering laboratory or radiologic studies to further evaluate causes of elevated BP. Laboratory or radiologic studies had to be ordered to specifically evaluate the BP (e.g. a urinalysis performed for a drug toxicology screen would not qualify). More than one action could be selected.

Secondary recognition outcome measures included the number of elevated BP patients in whom the provider did not document that the BP was elevated or did not take the appropriate action above. This measure captures the number of times a pediatrician may recognize an elevated BP, but either knowingly or unknowingly does not act according to recommendations or fails to take an action. Prior research suggests that even with electronic health records that flag abnormal BPs, providers still fail to recognize them in up to 58% of patients.²² Another secondary recognition outcome measure included the number of elevated BP patients without BP percentiles recorded.²⁹ This measure captures the number of times a pediatrician does not reference the sex-, height- and age-specific BP percentiles, an essential step in interpreting and recognizing an elevated BP. Pediatricians commonly document height and weight percentiles, but are less likely to document BP percentiles despite comparable challenges in interpreting normal versus abnormal values.³⁰

Abnormal laboratory values

Inclusion criteria for the abnormal laboratory values DE were patients with specific abnormal results: a) hemoglobin <11g/dL and mean corpuscular volume <75fL (i.e. microcytic anemia) in one and two year olds; b) lead >5μg/dL in one, two, and three year olds; c) any positive Neisseria gonorrhoeae, Chlamydia trachomatis, Treponema pallidum, or human immunodeficiency virus test in patients older than 10 years; d) positive group A streptococcal throat culture with negative rapid test in patients older than 1 year; e) thyroid stimulating hormone (TSH) <0.5μIU/mL or >4.5μIU/mL in patients one year and older. These five subacute laboratory results were selected because each is frequently ordered in primary care pediatric practices and can lead to harm if left unrecognized or untreated. These subacute tests are contrasted with critical laboratory results, such as a markedly elevated potassium, that are likely to already have existing safety systems to prevent follow-up delays. The primary outcome for this group was the number of patients who did not have an appropriate action documented after receiving any of these abnormal laboratory values, or had an appropriate action documented but with a delay per 100 patients with abnormal laboratory values. This provider “appropriate action” confirms an appropriate diagnosis was made, as not all providers document a diagnosis specifically. Appropriate actions were purposefully broad, including: a) starting iron, sending iron studies, or family conversation on dietary iron for microcytic anemia; b) family conversation on lead remediation or plan to retest for elevated lead levels; c) antibiotics started or referral to HIV specialist for positive sexually transmitted infections; d) antibiotics started or family conversation about positive test for positive group A streptococcal throat culture with negative rapid test; e) plan to repeat TSH test or referral to endocrinologist for elevated or reduced TSH. A delay was defined broadly as no appropriate action documented within 30 days for microcytic anemia and elevated lead levels, and within 7 days for the other laboratory results. As in the IOM definition, a DE can occur because the clinician fails to create a “timely explanation of the patient’s health problem”.¹

The secondary recognition outcome measure included the number of patient charts with abnormal laboratory values as defined above where the provider did not document that the result was abnormal or provide a diagnosis (e.g. anemia, syphilis, etc.), or did not document the appropriate action as above. This measure captures the number of times a pediatrician may recognize the diagnosis, but either knowingly or unknowingly does not act according to recommendations.

Adolescent depression

An appreciable body of literature indicates at least 10% of adolescents suffer from depression at any given time across many different settings and time periods, although in many studies this rate is higher.^12–16 Research also suggests that many adolescent depression cases are missed by primary providers suggesting under-diagnoses are vastly more of a problem than over diagnoses.^17,31,32 We first considered using documented depressive symptoms (e.g. poor school performance, interrupted sleep patterns, increased disruptive behaviors, etc.) without appropriate provider identification or referral to mental health evaluation as the primary outcome. However, this metric was not chosen because pilot data suggested the number of adolescents with documented symptoms suggestive of depression prior to a mental health referral was extremely small. This echoes research that pediatricians’ use of adolescent and parental chief complaints to identify depressive risk factors consistently under identifies adolescent depression.³¹ Thus, a MOD for adolescent depression evaluation was defined to occur when a provider did not pursue any evaluation for depression by either documenting concerns for depression or excluding concerns for depression at that health supervision visit. This MOD indicates whether providers took advantage of the health supervision visit to screen for depression, as recommended by the AAP and the United States Preventative Services Task Force, either with formal screening tools or clinical judgment. Given that one out of ten adolescents are depressed, this screen is of utmost importance.^12–16 Inclusion criteria were patients ≥ 11 years old who were seen for a health supervision visit. We also recorded the measured incidence of depression or sub-syndromal depression in these patients. Eligible adolescent charts were checked 30 days after the visit to allow time for clinicians to evaluate for the diagnosis.

Statistical Analysis

Descriptive statistics were used to describe central tendencies and frequencies for each of the three DEs or MOD separately. Patient ages were dichotomized based on a median split for each DE or MOD. To compare associations between age, gender, and insurance status with each of the DE or MOD proportions, three separate mixed-effects logistic regression models were used to take into account the hierarchical nature of the data structure, with month-specific and practice-specific intercepts considered random, while other factors considered fixed. The same approach was used to compare error proportions among the five laboratory tests of interest. As significant differences were seen between DE proportions for different laboratory tests, secondary analyses controlled for laboratory test. Practice-level characteristics were not included as covariates in the model since randomization occurred at the practice level stratified by potential confounders. However, the practice-level outcome variations potentially due to measured or unmeasured characteristics were taken into account in the practice-specific random intercept. All data analyses were completed with SAS v9.3. This study was approved by the AAP’s and the Albert Einstein College of Medicine’s Institutional Review Boards.

Results

Of the 25 practice sites that were randomized and submitted data, 8 were randomized to investigate elevated BP, 9 to abnormal laboratory values and 8 to adolescent depression evaluation (Table 2). All practices used an electronic health record and 1,170 total patients were included in this study across the three groups.

Table 2.

Demographics of Included Practices: N(%)

	Diagnostic Error
Characteristic	Adolescent Depression	Elevated Blood Pressure	Abnormal Laboratory Values	All
Number of Practices	8	8	9	25
University Affiliation	6 (75)	6 (75)	5 (56)	17 (68)
Previously worked “a lot” on one of the three errors of interest: Percent Yes	3 (38)	3 (38)	4 (44)	10 (40)
Total annual visits per full time physician or physician extender equivalents: Mean (sd)	3294 (1274)	3213 (1339)	3221 (1668)	3241 (1388)
Patient Demographics Percentage:
Mean (sd)
White, non-Hispanic/Latino	33 (19)	49 (21)	28 (11)	36 (19)
Hispanic/Latino origin	16 (9)	11 (5)	30 (18)	19 (15)
Black/African American	34 (25)	37 (20)	32 (23)	34 (22)
Asian	3 (4)	4 (3)	5 (4)	4 (4)
Native Hawaiian/other Pacific Islander	–	–	1 (3)	0.4 (2)
American Indian/Alaska Native	0.1 (0.4)	–	0.6 (1)	0.2 (0.7)
Other	15 (13)*	1 (4)	3 (6)	6 (10)

^*Two practices in this group had a large Somali population

Elevated blood pressure

For the eight practices randomized to DEs of elevated BP, 389 patients with either elevated systolic or diastolic BP were included. Two hundred and twelve patients did not receive an appropriate action (DE proportion 54%; range 21% to 96%). (Table 3) Of the 177 patients who did receive an appropriate action, 77% had their BP rechecked and documented, 20% had a plan to recheck the BP at a future visit, 9% were referred to a specialist, and 1% had additional laboratory or radiologic studies. Six percent received multiple actions. (Table 4) Among patients with elevated BP, 198 (51%) did not have documentation reflecting recognition of elevated BP or the appropriate action taken, and 230 patients (59%) did not have documentation of BP percentiles. In the mixed-effects model analysis, none of these outcomes was associated with age, gender, or insurance status.

Table 3.

Proportions of Diagnostic Errors (DE) or Missed Opportunity for Diagnosis (MOD)

Diagnostic Error or Missed Opportunity for Diagnosis	Number	DE or MOD proportion per 100 eligible patients
Elevated Blood Pressure
Primary DE Outcome: N=389 No appropriate action for elevated systolic or diastolic blood pressure	212	54%
Secondary Outcomes:  No documentation that blood pressure was elevated, or appropriate action taken	198	51%
No documentation of blood pressure percentiles	230	59%
Abnormal Laboratory Values
Primary DE Outcome: N=381 No appropriate action for abnormal laboratory value without delay	41	11%
Secondary Outcome:  No documentation that laboratory value was abnormal, provided a diagnosis, or appropriate action taken	33	9%
Adolescent Depression
Primary MOD Outcome: N=400  No documentation that provider pursued an evaluation for adolescent depression	249	62%

No proportions were associated with patient age, gender, or insurance status in mixed-effects analyses (laboratory models also controlled for laboratory test type) at a two-tailed alpha level of 0.05 except not pursuing an evaluation for adolescent depression was associated with younger dichotomized age.

Table 4.

Appropriate Blood Pressure Actions and Abnormal Laboratory Value Breakdown

Diagnostic Error	Category	N(%)
Elevated Blood Pressure N=389	177 patients received appropriate actions:
	BP rechecked	136 (77*)
	Plan to recheck BP at future visit	36 (20)
	Referred to specialist	16 (9)
	Additional laboratory or radiologic studies	1 (1)
	Multiple actions	11 (6)
Abnormal Laboratory Values N=381	41 patients (11%) did not receive an appropriate action without delay:
	Microcytic anemia	23 (17±)
	Elevated or reduced TSH tests	10 (30)
	Positive group A streptococcal throat culture with negative rapid tests	3 (5)
	Elevated lead levels	3 (4)
	Positive sexually transmitted infection tests	2 (3)

^*Percent of all patients with appropriate actions. Percentage is greater than 100% because patients could receive multiple actions.

^±Percent of patients with this abnormal laboratory value who did not receive an appropriate action without delay

Abnormal laboratory values

For the nine practices randomized to DEs for abnormal laboratory values, 381 patients with an abnormal laboratory value were included: 36% with microcytic anemia, 19% with elevated lead levels, 19% with a positive sexually transmitted infection test, 17% with a positive group A streptococcal throat culture with negative rapid test, and 9% with elevated or low TSH. Among patients with abnormal laboratory values, 41 did not have an appropriate action documented without delay (DE proportion 11%; range 0% to 42%). (Table 3) Specifically, 23 microcytic anemia values did not have an appropriate action documented without delay (17%), 3 elevated lead levels (4%), 2 positive sexually transmitted infection tests (3%), 3 positive group A streptococcal throat culture with negative rapid tests (5%), and 10 elevated or reduced TSH tests (30%). (Table 4) In the mixed effects models, abnormal laboratory value DE proportions were significantly different between laboratory tests (p=0.002) with abnormal TSH tests receiving the least follow-up.

Among patients with abnormal laboratory values, 33 patients (9%) had no documentation reflecting recognition of the abnormal laboratory value or an appropriate action documented without delay. In the mixed effects analysis controlling for laboratory test, none of these outcomes were associated with age, gender, or insurance status.

Adolescent depression

For the eight practices randomized to investigate MOD of adolescent depression evaluation, 400 adolescent health supervision visits were examined and providers did not pursue an evaluation for adolescent depression in 249 adolescent patients (62%; range 4% to 96%). (Table 3). Twenty-five patients (6.3%) had a documented diagnosis of depression or subsyndromal depression at this clinic visit. In the mixed-effects model analysis, not pursuing an evaluation for adolescent depression was associated with dichotomized age: 77% in 11 to 14 year olds versus 58% in 15 to 22 year olds (p=0.03). The incidence of recognized adolescent depression was not associated with age, gender, or insurance status.

Discussion

In one of the first studies to examine the frequency of pediatric DEs and MOD in a national multisite cohort, the high-frequency/sub-acute DE or MOD proportion in pediatric primary care was 54% for patients with elevated BP, 11% for patients with abnormal laboratory values and 62% for adolescents with an opportunity to evaluate for depression. DE proportions were not appreciably better when examining the number of times a pediatrician may have recognized an abnormal condition, but either knowingly or unknowingly did not act according to recommendations. These proportions are comparable to previous single center studies^17,22,25 and support the pressing need to reduce high-frequency/sub-acute DEs in pediatric primary care. Because the errors arise from three distinct diagnostic stages, each error can serve as a model for future pediatric DE reduction research in its respective domain.

By some measures, DEs are “the most common, most costly, and most dangerous of medical mistakes.”^3,35 Our prior work suggested that high-frequency/sub-acute DEs are of strong interest to primary care pediatricians, potentially more so than low frequency/acute errors like meningitis.⁸ The multi-center data presented here suggest that DEs and MOD for elevated BP, abnormal laboratory values, and adolescent depression evaluation occur at an appreciable frequency. Knowing that these DEs and MOD lead to long term morbidity,^{11–16, 20, 21} it is crucial to pursue strategies to reduce their incidence. Additionally, practices which enrolled in a quality improvement project to reduce DEs, are likely not representative of all practices given their expressed interest in DE improvement and their baseline characteristics. Almost 70% of our practices were university affiliated and 40% had already worked “a lot” on one of these errors. An electronic health record study of depression diagnoses in pediatric primary care suggested only 2.1% of adolescents carried a diagnosis of depression,³² while the comprehensive 2014 National Survey on Drug Use and Health suggested this incidence is 11.4%.¹⁶ Our group of practices presented a frequency of 6.3%, higher than the electronic health record study, but still leaving potentially at least one out of every three depressed adolescents unrecognized. Further, appropriate actions for both the elevated BP and abnormal laboratory value DEs were purposely broad, suggesting that some actions would be considered inappropriate if examined more closely. These issues may contribute to an underestimation of true DE or MOD proportions in the Project RedDE cohort when compared to other primary care pediatric practices.

In addition to learning from DEs and near misses, the IOM report on DEs¹ suggested seven other goals for reducing DEs, including improving health information technology support, establishing processes to identify DEs, and creating systems to reduce DEs. This study lays the foundation for metrics that can be used across various practice types to track DEs and MODs relevant to primary care. It also highlights subpopulations to target for DE or MOD reduction, such as younger adolescents, who were evaluated for depression at a lower proportion than older adolescents.^{11–15, 17} Additionally, abnormal TSH tests were more likely to have DEs than other pediatric primary care tests, suggesting this test may be a target for intervention.

Limitations of this study include potential variability in the application of data definitions across practices as no direct site visits or chart review verifications were performed by the research team. The QuIIN staff and the research team were available to answer questions during the data collection phase and all clarifications were shared across sites to increase adoption. Additionally, retrospective data contains a risk of misclassification and confirmation biases, as records may be incompletely reviewed, documentation may be incomplete, and/or providers may be looking for data to support or refute their practice model. The DE and MOD definitions used were necessarily broad in order to facilitate pragmatic data collection by front line providers. For this reason, we were unable to focus on the diagnosis of pediatric hypertension, and instead focused on the first step in the diagnostic process involving multiple medical team members: recognizing and diagnosing elevated BP. Future studies can and should further refine these definitions for improved accuracy and concept representation. A formal power analysis was not conducted for the baseline analysis of associations between age, gender, and insurance status with each of the DE or MOD proportions, and therefore type II error could be present. Practices were not necessarily representative of all United States pediatric practices, as 15% of patients were Hispanic/Latino and 22% were Black/African American. Finally, practices were asked to evaluate the first 10 patients’ charts that met inclusion criteria from each month. While this is not a randomized assignment for chart review, we believe it does reduce the potential for biased chart sampling. We cannot comment on whether patients accessing care in the beginning of a month differ from other patients.

Conclusion

We found DEs and MODs occur at an appreciable frequency in pediatric primary care. Practitioners should work to measure and reduce these sub-critical/high frequency errors, as they can lead to care delays and patient harm.

What’s New.

Primary care pediatric diagnostic errors and missed opportunities for diagnosis occur at appreciable rates: 54% for patients with elevated blood pressure, 11% for patients with abnormal laboratory values, and 62% for adolescents with an opportunity to evaluate for depression.