Effect of educational and electronic medical record interventions on food allergy management

Abstract

Background:

The growing prevalence of food allergies indicates a responsibility among primary care providers to ensure that their patients receive accurate diagnosis and management.

Objective:

To improve physician knowledge and management of food allergies by implementing educational and electronic medical record interventions.

Methods:

Pre- and posttest scores of pediatric residents and faculty were analyzed to assess the effectiveness of an educational session designed to improve knowledge of food allergy management. One year later, a best practice advisory was implemented in the electronic medical record to alert providers to consider allergy referral whenever a diagnosis code for food allergy or epinephrine autoinjector prescription was entered. A review of charts 6 months before and 6 months after each intervention was completed to determine the impact of both interventions. Outcome measurements included referrals to an allergy clinic, prescription of self-injectable epinephrine, and documentation that written emergency action plans were provided.

Results:

There was a significant increase in test scores immediately after the educational intervention (mean, 56.2 versus 84.3%; p < 0.001). Posttest scores remained significantly higher than preintervention scores 6 months later (mean score, 68.0 versus 56.2%; p = 0.006). Although knowledge improved, there was no significant difference in the percentage of patients who were provided allergy referral, were prescribed an epinephrine autoinjector, or were given an emergency action plan before and after both interventions.

Conclusion:

Neither intervention resulted in improvements in the management of children with food allergies at our pediatrics clinic. Further studies are needed to identify effective strategies to improve management of food allergies by primary care physicians.

Based on numerous studies to date, food allergy likely affects ∼8% of children, with evidence of an increasing prevalence.¹ The recent food allergy practice parameters emphasize the importance of accurately diagnosing immunoglobulin E (IgE) mediated food allergy and ensuring that other clinical syndromes are not misclassified as food allergy because management can vary for different disorders.² The severity of a food allergy reaction cannot be predicted by the severity of previous reactions, nor can it be definitively predicted by IgE levels or the size of a wheal produced via skin-prick testing.³ Therefore, self-care management, including strategies to minimize the risk of accidental exposures to food allergens, prescriptions for self-injectable epinephrine, and written action plans for treatment of allergic reactions are necessary.² For those at risk for anaphylaxis, consultation with an allergy specialist can be beneficial.²

The growing prevalence of food allergies among children and adults indicates a responsibility among primary care providers to ensure that their patients receive an accurate diagnosis and that the necessary education on medical management is provided. Rates of self-reported food allergy rose to 14.9% in 2006.⁴ However, despite this rise in self-reported food allergy, there was no significant difference in physician-diagnosed food allergy prevalence from 2001 to 2010,⁴ which indicates that an increasing number of people are avoiding foods without seeking proper medical advice.⁴.

Since 2008, physicians at Mount Sinai Hospital have been evaluating the way that food allergies are managed among general pediatricians in our institution. In a previous study, it was noted that 67% of children with physician-documented food allergies were not evaluated by an allergist due to a lack of referral by the patient's general pediatrician.⁵ In addition, although 80% of these children were prescribed epinephrine autoinjectors, only 38% had provision of emergency action plans documented in the charts. Therefore, the aim of this study was to evaluate the efficacy of two different interventions to improve the management of food allergies in our pediatric population, one being an educational intervention and the second being an electronic medical record (EMR) intervention.

METHODS

Educational Intervention

In August 2011, an educational intervention was developed to improve knowledge among providers in regard to food allergy management. A 20-minute lecture was prepared based on the National Institute of Allergy and Infectious Diseases (NIAID) sponsored Guidelines for the Diagnosis and Management of Food Allergy in the United States.³ This lecture discussed the prevalence and pathogenesis of food allergies and emphasized the need for an accurate diagnosis and to provide prescriptions for self-injectable epinephrine, written emergency action plans, and subspecialty referral for all patients diagnosed with food allergies. This lecture was provided by the same physician (I.H.-G.) daily for 1 week during the preclinic conference to ensure that all house staff and general pediatric faculty at the Pediatric Associates practice at Mount Sinai Hospital received the same educational session. To assess the efficacy of increasing knowledge among physicians, a pretest was given and then posttests were given immediately after the educational session and 3 and 6 months later. The test that was used included validated questions from The Chicago Food Allergy Research Survey for Primary Care Physicians.⁶

EMR Intervention

A second strategy to improve the management of patients with suspected food allergy was implemented on August 31, 2012. A best practice advisory (BPA) was activated in Epic, the EMR at Mount Sinai Hospital. Whenever food allergy was listed as a diagnosis code in the chart or an epinephrine autoinjector was prescribed, this BPA was triggered, which encouraged providers to consider allergy referral for the patient. The diagnostic codes (International Classification of Diseases, Ninth Revision [ICD-9]) that triggered the BPA included food allergy (693.1), personal history of allergy to peanut (v15.01), personal history of allergy to milk product (v15.02), personal history of allergy to egg (v15.03), personal history of allergy to seafood (v15.04), personal history of allergy to other foods (v15.05), and/or anaphylaxis (995). A link to a consult order was embedded within this BPA to facilitate the referral process.

Data Acquisition to Assess the Efficacy of the Educational and EMR Interventions

To assess the impact that the educational and EMR interventions had on physician practice, a retrospective chart review of medical records with the ICD-9 codes previously listed was completed. In addition, charts with epinephrine autoinjector prescriptions were included. Patients seen 6 months before and 6 months after each intervention were included in the study. Charts were reviewed for documentation of food allergy concerns, epinephrine autoinjector prescriptions, standardized emergency action plans, and referrals for allergy evaluation. This study was approved by the Icahn School of Medicine at Mount Sinai Institutional Review Board with a waiver of consent.

Statistical Methods

A comparison of continuous data was performed with a t-test. A comparison of categorical data was performed with χ² analysis with a two-tailed p value. A p value of <0.05 was considered significant. For analyses, SAS statistical software (SAS v9.3; SAS Institute Inc., Cary, NC) and GraphPad (GraphPad Software Inc., La Jolla, CA) were used. Generalized estimating equations was used to statistically compare the before and after proportions.

RESULTS

Physician Knowledge after the Educational Intervention

The educational intervention was successful at improving physician knowledge. There was a significant increase in test scores immediately after intervention (the mean score increased from 56.2 to 84.3%; p < 0.001) (Table 1). Fewer individuals completed the follow-up test 6 months after the intervention. However, these scores remained significantly higher than the preintervention scores (post- versus preintervention scores, 68.0 versus 56.2%; p = 0.006) (Table 1). Improvement in knowledge was similar between the residents and attending physicians.

Table 1.

The impact of the educational intervention on physician knowledge

Physician Practice after the Educational Intervention

Despite the initial success in improving physician knowledge, the intervention failed to bring about improvement in physician's management of food allergies. In the 6 months before the intervention, 86 patients with food allergy concerns were seen, and, in the 6 months after the intervention, 88 patients with food allergy concerns were seen. There was no difference in the percentage of patients who were prescribed an epinephrine autoinjector or given an emergency action plan before and after the intervention (Table 2). In addition, there was no difference in the percentage of patients referred for allergy evaluation (64.8 versus 58.1%; p = 0.44) (Table 2).

Table 2.

The impact of the educational intervention on physician practices

Physician Practice after the EMR Intervention

Similar to the results of the educational intervention, there was no difference in the percentage of patients who were referred for allergy evaluation before and after the EMR intervention (Table 3). In addition, there was no difference in the percentage of patients prescribed an epinephrine autoinjector or given an emergency action plan (Table 3).

Table 3.

The impact of the electronic medical record intervention on physician practices

DISCUSSION

Based on the data collected, both the educational and EMR interventions failed to bring about a significant change in the way general pediatricians at our clinic manage patients with food allergies. The rates of allergy referrals, epinephrine prescriptions, and emergency action plans at our clinic were similar to a recent study of general pediatricians, which showed that 67% of children diagnosed with food allergies are referred to an allergist, 45% are given a prescription for injectable epinephrine, and emergency action plans are provided to 6%.⁶ On average, pediatricians comply with the NIAID food allergy management guidelines in two of five areas. Pediatricians who participated in the study mentioned above stated that the poor adherence rates are often due to a lack of familiarity with management guidelines and uncertainty about the pediatrician's role in food allergy management.⁶

Although physician knowledge was noted to increase as a result of the educational intervention, this increased knowledge did not improve physician practice. Potential reasons for the lack of change in physician practice may be issues of time constraints and difficulty navigating within the EMR. A review of the literature showed that many physicians view EMRs to be challenging because of many screens and options that lead to a high degree of complexity, which requires a large dedication of time and effort to master the system efficiently.⁷ As a result, many physicians find EMRs to be burdensome. By simplifying the steps within the EMR, we may be able to increase physician compliance and lead to better outcomes for patients.

Electronic alerts, also referred to as BPAs or clinical decision support systems, have had mixed effects on providers' prescribing practices. Studies that analyzed whether BPAs in EMRs improve physician practices provide contradictory results.^8–10 However, results of those studies revealed positive effects of BPAs. For example, one study showed that, in certain practices, BPAs led to a doubling in percentage of physician's ordering of influenza vaccinations.¹⁰ Such previous success of BPAs in improving patient outcomes led to our rationale for implementing an EMR intervention. In this case, there were several potential factors that led to a suboptimal BPA intervention. First, the timing of such alerts must be optimized to open toward the end rather than the initiation of the encounter. This will cut down on the multiple distractors during a well-child visit that may interfere with cognitive linking of the BPA to action. Second, several alerts are triggered during a well-child visit, which leads to the well-described entity of “alert fatigue,” in which important alerts are ignored along with unimportant ones.⁸ Overriding BPAs is a common occurrence. Chart review from several studies has demonstrated that 49 to 96% of drug alerts are overridden.⁸ In addition, physician surveys revealed that 40% of prescribers indicate that they override drug-drug interactions most of the time or always.⁹

To improve the impact of a BPA, it must be triggered at a proper time, present clear information, and be nondisruptive to the work flow. Although some studies showed that passive BPAs, such as the one we investigated, can effect change, those that require a single-step action are most effective.^10–12 Passive BPAs such as ours, which require multiple steps, are more likely to be ignored, very often due to time constraints and the condition of alert fatigue.⁸ Although the allergy referral order was linked to our BPA, additional steps are required for physicians to prescribe injectable epinephrine and to provide emergency action plans.

There are other alerts, known as “hard stops,” which do require immediate action from the provider. These have been shown to be more likely to effect change on the physician documentation level but tend to aggravate physicians and may lead to delay in ordering necessary medications.¹¹ For example, one study evaluated the effect of a “hard stop” within the emergency department that was designed to increase the rate of human immunodeficiency virus (HIV) testing. This hard stop, which was linked with an HIV order set to obtain laboratory studies for patients, increased documentation of offering HIV testing drastically, although it only led to a modest increase in the percentage of patients who actually were tested for HIV (1.9% pre- versus 5.8% postintervention).¹² Such alerts, although effective, can be very distracting and should only be used for mandatory physician actions because hard stops halts workflow. Continued evaluation of all alerts encountered may serve as targets to reduce the number of alerts triggered during an encounter. Several EMRs have begun to group several orders and coding triggers together. For example, if vancomycin is ordered, then a vancomycin trough is automatically populated in the order set, which allows for improved order efficiency. A similar target for food allergy coding may improve compliance with allergy referral, epinephrine prescriptions, and provision of allergy action plans.

There were several limitations to this study. First, this was a single-center study. As a chart review study, medical records were assumed to be complete; however, this may not always be the case. In addition, our sample size was limited to the New York City area, and results may not be generalizable to other practices. The majority of our patient population lived in Harlem and the Bronx; 74% of our patients receive managed Medicaid, 51% of our patients were Hispanic or Latino, and 36% were African American. A notable increase in patient volume occurred during the study period (2011–2013), which is mainly due to Mount Sinai Hospital becoming the primary care affiliate for several community organizations. Notably, the rates of food allergy diagnosis, allergy referrals for food allergy, epinephrine autoinjector prescriptions, and provision of emergency action plans remained consistent.

CONCLUSION

Both the educational and EMR interventions failed to bring about improvement in the management of children with food allergies at our general pediatrics clinic. Analysis of the results of this study indicated that further research is needed to understand the motivations of physicians to refer patients to allergists. In addition, we must determine what barriers and constraints impede referrals to allergy specialists. The patients seen at our facility are largely minorities of low socioeconomic status. It is possible that providers' beliefs about their patient's family situations and work status may impact the providers' decision to refer patients to subspecialists.

It is important that primary care providers adequately educate patients with food allergies, including preparedness in case of accidental ingestion and reaction. Although some patients may receive referral to an allergy specialist to address these points, not all patients follow through with the referral; thus, it is crucial that general practitioners provide an early foundation. Given the busy nature of medical clinics and complexity of EMRs, we must continue to find ways to educate providers and to use the EMR to make referrals, medication orders, and emergency action plans easier and less time consuming.