Development and validation of educational materials for food allergy

Abstract

Objective

To develop and validate a food allergy educational program.

Study design

Materials developed through focus groups, parental and expert review were submitted to 60 parents of newly referred children having a prior food allergy diagnosis and an epinephrine autoinjector. The main outcome was correct demonstration of an autoinjector.

Results

The correct number of autoinjector activation steps increased from 3.4 to 5.95 (of 6) after training (p<.001) and was 5.47 at 1 year (p<.05). The mean score for comfort with using the autoinjector (7 point Likert scale) before the curriculum was 4.63 (somewhat comfortable) and increased to 6.23 after the intervention (p<.05) and remained elevated at 1 year (6.03). Knowledge tests (maximum 15) increased from a mean score of 9.2 to 12.4 (p<.001) at the initial visit and remained at 12.7 at 1 year. The annualized rate of allergic reactions fell from 1.77 (historical) the year prior, to 0.42 (p<.001) after the program. On a 7 point Likert-scale, all satisfaction categories remained above a favorable mean score of 6: straight-forward, organized, interesting, relevant, and recommend to others.

Conclusions

This food allergy educational curriculum for parents, now available online at no cost, showed high levels of satisfaction and efficacy.

Food allergy affects 4–8% of children,^(1–3) and allergic reactions to foods can be severe and potentially fatal.^(4;5) Management requires avoidance of the offending food(s) and readiness to administer self-injectable epinephrine for anaphylaxis.⁽⁴⁾ Food allergen avoidance has many pitfalls because diligence is required for every meal and snack. Families must carefully read ingredient labels of manufactured products,⁽⁶⁾ take care in ordering foods in restaurants,⁽⁷⁾ understand how to avoid cross contact of safe foods with allergens and must incorporate strategies to avoid allergens at home, school, and other outings.⁽⁸⁾ Accidental ingestion leading to reactions is a reality for most children with food allergy.^(9;10) Treatment with epinephrine is often warranted, but not executed for a variety of reasons.⁽¹¹⁾ Prior focus group and survey studies have identified parental knowledge base deficits, particularly regarding triggers, environmental risks and perceptions of susceptibility.^(12;13) Improved education about management is suggested by the National Institutes of Allergy and Infectious Diseases (NIAID)-sponsored US food allergy guidelines.⁽⁴⁾

The Consortium of Food Allergy Research, sponsored by the NIAID, is a multi-center program addressing immune, genetic and environmental determinants of food allergy, and novel treatment strategies. Recognizing a need for improved education,⁽¹⁴⁾ the program also included development of educational resources. A preliminary study⁽¹⁵⁾ disclosed parental preferences and priorities for an educational curriculum. In this report, we review the steps used to create and validate a food allergy educational curriculum for families based on the preferences identified in the prior study. This curriculum is currently available by public access.

METHODS

Initial Drafting of Educational Materials

We drafted educational materials based upon the experience of experts including the authors who are pediatric allergists (SHS, SMJ) with over 35 years of food allergy management experience, dietitians working with children having food allergies (LC, MEG), nurses working with families with food allergies (SN, SKC) and a behavioral psychologist (PV). To determine the preferred design, content and form of educational materials, we held 4 focus groups (n=36) with parents of children with food allergies. The results of this sub-study were previously published.⁽¹⁵⁾ Briefly, parents preferred brief materials available in print and via the internet, with video formats as well. Topics of emphasis included signs/symptoms of food allergy, details about “cross-contact” when an allergen may contaminate an otherwise safe food in a variety of circumstances, label-reading, when and how to use self-injectable epinephrine, materials to address emotional and social aspects of management, and to have an overall “roadmap” for the curriculum. To inspire confidence, the parents indicated that the educational curriculum should be endorsed by highly respected academic and/or research institutions. The investigators used this information to draft a core set of materials (Table I).

Table 1.

Specific Educational Materials (there is also an educational video)

General Category	Individual information forms
Introduction	Basics for the newly diagnosed Checklist (guide to learning)
Understanding specific disorders	Anaphylaxis Proctocolitis (blood stools in infants) Food protein-induced enterocolitis Eosinophilic esophagitis
Avoiding allergens	Reading ingredient labels Food-specific information (separate forms for: egg, milk, peanut, tree nuts, soy, wheat, fish, shellfish, sesame)
Managing in and outside of the home	Preparing allergen-safe meals at home Avoiding cross-contact Restaurants Schools Childcare Summer camp
Living a safe and healthy life	Nutritional issues How to introduce new foods How to prevent or delay allergy Maintaining a healthy lifestyle

Development of specific informational forms

The initial drafts of the instructional materials in Table I were reviewed by each of the authors and also by a convenience sample of pediatric allergists specializing in food allergy (n=3). Edited drafts were then reviewed by a convenience sample of parents with children having food allergies attending allergy clinics in the two centers performing the study (Mount Sinai School of Medicine in New York, NY and Arkansas Children’s Hospital, Little Rock, AR). These participants were asked to evaluate each individual informational sheet and to provide written feedback with suggestions and comments. Edits were made based upon this advice to form the final set of materials. To address the parental request for a video format, a video was produced to review the materials in the curriculum, highlighting main points about food allergy management.

Longitudinal evaluation

The final materials were presented to a convenience sample of families presenting for a first visit to our allergy practices/clinics. Entry criteria required that the child was previously diagnosed with a food allergy and prescribed self-injectable epinephrine, but had not been previously evaluated at these centers. The parent was asked to complete a 15 question food allergy knowledge test constructed by the authors (SHS, MEG) to test key knowledge about anaphylaxis recognition, medication use, labeling, allergy definitions, allergen avoidance, and reaction treatment. Parents also answered a brief survey that included demographic information, comfort in using epinephrine (7 point Likert scale), a survey regarding past experience and satisfaction with educational materials, and to demonstrate the use of an epinephrine self-injector using a trainer device. The demonstration evaluated 6 steps (recognizing the device [i.e., not attempting to read the label for clues on how to use it], removing the lock-cap, selecting the appropriate body site, pressing the correct end of device to the body, applying pressure to activate it, and holding it in place for several seconds rather than jabbing and immediately removing it).⁽¹⁶⁾ After a food allergy diagnosis was confirmed, a single investigator at each site (MG or LC) observed and scored the injector demonstration, showed the participant the video and reviewed the pertinent educational materials and provided them to the family. This was followed by a repeat administration of the tests and surveys. The families were then provided with a survey to gauge their satisfaction with the materials. After completing all assessments, parents were instructed to contact the investigators if there were any allergic reactions and were called 6 months after the program was administered to review any allergic reactions using a structured questionnaire.⁽¹⁷⁾ At one year, they were re-evaluated with the same tests and surveys in person, if possible, or by telephone. The study was approved by the Institutional Review Boards of Mount Sinai School of Medicine and Arkansas Children’s Hospital, and all participants provided signed consent.

Statistics

The primary endpoint was epinephrine injector technique based upon a prior study⁽¹⁶⁾ that indicated that only 32% of parents correctly demonstrate administration of self-injectable epinephrine upon presentation to an allergist following a prior prescription. We hypothesized that education would increase this to at least 64% (double) and a sample size of 50 would have 85% power to detect this difference. This sample size would also have >80% power to detect additional endpoints identified prior to enrollment that were based upon prior studies regarding comfort levels with using epinephrine and reduction in rate of allergic reactions.^(18;19) Our secondary endpoints were increased comfort with using epinephrine by 30% and a reduction in allergic reactions by 50% compared with the year prior. Anticipating a drop-out rate of 17%, we recruited 60 families. Paired data were analyzed using Wilcoxon signed-rank test for paired samples. Chi square was used for categorical data. Two-tailed tests were used. Data were analyzed with SPSS 16.0 (2007; SPSS Inc, Chicago, Ill). A p-value <.05 was considered significant.

RESULTS

Initial draft of educational materials and revisions

The initial draft of materials was reviewed by 32 parents of children with food allergies. There were 29 mothers, 3 fathers, 24 white, 1 Asian, 7 African-American, and one of Hispanic ethnicity. The foods being avoided included peanut (27) egg (23), milk (16), tree nuts (18), shellfish (12), fish (8), wheat (6), soy (3), and sesame (5). Comments were received on all of the forms, some with specific advice and some with general comments. Overall, 152 comments were logged. Examples of comments included general positive reactions (“I like how it stressed to inform caregivers”, “I wish I had this when my child was diagnosed”), general advice (“baking without egg is a pain, you should describe substitutes”, “Use more laymen’s terms,” “use bold font for main points”), and specific changes (“add more alternative flours to the list,” “add wipe eating surfaces in the restaurant,” “add that soy is found in processed meats”). Each comment was reviewed by all authors and revisions were made by consensus. Almost all suggestions were followed. Some examples where changes were not made included indicating that a family should call the manufacturer of all products (no matter what is stated on the label) for concern of unintended ingredients. It was felt that this directive would be too burdensome on families with little evidence of need.⁽⁶⁾ Another example is a request to describe how to discuss food allergies with other parents at a school. Trying to address this suggestion was considered too complex for a brief informational sheet, so additional resources were listed on the school and camp informational sheets.

Longitudinal study

Sixty families were recruited (30 from each site). The mean age of the participating children was 3.6 years (range, 6 months to 14 years). The number of years spent managing food allergy was >2 (43%), 1–2 (15%) and <1 (42%). All participants had completed high school and 67% had earned at least a college degree. Race and ethnicity were as follows: 81% white, 17% African-American, 2% Asian, and 3% Hispanic. Family income was <$40,000 (28%), $40–80,000 (17%) and > $80,000 (55%). The numbers avoiding various foods were: peanut (n=41), egg (n=32), milk (n=30), tree nuts (n=26), wheat (n=15), soy (n=12), shellfish (n=8), fish (n=8). Fifty two families participated in the one year follow up (87%); 19 of them completed the materials via telephone or mail and 33 in person.

Autoinjector competency and comfort

Autoinjector competency was evaluated pre- and post-training. Only 18% correctly performed all 6 steps prior to training, and this increased to 95% after training (mean scores increased from 3.4 to 5.95, p<.001). The specific deficits are shown in the Figure. Among the 33 participants who were seen in person at 1 year, the mean score declined to 5.47, which remained significantly higher than baseline. The decrease in score was accounted for primarily by 2 sub-scores that decreased significantly from post-baseline, for recognizing the device (i.e., not attempting to read the label for clues on how to use it) (p<.004) and holding it in place for several seconds (p<.03). Participants were asked to rate their comfort with using epinephrine on their child, from 1 (not comfortable at all) to 7 (totally comfortable). The mean comfort score before the educational session was 4.63 (somewhat comfortable) and increased to 6.23 after the intervention (p<.05) and remained elevated at 1 year (6.03).

Food Allergy knowledge and reaction rates

The specific questions, answers and percent correct responses to knowledge based questions are shown in Table II. Knowledge tests increased from a mean score of 9.2 to 12.4 (p<.001) at initial visit and were maintained at 12.7 at the 12 month follow up. Participants were queried with regard to the number of allergic reactions that were experienced in the year prior to introducing the educational materials. A total of 65% of the participants had experienced at least 1 reaction, with an annualized rate (number of reactions per year per participant) of 1.77. In the year following the educational curriculum, the rates fell to 32% having experienced a reaction, with an annualized rate of 0.42 per person per year (p<.001).

Table 2.

Results of knowledge quiz

Food Allergy Knowledge Quiz Questions %	% Correct Pre (n=60)	% Correct Post (n=60)*	% Correct 1 yr (n=52)
The EpiPen© (or Twinject©) should be refrigerated, if possible, when not in use. (F)	88	98	100
The EpiPen© (or Twinject©) can be injected through clothing. (T)	75	98	98
“May contain” labels are required by law if a product may have had contact with an allergen. (F)	22	48	64
Epinephrine (adrenaline) side effects are more severe if a child is given it when he or she is not really having anaphylaxis. (F)	55	87	91
An anaphylactic reaction can get better after treatment and then come back again an hour or two later. (T)	72	85	91
Antihistamines (like Benadryl©, Zyrtec©, Allegra©) reverse wheezing during anaphylaxis. (F)	60	77	81
Anaphylaxis typically develops 1–2 hours after a food is eaten. (F)	67	87	89
“Food intolerance” is another way of saying “food allergy.” (F)	63	83	81
Food allergy can be severe, but not fatal. (F)	83	90	94
In the US, all ingredients in processed foods must be labeled. (F)	28	58	64
According to US labeling laws, food labels must use plain English to say if the food contains “major allergens” including milk, egg, wheat, soy, peanut, tree nuts, fish and Crustacean shellfish. (T)	82	98	96
US labeling laws require food allergens to be disclosed in medications. (F)	30	73	58
A self-serve buffet is safe for a person with food allergies if each food has its own serving spoon or fork. (F)	93	100	100
A child who ate a food to which he/she is allergic and now has coughing, a hoarse voice, trouble swallowing should be given an antihistamine, watched closely, and if the problem continues more than 5 minutes or worsens should then receive injected epinephrine. (F)	35	58	74
Hand sanitizers (such as Purell©) are useful ways to remove allergens from hands. (F)	63	93	89

Knowledge tests increased from a mean score of 9.2 to 12.4 (p < 0.001) at initial visit and maintained at 12.7 at the 12 month follow up visit.

Satisfaction and self-reported learning

After their exposure to the curriculum, participants were asked to assess their satisfaction. When asked to compare the materials with any materials they had received when their child was first diagnosed, the materials in this program were rated on a 4 point scale as much better (70%), better (30%), or same/worse (0%). Overall, after providing the curriculum initially, participants described that they learned (on a 4 point scale) a lot (58%), some (35%) a little (7%) or nothing (0%). A year later, 77% indicated they learned a lot, 19% some and 4% a little, representing a significant increase in those reporting having learned a lot (p<.05). Table III summarizes the results of additional queries regarding satisfaction with the materials.

Table 3.

Summary of satisfaction with the materials

Feature	Score (mean/highest possible) at initial presentation (n=60)	Score (mean/highest possible) at 1 year (n=52)
Overall Program Assessment (4 point Likert)
Learning (a lot-4 to nothing-1)	3.5/4	3.7/4
Overall satisfaction (very satisfied-4 to very dissatisfied-1)	3.5/4	3.7/4
Material was (7 point Likert):
Straightforward (7) to Difficult (1)	6.6/7	6.5/7
Very well organized (7) to Poorly organized (1)	6.5/7	6.4/7
Interesting (7) to Boring (1)	6.4/7	6.5/7
Useful (7) to Useless (1)	6.6/7	6.5/7
Recommendation for others
Highly recommend (7) to not recommend (1)	6.8/7	6.7/7

DISCUSSION

We designed a food allergy curriculum based upon initial assessment of parental needs and preferences,⁽¹⁵⁾ drafting by experts, with revisions undertaken after additional review by parents and experts. We validated the materials, showing improvement in technique of using epinephrine injectors, increased comfort with treatment, improvement in knowledge about food allergy, and overall satisfaction with the materials. These benefits were substantially maintained at 1 year, with a noted reduction in allergic reactions compared with the year prior to study entry. These materials are available free of charge at http://www.cofargroup.org/ at the Food Allergy Educational Program of the Consortium of Food Allergy Research.

Living with food allergy is difficult. Quality of life is impaired for patients and their families.^(20;21) There is evidence that difficulty in accessing educational resources is one factor adding to distress.⁽²²⁾ Recent studies regarding food allergic concerns for schools,⁽²³⁾ restaurants,⁽²⁴⁾ and for reading product ingredient labels⁽²⁵⁾ conclude that more education is needed to improve patient confidence and adherence. Unfortunately, our recent needs assessment surveys of nurses, pediatricians and dietitians^(14;26;27) and the work of others^(12;13) show numerous deficits and unmet needs with regard to education for children with food allergies. Taken together, the curriculum developed here could fill a significant educational gap.

Participants in the longitudinal study had all seen a physician and had been diagnosed with food allergy and prescribed self-injectable epinephrine prior to being evaluated in the study. However, they had poor ability to activate the medication they had been prescribed and had deficits in knowledge and comfort indicating a low likelihood that they could have properly responded to an anaphylaxis emergency. This suggests a deficit in education, because when exposed to our curriculum, there were significant changes that persisted after a year. Therefore, we conclude that we have demonstrated both a gap in treatment and value of the educational program, but also have observed the need of physicians to ensure their patients are educated on these topics.

There were two areas in the educational program that did not meet our fullest expectations. The first regards understanding of US labeling laws. The 3 survey questions that continued to score below 70% correct concern details of food labeling. Our prior studies showed that families make assumptions about labeling that are probably based upon common sense perceptions, but do not match the law.⁽²⁸⁾ The educational materials in this program provide all of the details about the law, but poor scores suggest a need to ensure that patients’ attention is focused on these details. Improved labeling laws would be another means to address this gap. The second area of weakness was that epinephrine self-injector demonstration scores decreased after 1 year. The score reduction was accounted for by the subject taking a pause to familiarize themselves with the device and for not holding the device in place for several seconds. The observation underscores the need to practice and review the technique of administration at least yearly.

There are some limitations to the study and to the development of the curriculum. Participants may have scored the materials favorably to please the study coordinators, although this bias may be tempered by the fact that initial parental review of the drafts resulted in criticisms that were addressed. Additionally, we did not approach this as a clinical trial, so there was no control group. The study was designed using a quasi-experimental design to create a comprehensive curriculum and validate its utility based upon participant feedback, improved knowledge and satisfaction and these goals were achieved. Definitive validation and evaluation would require an experimental double-blind study design. It is notable that prior studies evaluating similar outcomes provide insights into the validity of our results. For example, we relied upon historical rates of reactions reported for the year prior to exposure to the curriculum. The annualized rate of allergic reactions among young children receiving “standard” advice in food allergy specialty centers was previously reported as 0.81/year,⁽¹⁷⁾ whereas it was half that in our year one follow up assessment after exposure to the curriculum (reduced from 1.77 to 0.42 reactions per year), supporting our conclusion that the education was effective for reducing the rate of allergic reactions.

Another limitation of the study is the generalizability of the results. We focused on participants presenting to two food allergy centers. These patients probably represent those with more severe food allergies. However, it must be noted that the participants had not been to a food allergy center prior to contact with this program. Nonetheless, their referral to a center may indicate a more severe disease state. The efficacy of the curriculum for this group suggests that families with less severe food allergies could equally benefit from the curriculum. The population was diverse with regard to ages, allergies, and socioeconomic status, but only included English speaking people, high school graduates, and primarily mothers. Compared with a recent US prevalence study of children with food allergies using an internet based survey that included Spanish-speaking families,⁽³⁾ our participants were wealthier (28% in our study under $40,000/yr versus 46% under $50,000) and there were fewer Hispanics (2% versus 21%). Additional study of the program’s effectiveness in other populations is warranted, including translation into Spanish, validation among non-English speaking cohorts, and inclusion of more fathers as primary caretakers. We believe the curriculum can be modified for education of additional stakeholders, such as school personnel. Lastly, it should be noted that the program did not design its own Food Allergy Emergency Plan/Care Plan, but relied upon the form that is available through the Food Allergy & Anaphylaxis Network (www.foodallergy.org). This additional component is referred to in the curriculum.

In conclusion, we developed a curriculum that addresses the educational needs expressed by parents of food-allergic children. The materials received high satisfaction ratings and were effective in improving knowledge, technique of use of self-injectable epinephrine, and for reducing allergic reactions; it is currently available without charge for use by any stakeholder. Additional study and validation is warranted, and future studies may also focus upon materials for children and other caregivers.

Figure.

Auto-injector competency scores. Overall score was statistically significantly increased from baseline and maintained at 12 months.