Development of a Tool Predicting Severity of Allergic Reaction During Peanut Challenge

Abstract

Background

Reliable prognostic markers for predicting severity of allergic reactions during oral food challenges (OFC) have not been established.

Objective

We sought to develop a predictive algorithm of a food challenge severity score (CSS) to identify those at higher risk for severe reactions to a standardized peanut OFC.

Methods

Medical history and allergy tests were obtained for 120 peanut-allergic participants who underwent double-blind, placebo-controlled food challenges (DBPCFCs). Reactions were assigned a CSS between 1 to 6 based on cumulative tolerated dose and a “severity clinical indicator.” Demographic characteristics, clinical features, peanut component IgE values, and a basophil activation marker were considered in a multi-step analysis to derive a flexible decision rule to understand risk during peanut of OFC.

Results

18.3% participants had a severe reaction (CSS >4). The decision rule identified the following three variables (in order of importance) as predictors of reaction severity: ratio of %CD63hi stimulation with peanut to %CD63hi anti-IgE (CD63 ratio), history of exercise-induced asthma, and forced expiratory volume in 1 sec/forced vital capacity (FEV1/FVC) ratio. The CD63 ratio alone was a strong predictor of CSS (p<0.001).

Conclusion

The CSS is a novel tool that combines dose thresholds and allergic reactions to understand risks associated with peanut OFCs. Lab-values (CD63 ratio), along with clinical variables (exercise-induced asthma and FEV1/FVC ratio) contribute to the predictive ability of the severity of reaction to peanut OFC. Further testing of this decision rule is needed in a larger external data source before it can be considered outside of research settings.

INTRODUCTION

Epidemiologic studies indicate that the prevalence of food allergies, especially to peanuts, has increased worldwide in the last few decades.^1–3 Food and peanut allergies are now estimated to affect about 6–10% and 1–3% of the overall population, respectively. Among children with food allergies, about 30% of food-allergic individuals have allergies to multiple foods.^4–7 Emergency department visits and hospitalizations due to food-induced anaphylaxis have significantly increased in recent years⁸ and, in the US, these now number approximately 200,000 per year with further increases expected.⁹ The risk of a severe reaction is higher for those with peanut and tree-nut allergies and it is estimated that more than 50% of these patients experience at least one severe allergic reaction.⁴

The diagnostic approach to food allergies can be complex.^{10, 11} Skin prick tests (SPTs) and measurements of food allergen-specific immunoglobulin E (IgE) are commonly performed; however, these tests are not definitive because they lack specificity and result in a high number of false-positives.^{10, 11} The current gold standard for confirming or excluding a food allergy is the double-blind, placebo-controlled food challenge (DBPCFC).^{10, 11} However, DBPCFCs carry the risk of systemic and potentially life-threatening reactions, including severe anaphylaxis, requiring hospitalization or intensive care.¹²

The development of clinically useful tools for stratifying patients according to potential risk of severe reactions during peanut OFC would therefore be of great clinical value. Predictive models using multivariate logistic regression analysis already have been developed for egg, wheat, and milk allergy and these are promising in efforts to stratify risk for severity of allergic reaction during peanut OFC with these foods.^13–15 For peanut allergy, certain biomarkers have been associated with reaction severity during peanut OFCs, such as isolated sensitization to Ara h 2, epitope diversity (i.e., extent of IgE binding to the combination of Ara h 1, 2, and 3), and results of changes in levels of basophil CD63 or CD203c staining in basophil activation tests (BATs) upon ex vivo activation of basophils with peanut allergen.^16–19 In this study, we sought to develop a predictive algorithm to identify individuals with peanut allergy who are at higher risk for severe reactions during standardized peanut OFC. To do this, we tested data obtained from participants eligible for a large phase 2, randomized clinical study using peanut oral immunotherapy, including well-characterized clinical data and the participant’s responses to validated and standardized peanut OFCs, as well as results of complete blood counts, SPTs, BATs, measurements of peanut-specific IgE (sIgE) and IgG₄ (sIgG₄), total IgE (tIgE) and peanut component resolved diagnostics (CRDs) (sIgE to Ara h 1, 2, 3, 8, 9). This permitted us to define biomarkers, which we incorporated into a unique food challenge severity score (CSS) and a predictive algorithm to identify those individuals at higher risk for severe reactions to a standardized peanut OFC.

METHODS

Study population and preliminary visit

All aspects of the study were authorized by the Stanford University School of Medicine Institutional Review Board (Stanford, CA) and were overseen in collaboration with NIAID and the DAIT Data Safety Monitoring Board. One hundred and twenty patients between the ages of 7 to 55 years with a convincing history of peanut allergy were recruited to undergo standardized DBPCFCs at the Sean N. Parker Center for Allergy and Asthma Research at Stanford University as part of screening for clinical trial enrollment (POISED: Peanut Oral Immunotherapy: Safety, Efficacy and Discovery; Clinical Trials Identifier: NCT02103270). A detailed medical history, physical examination, spirometry, SPT, sIgE and tIgE testing, BATs, and CRDs were completed at screening. Patients with a prior history of a peanut-allergic reaction requiring intubation or eliciting hypotension were excluded, while patients with previous reactions to peanut requiring epinephrine for other severe symptoms were eligible.

DBPCFCs

Standardized DBPCFCs to peanut and oat (placebo) were performed according to validated guidelines.^{11, 20, 21} A normal physical examination and Forced Expiratory Volume in 1 second (FEV₁) ≥ 80% were confirmed prior to the start of each OFC. The protocol consisted of 7 escalating doses (5, 20, 50, 100, 100, 100, 125 mg) of the food protein in flour form, concealed in either applesauce or pudding, ingested by the participant every 15 minutes, as tolerated. Vital signs and pertinent physical examinations were repeated every 15 minutes, or more frequently at the discretion of the clinician. If a positive allergic reaction was observed during OFC^{20, 22}, the OFC was immediately discontinued and appropriate treatments were administered as per guidelines and published procedures.^{20, 23, 24} Reaction severity was scored according to a combination of Bock and Practall grading scale (range 0–3) ^22,24 and assigned a severity clinical indicator (SCI) of negative or positive. A positive SCI categorized those participants who had any of the following reactions: severe urticaria (involving more than 2/3 of body surface area), severe/disabling abdominal pain, vomiting and diarrhea in severe abdominal symptoms, any wheezing, any voice changes (airway obstruction), or hypotension. A negative SCI score was assigned for milder reactions. None of the patients in this cohort experienced voice changes or hypotension. To evaluate for sensitivity to peanut protein quantitatively, the cumulative tolerated dose (CTD; the amount of peanut ingested prior to treating the participant for objective symptoms) was also quantified.

Food Challenge Severity Score

We developed a clinical severity scoring system, with scores between 1 and 6. This new scoring system combines two important factors, the SCI and the CTD, as detailed in Table 1; e.g., CSS 1 indicates a mild reaction on ingestion of a large amount of peanut and CSS 6 indicates a severe reaction following ingestion of a small amount of peanut.

Table 1.

Distribution of Challenge Severity Score

Cumulative Tolerated Dose (mg)	Severity Challenge Indicator(SCI)	Challenge Severity Score (CSS)	Number of Subjects (%)
175 – 375	Negative	1	15 (12.5%)
25 – 75	Negative	2	44 (36.7%)
< 25	Negative	3	39 (32.5%)
175 – 375	Positive	4	9 (7.5%)
25 – 75	Positive	5	12 (10%)
< 25	Positive	6	1 (0.8%)

Note: SCI is a measure of reaction severity.

Basophil Activation Tests

BATs were performed as previously described.²⁵ Whole blood was collected in heparin tubes, kept at 4°C for 24 hours, and then stimulated for 30 minutes with peanut (prepared from Byrd Mill peanut flour) as previously described ²⁵ (peanut concentrations of 1000, 100, 10, 1, and 0.1 ng/mL) at 37°C. Anti-IgE, IL-3, or Roswell Park Memorial Institute (RPMI) media alone were used as controls. Cells were stained with CD203c, CD63, CD123, and HLA-DR. Basophils were gated as CD123⁺HLA-DR⁻. Basophil expression of CD63 and CD203c was analyzed with FlowJo (Ashland, Oregon). In total, we assessed 15 BAT variables as presented in eTable 1. As previously reported, the following forms of CD203c and CD63 were evaluated: the ratio of CD203c stimulation with peanut to CD203c RPMI (negative control), %CD63^hi stimulation with peanut, and the ratio of %CD63^hi stimulation with peanut to %CD63^hi anti-IgE.²⁶ The ratio of the percentages of CD63^hi basophils after stimulation with varying concentrations of peanut vs. anti-IgE (CD63 ratio) were also included to assess the magnitude of the response.²⁵

Statistical analysis and variable selection process

Descriptive statistics such as means, medians, standard deviations and interquartile ranges were used to compare baseline characteristics by CSS. Due to only one subject with an assigned CSS of 6, scores 5 and 6 were collapsed when reporting demographics. Graphical tools were used to assess distributional aspects. Log-transformations were applied when normality assumptions were violated. Kruskal-Wallis and Fisher’s exact tests were used to draw comparisons on CSS by continuous and categorical characteristics, respectively.

Our variable selection decisions are outlined in eFigure 1. Among the 94 available clinical, laboratory, and demographic features, self-reported overall and detailed allergic rhinitis variables were excluded (n=7) due to non-concordance against clinical testing (eTable 2), as our interest was in arriving at an objective decision rule. Further, absolute counts were included for analysis from the CBC (excluded n=4 variables).

We then took a three-staged approached to arriving at our model. In the first stage, one BAT variable out of 15 possible BATs was pre-selected using an Area under the Curve (AUC), as most laboratories would have resources to measure one BAT (either CD203c or CD63) following stimulation with one concentration of peanut. Logistic regression models were fit to SCI as a function of each log-transformed BAT, and the BAT with the highest AUC was selected to be included as a potential predictor in the second stage of the process.

In the second stage, variables of importance for CSS were identified from the subset of 69 clinical, laboratory, and demographic features, including the pre-selected BAT from the first stage, using conditional random forest (RF) approaches.²⁷ Conditional RF methods use a permutation-based significance testing procedure to select variables as opposed to the default approach that utilizes the Gini statistic, avoiding the bias of selecting variables that have many possible splits found in traditional RF methods.²⁸ Variables were chosen as relevant if their importance value was above the absolute value of the lowest negative-scoring variable.²⁹

In the third stage, a clinically interpretable decision rule using classification and regression trees (CART) was derived from the subset of variables deemed important via RF in the second stage. CART is a tree-based learning technique for classifying observations.³⁰

To compare the results derived from variable selection via RF and prediction via CART to more widely used statistical methods, the least absolute shrinkage and selection operator (LASSO) was applied to the 69 pre-selection features. LASSO is a regression analysis method that performs both variable selection and regularization in order to gain higher predictive ability of the final fitted model.³¹ The Root Mean Squared Error (RMSE) from CART and the LASSO regression model were compared.

Two supplementary analyses were also performed. First, CSS was fit as a function of log-transformed CD63 ratio alone in a linear regression model, and the change in CSS and p-value were reported. Second, four logistic regression models were fit to an indicator for whether or not a subject reacted to the peanut OFCs at or before 175mg as a function of the CD63 ratio, Ara h 2, sIgE, and SPT, respectively. A cutoff of 175mg peanut protein was chosen to be comparable to other published literature, looking at a dose less than a full peanut (240mg).^{32, 33} Youden’s index was used to determine a threshold for each predictor which maximized the AUC.³⁴ The threshold, and AUC (95% CI) were reported for each of the four predictors.

The entire three-stage process and LASSO were internally validated using repeated 10-fold cross validation as commonly performed ³⁵. All analyses were conducted using R, version 3.2.2.^{36, 37} Additional details of statistical methods can be found in eMethods.

RESULTS

Total study population

The median participant age was 11 years. Baseline demographics are reported in Table 2. Eighty-two percent of the cohort was under 18 years of age; 33% of the cohort was female. Participants had comorbid atopic conditions including other food allergies (70%), asthma (67%), atopic dermatitis (73%), and allergic rhinitis (AR) (75%).

Table 2.

Baseline demographics

	Challenge Severity Score (CSS)
	1 (n=15)	2 (n=44)	3 (n=39)	4 (n=9)	5,6† (n=13)	Total (N=120)	P-value*
Age at enrollment (y), median (IRQ)	9 (4)	11 (7)	11 (13)	13 (18)	9 (2)	11 (7)	0.10
Child (less than 18 yo), no. (%)	14 (93.3)	38 (86.4)	28 (71.8)	6 (66.7)	12 (92.3)	98 (81.7)	0.15
Female, no. (%)	6 (40.0)	8 (18.2)	17 (43.6)	4 (44.4)	4 (30.8)	39 (32.5)	0.10
Race, no. (%)
Caucasian	14 (93.3)	34 (77.3)	21 (53.9)	7 (77.8)	9 (69.2)	85 (70.8)	0.04
Asian	3 (20.0)	11 (25.0)	22 (56.4)	2 (22.2)	4 (30.8)	42 (35.0)	0.02
Black or African American	0	3 (6.8)	0	0	0	3 (2.5)	0.48
Native Hawaiian or Pacific Islander	0	0	1 (2.6)	0	0	1 (0.8)	0.63
Not Hispanic or Latino, no. (%)	15 (100.0)	42 (95.5)	39 (100.0)	9 (100.0)	12 (92.3)	117 (97.5)	0.43
BMI (kg/m2), mean (SD)	17.8 (3.2)	18.6 (4.7)	20.3 (6.0)	20.5 (3.4)	17.7 (2.5)	19.1 (4.8)	0.12
Number of food allergies, mean (SD)	3.1 (2.5)	3.9 (2.5)	4.0 (2.8)	3.0 (2.3)	3.6 (2.7)	3.7 (2.6)	0.71
Subjects with food allergy to more than one food, n (%)	11 (73.3)	33 (75.0)	27 (69.2)	5 (55.6)	8 (61.5)	84 (70.0)	0.72
Asthma and allergy history, no. (%)
Asthma	6 (40.0)	26 (59.1)	30 (76.9)	7 (77.9)	11 (84.6)	80 (66.7)	0.04
Age of onset (y), mean (SD)	2.9 (1.9)	3.6 (2.7)	3.7 (3.2)	4.5 (3.3)	3.2 (2.1)	3.6 (2.8)	0.91
AD	11 (73.3)	33 (75.0)	28 (71.8)	4 (44.4)	11 (84.6)	87 (72.5)	0.37
Age of onset (y), mean (SD)	0.8 (1.4)	0.9 (1.2)	2.6 (5.8)	0.5 (0.4)	0.6 (0.8)	1.4 (3.5)	0.83
AR	11 (73.3)	34 (77.3)	28 (71.8)	8 (88.9)	9 (69.2)	90 (75.0)	0.84
Age of onset (y), mean (SD)	5.5 (2.2)	5.1 (4.0)	5.7 (3.6)	5.9 (3.3)	3.3 (1.6)	5.2 (3.5)	0.19

AD, Atopic dermatitis; AR, allergic rhinitis; BMI, body mass indicator

^*Kruskal-Wallis and Fisher’s exact tests were used to calculate p-values for continuous and categorical variables by CSS, respectively.

^†Collapsed CSS 5 and 6 due to only 1 subject in the CSS 6 category.

Characteristics of study population and CSS outcomes

OFCs were discontinued after objective signs or symptoms of an allergic reaction were observed. Symptoms ranged from mild urticaria to more severe symptoms such as wheezing or diffuse urticaria. The mean CSS was 2.7 with a standard deviation of 1.2. The majority (82%) of subjects were assigned a CSS of 1–3, indicating less severe reactions in the DBPCFC (Table 1). All challenge reactions were treated with no sequelae. We observed no biphasic reactions. Eighteen children (7–17 years) and 4 adults (18–53 years) had more severe scores (CSS greater than 3). Of the 22 participants with a more severe CSS, 50% had wheezing, 45% had severe abdominal pain, and 5% had severe urticaria (eTable 3). Each of the 120 participants developed an objective allergic reaction to a CTD of ≤ 500 mg peanut protein during their DBPCFC. The median CTD was 75.0 mg.

A higher proportion of Caucasians were in the lowest CSS group (CSS 1, 93.3%, p = 0.04, Table 2). There were more patients with a history of asthma in the highest CSS group (CSS 5–6, 84.6%, p= 0.04), however, per our protocol, DBPCFCs were only conducted if FEV₁% was ≥ 80%. Consequently, mean FEV₁% was 95.5% and mean FEV₁/FVC ratios 84% (Table 3). There was no evidence to suggest that peanut SPT average wheal size, sIgE, tIgE, CRD tests, or complete blood counts were significantly different across CSS levels (Table 3).

Table 3.

Clinical and lab characteristics

	Challenge Severity Score (CSS)
	1 (n=15)	2 (n=44)	3 (n=39)	4 (n=9)	5,6‡ (n=13)	Total (N=120)	P-value *
FEV1 % at DBPCFC, mean (SD)	94.5 (10.1)	97.5 (12.9)	94.7 (11.2)	91.1 (7.6)	94.7 (13.1)	95.5 (11.7)	0.72
FEV1/FVC at DBPCFC, mean (SD)	0.86 (0.07)	0.86 (0.07)	0.82 (0.07)	0.80 (0.08)	0.82 (0.06)	0.84 (0.07)	0.02
Peanut wheal size (mm), mean (SD)	12.2 (7.45)	13.3 (6.4)	13.4 (5.6)	12.7 (5.8)	13.5 (7.1)	13.3 (6.1)	0.96
Complete Blood Count, mean (SD)
Absolute Basophil Count	0.05 (0.03)	0.04 (0.03)	0.04 (0.02)	0.04 (0.02)	0.04 (0.01)	0.04 (0.02)	0.90
Absolute Neutrophil Count	3.3 (1.2)	2.9 (1.3)	3.2 (1.4)	3.7 (1.1)	3.2 (1.8)	3.1 (1.4)	0.17
Absolute Lymphocyte Count	2.4 (0.7)	2.2 (0.6)	2.2 (0.6)	2.4 (0.8)	2.2 (0.5)	2.2 (0.6)	0.78
Absolute Monocyte Count	0.5 (0.2)	0.5 (0.2)	0.5 (0.2)	0.5 (0.1)	0.5 (0.2)	0.5 (0.2)	0.92
Absolute Eosinophil Count	0.5 (0.2)	0.4 (0.3)	0.4 (0.2)	0.4 (0.3)	0.4 (0.2)	0.4 (0.2)	0.61
CD203c Basophil Activation Test, mean (SD) †
PF 0.1	1.28 (0.40)	1.14 (0.15)	1.20 (0.18)	1.42 (0.24)	1.17 (0.17)	1.20 (0.22)	0.01
PF 1	1.28 (0.27)	1.28 (0.22)	1.34 (0.25)	1.49 (0.27)	1.38 (0.24)	1.32 (0.24)	0.09
PF 10	1.52 (0.52)	1.66 (0.36)	1.66 (0.40)	1.88 (0.42)	1.72 (0.40)	1.67 (0.40)	0.11
PF 100	1.74 (0.66)	1.85 (0.49)	1.86 (0.51)	2.02 (0.44)	1.91 (0.36)	1.86 (0.50)	0.44
PF 1000	1.73 (0.63)	1.82 (0.48)	1.75 (0.47)	1.97 (0.44)	1.86 (0.28)	1.80 (0.47)	0.39
%CD63high Basophil Activation Test, median (IQR)†
PF 0.1	0.48 (1.02)	0.60 (1.24)	0.54 (0.81)	0.45 (0.66)	0.90 (1.18)	0.55 (1.25)	0.75
PF 1	0.31 (0.62)	1.66 (4.35)	2.21 (3.24)	2.06 (3.35)	4.83 (5.83)	1.92 (4.20)	0.01
PF 10	3.33 (6.63)	10.70 (25.97)	14.20 (18.37)	12.60 (15.80)	24.40 (19.80)	11.90 (20.83)	0.004
PF 100	10.50 (25.61)	24.00 (31.68)	20.50 (30.99)	9.59 (18.70)	36.80 (22.50)	22.40 (29.48)	0.02
PF 1000	6.82 (22.54)	14.85 (25.04)	13.50 (13.63)	11.30 (17.68)	14.90 (16.79)	13.80 (18.85)	0.24
PF10/%CD63high anti-IgE	0.46 (1.59)	0.70 (1.42)	1.51 (1.84)	2.42 (7.57)	1.93 (5.07)	1.21 (1.82)	0.01
Antibodies (ng/L), median (IQR)
Peanut IgG4 (ug/L)	430.0 (1870.0)	1110.0 (2345.0)	640.0 (930.0)	450.0 (620.0)	1600.0 (1220.0)	830 (1,600)	0.05
Peanut IgE	77.1 (277.7)	178.9 (412.5)	193.0 (478.0)	21.9 (217.3)	383.1 (614.6)	192.9 (458.0)	0.13
Total IgE	1342.0 (2364.4)	1321.3 (2526.6)	1149.2 (1766.6)	592.9 (1461.6)	1351.8 (2249.7)	1284.7 (2379)	0.49
Peanut components (ng/L), median (IQR)
Ara h 1	27.3 (166.3)	21.4 (115.1)	57.1 (191.2)	3.6 (15.5)	179.6 (212.5)	34.9 (177.8)	0.18
Ara h 2	39.3 (107.8)	70.6 (209.5)	87.1 (197.9)	12.8 (140.9)	196.7 (216.9)	85.0 (207.1)	0.09
Ara h 3	0.9 (37.2)	3.2 (22.2)	9.7 (32.2)	1.1 (5.3)	25.9 (39.3)	5.7 (38.6)	0.31
Ara h 8	0.0 (2.5)	0.0 (2.5)	0.0 (1.6)	0.0 (0.6)	0.0 (0.4)	0.0 (1.8)	0.95
Ara h 9	0.0 (0.0)	0.0 (0.0)	0.0 (0.0)	0.0 (0.0)	0.0 (0.0)	0.0 (0.0)	0.16

BAT, Basophil activation test; FEV₁, forced expiratory volume in 1 second; FVC, forced vital capacity; DBPCFC, double-blind placebo-controlled food challenge

^*Kruskal-Wallis and Fisher’s exact tests were used to calculate p-values for continuous and categorical variables by CSS, respectively.

^†PF 100 indicates values for basophils stimulated with peanut flour at the 100 ng/mL concentration, etc.

^‡Collapsed CSS 5 and 6 due to only 1 subject in the CSS 6 category.

Preselection of Basophil Activation Test

The %CD63^hi response to 10 ng/mL peanut divided by %CD63^hi anti-IgE (CD63 ratio) was chosen among all BATs to be further considered in RF based on having the highest AUC (0.67) (eTable 1).

Predictive algorithm for the CSS

Of the set of 69 features, 4 features were identified as important predictors of the CSS using RF selection and the threshold criteria previously described (Figure 1). The CD63 ratio, history of exercise-induced asthma (EIA), the ratio of FEV₁ to FVC at DBPCFC, and absolute blood basophil count were then entered into a CART model, with three appearing in the final decision rule (Figure 2). Our analysis demonstrated that the CD63 ratio, a history of EIA, and the FEV₁/FVC ratio at DBPCFC are important predictors of reaction severity. Participants with a CD63 ratio greater than 5.49 were at highest risk of experiencing a severe reaction with a median CSS of 4. For CD63 ratios between 0.47 and 5.49, a history of EIA may increase the risk of a severe reaction. For CD63 ratios ≤ 0.47, an FEV₁/FVC ratio of ≤ 0.8 may result in slightly higher risk of a severe reaction compared in participants with higher FEV₁/FVC ratios. To assess the validity of this model, a repeated 10-fold cross-validation was used and yielded an RMSE of 1.16 for the three-staged modeling process.

FIG. 1.

Random forest plot highlights variables which were deemed important: the CD63 ratio, a history of exercise-induced asthma (EIA), FEV1/FVC at DBPCFC, and the absolute blood basophil count.

Included variables are listed on the y-axis (n=69), with corresponding variable importance on the x-axis. Variables were deemed important predictors of the severity score if their importance value was larger than the absolute value of the lowest negative-scoring variable (to the right of the dashed red line). Therefore, the ratio of %CD63hi peanut protein concentration 10 to anti-IgE, a history of exercise-induced asthma (EIA), forced expiratory volume in 1 second (FEV1)/forced vital capacity (FVC) at double-blind placebo-controlled food challenge (DBPCFC), and absolute blood basophil count were considered important variables. sIgE: peanut-specific IgE; sIgG4: peanut-specific IgG4; tIgE: total IgE; SPT: skin prick test; BMI = body mass index.

FIG. 2.

Using classification and regression trees (CART), 3 features were used to classify participants and assess risk of severity of reaction (CSS): the CD63 ratio, a history of exercise-induced asthma (EIA), and FEV1/FVC at DBPCFC.

ratio of heparin %CD63^hi peanut protein concentration 10 to anti-IgE (CD63 Ratio), forced expiratory volume in 1 second (FEV1)/forced vital capacity (FVC) at double-blind placebo-controlled food challenge (DBPCFC), and a history of exercise-induced asthma.

Features stratified by the terminal nodes of the CART are displayed in Table 4. Notably, node 5, which has a median CSS of 4 is characterized by a population of participants with the highest IgE values (total, peanut, Ara h 1, 2, 3, 8, and 9) and highest CD63 ratios.

Table 4.

Baseline characteristic differences by CART terminal nodes

	CV RMSE = 1.16
Characteristic	Node 1 (n=11)	Node 2 (n=23)	Node 3 (n=25)	Node 4 (n=54)	Node 5 (n=7)
Continuous variables, Mean (SD)
Peanut SPT	4.27 (1.31)	5.78 (2.81)	4.88 (1.84)	5.08 (2.12)	5.50 (1.89)
FEV1 at DBPCFC	92.18 (9.53)	99.4 (13.87)	91.24 (8.85)	96.5 (11.59)	94.57 (14.30)
FEV1/FVC ratio at DBPCFC	0.75 (0.05)	0.88 (0.05)	0.81 (0.07)	0.85 (0.06)	0.80 (0.10)
Number of food allergies	3.00 (3.22)	4.00 (2.59)	4.36 (2.66)	3.65 (2.43)	2.57 (2.15)
BMI	18.63 (3.91)	20.12 (6.79)	19.44 (4.48)	18.32 (3.89)	21.09 (6.24)
CD63 ratio	0.20 (0.16)	0.20 (0.17)	1.73 (1.45)	2.01 (1.12)	10.72 (4.41)
Continuous Variables, Median (1st quartile, 3rd quartile)
Age at enrollment	11 (9, 16)	9 (8, 14.5)	12 (9, 16)	10.5 (8, 14)	13 (10.5, 2.5)
Peanut diagnosis age	2 (1, 3)	2 (1, 3)	1 (0.8, 2)	1.5 (1, 2)	1.5 (1, 3.5)
Peanut allergy duration	9.2 (6.8, 13.3)	8 (6, 13)	9.3 (7.5, 13.3)	8.7 (7, 12)	11.5 (9.8, 29.5)
Ara h 1 (ng/L)	1.2 (0, 3.6)	0.6 (0, 26.0)	52.5 (3.0, 135)	82.6 (10.2, 224)	134.7 (60.0, 232)
Ara h 2 (ng/L)	6.3 (1.8, 14.0)	8.9 (1.4, 64.8)	75.6 (25.1, 169)	152.9 (52.6, 409)	193.5 (104.1, 439)
Ara h 3 (ng/L)	0 (0, 1.3)	0.4 (0, 3.0)	4.1 (0.5, 40.3)	18.2 (3.0, 46.4)	25.9 (14.0, 43.4)
Ara h 8 (ng/L)	0 (0, 1.3)	0.3 (0, 12.2)	0 (0, 0.7)	0 (0, 1.5)	0.4 (0.1, 2.2)
Ara h 9 (ng/L)	0 (0, 12.1)	0 (0, 0.2)	0 (0, 0)	0 (0, 0)	0.5 (0, 2.6)
Peanut IgG4 (μg/L)	480 (255, 715)	360 (210, 1520)	940 (350, 1600)	910 (402, 2388)	2460 (1055, 4535)
Peanut IgE (ng/L)	14.4 (2.9, 65.8)	47.3 (8.5, 219)	192.8 (36.1, 312)	316.0 (147, 767)	446.5 (218, 1252)
Total IgE (ng/L)	1342.0 (708, 3370)	1193.2 (603, 2302)	1281 (683, 2403)	1339.6 (699, 3226)	3991.8 (473, 8808)
Categorical Variables, Percent
Male	81.8	82.6	68	61.1	42.9
Caucasian	90.9	73.9	64	68.5	71.4
Asian	18.2	43.5	32	37	28.6
Black	0	4.3	4	1.9	0
Non-Hispanic	100	95.7	100	96.3	100
Peanut Dx only	54.5	17.4	24	29.6	57.1
Tree nut Dx	36	70	68	59	43
History of Asthma	63.6	52.2	100	53.7	100
History of AR	72.7	87	84	66.7	71.4
History of AD	63.6	73.9	84	72.2	42.9

AD, Allergic dermatitis; AR, allergic rhinitis; BMI, body mass indicator; CART, classification and regression trees; FEV₁, forced expiratory volume in 1 second; FVC, forced vital capacity; DBPCFC, double blind placebo controlled food challenge; SPT, skin prick test.

To determine the predictive ability of the CD63 ratio using standard regression techniques without the presence of other features, we also fit a linear regression model for CSS as a function of log-transformed CD63 ratio. We found that the CD63 ratio was a significant predictor of CSS alone, with a cross-validated root-mean-square-error (RSME) of 1.08 (eTable 4).

LASSO regression

After including all 69 pre-selected features into a LASSO model, history of EIA and the CD63 ratio were selected as statistically significant predictors of CSS (eTable 5). The repeated 10-fold cross-validation RMSE for this model was 2.47.

DISCUSSION

The DBPCFC is currently the gold standard for diagnosing food allergies and is most often performed in a research setting where the patients are likely to be allergic. In these allergic patients, while a positive outcome confirming food allergy is likely on DBPCFC, severe symptoms can be life threatening.^{10, 11,12} Identification of possible prognostic indicators that are associated with severe outcomes during a peanut OFC may aid in appropriate risk stratification and management when a OFC is being considered by a trained professional. Previous studies for peanut allergy have focused on clinical features and certain tests, such as SPTs, ³⁸ sIgE,¹¹, or CRDs that can assist in distinguishing between the absence or presence of a food allergy.³⁹ Many investigators have already shown that sIgE to Ara h 2, as opposed to sIgE to whole peanut, has greater specificity for predicting reactivity during peanut OFC (ranging from 85%–95%, depending on the cohort).^40–46

Few studies have examined biomarkers to predict the severity of a reaction. Our approach was to examine demographics, clinical and laboratory data in relation to a potential new predictor tool using a CART statistical approach. BATs can be a useful tool in predicting the severity of reaction during peanut OFC in peanut-sensitized individuals,^{17, 19, 47} despite earlier studies to the contrary.⁴⁸ Other authors have published evidence on severity scores; however, these did not incorporate history of exercise-induced asthma, CD63 ratio, or FEV1/FVC ratio at DBPCFC.^{17, 19} Some groups have also tried to use clinical characteristics to predict increased adverse events during oral immunotherapy (OIT). One study found baseline allergic rhinitis and peanut SPT wheal size were significant predictors of higher overall AE rates. SPT wheal size predicted increased gastrointestinal AEs, and AR predicted increased systemic reactions during OIT.⁴⁹

Our study is a prospective study of a well-characterized population of peanut-allergic children and adults who underwent DBPCFCs. Patients with a prior history of severe reactions to peanut were included, except for those patients with a history of a peanut-allergic reaction requiring intubation or eliciting hypotension. We aimed to develop a possible method to predict severe allergic reactions, as management of these reactions is labor-intensive, requires increased observation, and can be life-threatening.

There is no universal consensus to characterize what constitutes a severe reaction during peanut OFC. While other grading systems to assign ordinal values to severity of individual symptoms have been developed, not all symptoms should be assessed based on similar grading. For instance, severe pruritus grade 3 has a different clinical significance compared to wheezing grade 3.⁵⁰ Because there is no universal agreement on the criteria for administrating epinephrine during peanut OFC, the SCI was created to characterize symptoms that most practitioners would treat with epinephrine. The CSS is novel in that it not only groups similarly severe symptoms in a more clinically meaningful way using SCI, but it incorporates the amount of food allergen ingested.^{20, 51,} Further, CSS provides multi-level phenotyping of clinical symptom severity on an ordinal scale that captured the allergic features of the reaction. For example, a subject who is able to consume the equivalent of one peanut (240mg) and develops limited hives would be assigned a CSS of 1, whereas a subject who can only consume a low dose of peanut (5mg) and develops wheezing would receive a CSS of 6. Clinically these are important distinctions that have heretofore not been successfully characterized.

To our knowledge, this is the first study to use machine learning procedures, i.e. random forest and CART, which incorporate a combination of clinical and laboratory based parameters to develop an internally validated decision rule that could potentially be used by clinicians to better understand a patient’s risk of having a severe allergic reaction during peanut OFC. Unlike CART, previous models using standard regression procedures are not well-equipped to discover, in an agnostic fashion, complex interactions among features that are not pre-specified using dichotomous branching logic and optimal cut-off values among the features.

The basophil activation ratio of CD63 was prominently featured in our predictive model, similar to work done by Santos et al.¹⁷ The ratio of CD63 positive basophils after stimulation with peanut vs. anti-IgE was selected in the CART model in three different ranges to be an important initial discriminator of the CSS. A CD63 ratio of greater than 5.49 identified a subset of patients who had the highest predicted CSS. The absolute basophil count was chosen as one of four predictors in the RF model, but then excluded in the CART model, likely as no significant cut points could be identified to risk stratify participants.

In the predictive model (Figure 2), not only was a history of EIA important, but the CART analysis also selected the FEV₁/FVC ratio on the day of peanut OFC. Asthmatics are known to have higher risk for more severe allergic reactions.^{52, 53} Thus, it is not recommended to conduct a peanut OFC in a patient with respiratory symptoms or an active respiratory infection. Current research protocols specify an FEV₁/FVC ratio ≥ 80% cutoff prior to the start of peanut OFCs; this should also be considered for peanut OFCs in the allergist’s office as well.

Interestingly, antibody tests, including sIgE, sIgG4, ratios of sIgE/sIgG4, and peanut CRDs were similar in peanut allergic participants who had mild or severe reactions, and thus were not identified as important variables relative to the ones ultimately selected, particularly, the CD63 ratio. However, our data shows that the participants with the highest median CSS of 4 (node 5 in Fig. 2, Table 4) had on average higher IgEs (total, peanut, and CRDs) compared to subjects in other nodes. Even though these biomarkers had higher expression in node 5 compared to other nodes, they were not as predictive as the CD63 ratio (Figure 1). Additionally, we fit several logistic regression models to determine if any known predictors of severity (ie, SPT, Ara h 2, and sIgE) were better than the CD63 ratio, and found that the CD63 ratio was the superior biomarker (eTable 6).

With more clinical trials studying different therapies in peanut-allergic individuals, our tool could be used to aid the researcher in phenotyping those patients at risk for a more severe reaction at DBPCFC. We propose to currently apply this predictive decision rule in clinical research only. It is important to note that if a patient has a concurrent illness ⁴⁹, these co-existing conditions can increase the likelihood of positive outcomes during peanut OFC. Our DBPCFCs were done in controlled settings in which we excluded illnesses or other events known to put participants at risk; these are the same precautions one would take in an outpatient clinic or clinical research study.

To provide a comparison between CART and a more widely used method, we fit a LASSO regression model using the 69 features chosen in the pre-selection process. We found that a history of EIA and a higher CD63 ratio were significantly associated with higher CSS. Further, the RMSE of the LASSO model was higher compared to that of the CART (2.44 vs 1.16), suggesting that the CART method was a superior predictive model. Importantly, while the cross-validated RMSE of the linear regression model that only includes CD63 ratio was lower than that obtained using CART (1.08 vs 1.16), interpreting results from the CART model is arguably more intuitive than interpretation resulting from a standard regression model. From the regression model, a 10% increase in the CD63 ratio was associated with a 0.02 unit increase in CSS (eTable 4). Conversely, using CART, we see how CSS is predicted to increase when values of CD63 ratio exceed a threshold.

In conclusion, we present early findings from a well characterized, relatively large, peanut allergic cohort that underwent standardized OFCs as part of a large, phase 2 prospective study. The decision rule developed using the CSS may be considered as a predictive aid in clinical research settings to determined risk of severe reaction during peanut OFC However, further testing of this decision rule is needed in a larger validation cohort. One of the limitation of this study is that this study did not include children (who may have no clear history of EIA), and this model cannot, at this time, be applied in younger children. The severity predictors we discovered in our CART and RF analyses highlight the importance of the CD63 ratio which may aid the researcher in deciding to conduct peanut OFCs.

ABBREVIATIONS

AUC

Area under the Curve

BAT

Basophil Activation Test

CART

Classification and Regression Tree

CRDs

Component Resolved Diagnostics

CSS

Challenge Severity Score

DBPCFC

Double-Blind, Placebo-Controlled, Food Challenge

EIA

Exercise-induced Asthma

FEV₁

Forced Expiratory Volume in 1 second

FVC

Forced Vital Capacity

LASSO

Least Absolute Shrinkage and Selection Operator

RF

Random Forest

RMSE

Root Mean Squared Error

SCI

Severity Clinical Indicator

sIgE

Peanut-specific Immunoglobulin E

sIgG₄

Peanut-specific Immunoglobulin IgG₄

SPT

Skin Prick Test

tIgE

Total IgE