Forecasting Individual Headache Attacks Using Perceived Stress: Development of a multivariable prediction model for persons with episodic migraine

Abstract

Objective

To develop and validate a prediction model that forecasts future migraine attacks for an individual headache sufferer.

Background

Many headache patients and physicians believe that precipitants of headache can be identified and avoided or managed to reduce the frequency of headache attacks. Of the numerous candidate triggers, perceived stress has received considerable attention for its association with the onset of headache in episodic and chronic headache sufferers. However, no evidence is available to support forecasting headache attacks within individuals using any of the candidate headache triggers.

Methods

This longitudinal cohort with forecasting model development study enrolled 100 participants with episodic migraine with or without aura, and N = 95 contributed 4626 days of electronic diary data and were included in the analysis. Individual headache forecasts were derived from current headache state and current levels of stress using several aspects of the Daily Stress Inventory, a measure of daily hassles that is completed at the end of each day. The primary outcome measure was the presence/absence of any headache attack (head pain > 0 on a numerical rating scale of 0 to 10) over the next 24 hour period.

Results

After removing missing data (n = 431 days), participants in the study experienced a headache attack on 1613/4195 (38.5%) days. A generalized linear mixed-effects forecast model using either the frequency of stressful events or the perceived intensity of these events fit the data well. This simple forecasting model possessed promising predictive utility with an AUC of 0.73 (95%CI: 0.71 to 0.75) in the training sample and an AUC of 0.65 (95%CI: 0.6 to 0.67) in a leave-one-out validation sample. This forecasting model had a Brier Score of 0.202 and possessed good calibration between forecasted probabilities and observed frequencies but had only low levels of resolution (i.e., sharpness).

Conclusions

This study demonstrates that future headache attacks can be forecasted for a diverse group of individuals over time. Future work will enhance prediction through improvements in the assessment of stress as well as the development of other candidate domains to use in the models.

Introduction

Migraine headache is a common disorder occurring in 18% of women and 6% of men. [1, 2] Many persons with migraine are disabled during their acute headaches with up to 90% of them reporting functional impairment and 53% exhibiting severe impairment requiring bed rest. [1] On the World Health Organization’s ranking of causes of disability, headache disorders are among the ten most disabling conditions for both genders combined and among the five most disabling for women. [2] Despite the common nature of headache disorders, we still do not understand what “triggers” an individual attack. [3]

Current treatment strategies for migraine include using abortive therapies to treat the acute attacks after they occur or preventative therapies that attempt to reduce the frequency of attacks over time. [4] Considerable evidence supports enhanced efficacy for early treatment of acute attacks, [5-8] but because headaches are currently unpredictable, patients are often unprepared to take medications in a timely manner or choose to wait to take medications until their pain escalates. One avenue to better treatment of recurrent headache disorders is to shift our treatment focus from reacting to headache attacks to proactive strategies based on forecasting future attacks. [9, 10]

There is a widespread belief among patients and physicians that there are identifiable precipitants of headache that if avoided or managed could be used to reduce headache frequency for an individual. [3, 11] A substantial pool of candidate triggers have been posited that include certain foods, food additives, alcohol, exercise, ovarian hormones (or menstrual cycles), changes in sleep, weather conditions and others. Of these, perceived stress has received particular attention and has been shown to be associated with the onset of headache in episodic and chronic headache sufferers. [12-14] However, there is currently no available evidence that future headache attacks can be forecasted for any individual using any of the candidate headache triggers.

The objective of this study was to develop and internally validate a headache-forecasting model for persons with episodic migraine that is based on perceived stress. We hypothesized that the frequency/intensity of stressful daily events could be used to forecast headache attacks within an individual over time.

Methods

Study design and setting

We conducted a prospective longitudinal study among individuals suffering from migraine headache living in the Winston-Salem, NC region from September 2009 to May 2014. The Headache Prediction Study (HAPRED) was designed to examine precipitating factors of migraine headache. The study was designed to pool 3 distinct protocols that utilized identical ecological (i.e., in the participant’s own environment) assessment procedures with the exception of the timing of certain measurements (i.e., day-time or night time). This analysis is the primary analysis from this longitudinal study that focuses on the role of perceived stress in predicting individual headache attacks. The HAPRED study was approved by the Wake Forest School of Medicine’s Institutional Review Board (IRB00008743). All participants provided written informed consent on entry into the study. The model development study is conducted in accordance with the TRIPOD guidelines. [15]

Study Population

Participants were recruited from the general community, neurology practices, and primary care clinics using direct television and print (e.g., fliers) advertising. Participants were eligible for inclusion if they were of either gender, suffered from episodic migraine with or without aura (International Classification of Headache Disorders 3^rd edition (beta version)), experienced greater than 2 headache attacks per month (pain > 0 on a numerical rating scale), and experienced between 4 and 14 headache days/month. Exclusionary criteria included the presence of a secondary headache disorder (e.g., brain tumor), a recent change in nature of headache symptoms over the last 6 weeks, not being able to read or speak English at a 6^th grade level, having a psychiatric hospitalization within the previous 12 months, meeting criteria for, or currently being treated for a DSM-IV diagnosis of substance abuse (not including Alcoholics Anonymous, etc.), pregnancy during the observation period, lack of stable residence or contact number, or meeting ICHD-3 (beta) criteria for medication overuse headaches. A Neurologist (C.T.) using a structured diagnostic interview confirmed headache diagnosis and other medical exclusions. Prior to the release of ICHD-3 (beta), diagnosis was based on ICHD-2 criteria. After ICHD-3 (beta) criteria became available, the new criteria were applied to all previous participants’ data.

Daily Assessments of Stress & Headache

To collect information about stress and headache activity, participants completed two diary entries each day (i.e., once in the AM and once in the PM). Due to the length of the study, several different electronic diary systems were used. Participants enrolled from September 2009 to mid-March 2011 used a Palm Pilot with Pendragon forms software, participants enrolled from mid-March 2011 to mid-April 2013 used an iPod touch with Pendragon forms software, and participants enrolled from mid-April 2013 to May 2014 completed web-based diaries using REDCap software [16] (i.e., the participants used their personal devices to access our web forms).

The diaries assessed the presence or absence of headache activity, headache characteristics (i.e., associated symptoms), and/or abortive medications used since making the last diary entry. Headache intensity ratings were assessed using a 0 – 10 Numerical Rating Scale (NRS) corresponding to current level of pain and the worst pain experienced since the last entry. During the PM diary entry, or the entry before bed time for third shift workers, stress was assessed using the Daily Stress Inventory. [17] The DSI is a well-validated 58 item scale that allows a subject to identify stressful events (i.e. daily hassles) that they have experienced in the last 24 hours. For each item that is endorsed, a stressfulness impact rating is made ranging from 1 (“occurred but was not stressful”) to 7 (“caused me to panic”). Several indices can be derived from the measure including the number of stressful items endorsed (FREQ), the sum of the impact ratings (SUM), and the average impact rating of the events (AIR). Each of these predictors is approximately normally distributed across participant-days given the large number of items that calculate them. Because of the moderate-high correlation between these indices, we planned to use each as separate predictors for the models.

Primary Outcome

The primary analysis is the prediction (forecast) of a future headache attack based on current levels of stress and headache. For this analysis, the primary outcome is the presence/absence of any headache attack (head pain > 0 on a NRS) occurring over the 24 hour period recorded following a PM diary entry.

General Modeling Strategy

In previous work, we have explored the time-series properties of headache attacks [18, 19] and have proposed a stress-arousal model for headache intensity in chronic headache sufferers. [13] Both of these efforts informed our current forecasting model, which was designed to harness the association between current stress levels (Stress_t) and future headache attacks (Headache_{t + 1}). This association is based entirely on the expectation, previously demonstrated by our team [13] and others [14], that a high level of current stress is associated with increased probability of an attack after the stress has subsided (i.e., “let down”). We also wished to incorporate the first-order autocorrelation often seen in headache attacks that is observed when a suffer experiences an attack that lasts for several days (i.e., a headache today is likely to also be present tomorrow). To capture this unique association, we incorporated a predictor that codes the presence or absence of a headache at the time of the diary entry (Headache_t). Thus we proposed a very simple model that presupposes that the probability of an individual’s headache tomorrow is conditional only on today’s stress and headache levels. This model is designed to serve as a starting place to pursue the new endeavor of forecasting individual headache attacks, yet due to its simplicity (i.e., it does not require information beyond the previous diary entry), allows for predictions that are not sensitive to missing diary entries over time as would be a more complex time-series model (e.g., ARIMA). Finally, by estimating the model for a new participant using a mixed model approach (i.e., random effects), we are able to ‘draw strength’ from the information collected by previous participants (see: [20]). This modeling approach allows each participant to have their own intercept (i.e., a random intercept) that reflects the fact that individuals have varying base rates of headache activity. However, the effects of each of the predictors in the model are specified as fixed effects that reflect a single population-level effect of stress on headache. Stated simply, the model allows for individual differences in headache rates but assumes that stress has a uniform effect on predicting future headache attacks.

Sample Size Considerations

The study was designed to examine the impact of stress on the future probability or intensity of individual headache attacks. The power considerations were based on predicting the presence/absence of a headache. An a priori statistical power calculation was based on the observed associations in our pilot data and assuming an episodic (<= 15 days/month) headache frequency. [13] Enrolling N = 104 patients would provide power = 0.81 to detect absolute increases in the probability of experiencing a headache as small as 4% per unit change in the AIR assuming a base rate of 25% of days with a migraine attack. Observed effects smaller than this magnitude are unlikely to be clinically meaningful.

Statistical Analysis

The plan of analysis first estimated several a priori selected candidate forecasting models and then evaluated the utility of each selected model in forecasting headache over time (i.e., as would be done in practice). To estimate our headache-forecasting model, we utilized a hierarchical binomial regression model that examines the probability of a future headache attack using today’s stress (either: AIR, FREQ, or SUM) and presence/absence of headache. A logit link function was used with a separate model created for each predictor. This model assumes that the relationship between the stress/headache risk factors and the future probability of headache is linear on the logit scale. Several exploratory models were conducted that examined the combination of the stress predictors and a headache x stress interaction term. The utility of the final model was evaluated using the statistical significance of each predictor and the Bayesian Information Criteria (BIC) model selection criterion that results in a fit score (lower is better) that penalizes a model for having more predictors. If two models are an equally good fit with the data, the model that uses fewer predictors will have a lower (better) BIC score. Additionally, each model was evaluated using estimates of precision and calibration of the forecasts. Specifically, the Brier score and its components uncertainty, reliability, and resolution were estimated. [21-23] Finally, an internal validation of the model was conducted by using leave-one-out validation (LOOV). [24] This approach estimates the model using all but one observation, and then estimates the predictive accuracy for this observation by applying the estimated model to this left-out observation and repeating the process for all observations in the sample. To compare differences between headache and non-headache days, Mann-Whitney and chi-squared testing was applied. All analyses were conducted using R3.2.2 statistical software (Vienna, Austria). Statistical significance is interpreted at p < 0.05.

Results

Between September 2009 to May 2014, N = 100 participants were enrolled in the study with n = 5 dropping out of the study and lost to observation (e.g., no further contact). Table 1 displays the demographic and headache characteristics of the participants enrolled in the study with evaluable data. For these participants N = 95 contributed 4626 days of diary data (mean: 49 days, range: 11 - 96 days). Embedded in this series of data were n = 431 (9.3%) diary entries that were either missing (i.e., the participant did not complete a diary entry, or the diary was corrupted) or not available for analysis (i.e., the final day of the diary could not be used to forecast a future day). Participants in the study experienced a headache attack on 1613/4195 (38.5%) days. The characteristics of the participants, along with the characteristics daily diaries are displayed in Table 1. The average participant reported low to moderate levels of affective distress, and moderate levels of headache related disability. As can be seen in the table, days that preceded a headache were associated with greater stress than days without a headache.

Table 1.

Participant Demographics of those individuals included in the analysis

Participant-Level Characteristics (N = 95)
Gender
Female	86 (90.5%)
Male	9 (9.5%)
Race/Ethnicity
Caucasian	83 (87.4%)
African-American	9 (9.5%)
Other	3 (3.1%)
Age	40.3 (12.9)
Diagnosis
Migraine	79 (83.2%)
Migraine with Aura	10 (10.5%)
Migraine with TTH	6 (6.3%)
MIDAS	10 [3.25, 18]
CES-D	9 [4, 17.5]
STAI	36 [32, 43.5]
Day-Level Characteristics (N = 4195)
	No Headache Tomorrow	Headache Tomorrow	p-value
† N	2241	1437
Stress
FREQ	5 [2, 9]	6 [3, 11)	< 0.001
SUM	11 [4, 21]	14 [7, 27]	< 0.001
AIR	2 [1.29, 2.94]	2.4 [1.7, 3.0]	< 0.001
Headache	765 (34.1%)	698 (48.6%)	< 0.001

^†Missing 431 days due to missing/lost diaries

TTH: Tension Type Headache, MIDAS: Migraine Disability Assessment,

CES-D: Center for Epidemiologic Studies Depression Scale, STAI: State-Trait Anxiety Inventory

AIR: Average intensity rating, FREQ: Frequency of stressful events, SUM: Sum of stressful events

Forecasting Models

To evaluate the model fit, a series of models of increasing complexity were estimated. The estimated models are reported in Table 2. Model 1 just included an intercept for each participant (i.e., predicted future headache attacks using just each participant’s base rate) fit the data reasonably well with modest predictive utility, AUC: 0.69 (95%CI: 0.68 to 0.71). This discriminative ability is reflective of the fact that individual’s base rates are good predictors of future attacks. Model 2 added the current headache status and improved the model fit to AUC: 0.72 (95%CI: 0.71 to 0.75), an improvement in model fit that was statistically significant, p < 0.0001 and also exhibited a superior BIC score. The final set of models (Models 3 through 5) examined the addition of one of the current stress predictors in predicting future attacks. Three versions of this model were estimated that included various aspects of the daily stress (e.g., either the average stress rating across events (Model 3: AIR), the frequency of events (Model 4: FREQ), and the sum of stress event ratings (Model 5: SUM). Each of these models exhibited statistically significant improved model fit (p < 0.001) compared to the previous models, with AUC’s: ≥ 0.73, and each term in the model exhibiting statistical significance and each model exhibiting an improved BIC over headache status alone. Interestingly, each of the stress models exhibited very similar predictive performances (see Figure 1) with the model based on AIR having only a slightly superior performance compared with the others as judged by the BIC score, AUC: 0.73 (95%CI: 0.71 to 0.75). Adding multiple stress predictors to the same model did not improve model fit beyond any predictor alone, nor did adding interaction terms between the predictors (i.e., stress was not more/less influential on days when someone has a headache). The AIR model was selected as the best model because of model fit as well as the fact that it incorporates both elements of stress frequency and perceived stress ratings. This model was further evaluated using internal validation.

Table 2.

The forecasting models

Model	Predictors	estimate (95%CI)	p-value	logLik	BIC
1. Unconditional				−2613	5243
2. Current Headache	headache	1.50 (1.20, 1.89)	0.0005	−2387	4815
3. AIR	headache	1.45 (1.15, 1.82)	0.0018	−2317	4707
	AIR	1.23 (1.12, 1.34)	< 0.0001
4. FREQ	headache	1.43 (1.13, 1.80)	0.0025	−2320	4713
	FREQ	1.03 (1.02, 1.05)	0.0001
5. SUM	headache	1.41 (1.12, 1.78)	0.0036	−2317	4708
	SUM	1.01 (1.00, 1.02)	< 0.0001

Subscales of the daily stress inventory: AIR (average intensity rating), FREQ (frequency of stressful events), SUM (sum of stressful ratings)

Figure 1.

The utility of the forecasting model to discriminate between future headache and non-headache attacks is displayed using ROC curves. The three stress models based on three different subscales of the Daily Stress Inventory are depicted using dark lines. The internal validation of each of these models is displayed using dotted lines. Each of the models forecasted headaches with similar utility, and exhibited similar internal validation performance.

Internal validation of stress models

For a preliminary evaluation of how the AIR stress model might perform in out-of-sample predictions (i.e., predicting observations that it was not specifically trained to forecast), a leave-one-out validation was conducted. This validation resulted in reduced performance with an AUC of 0.65 (95%CI: 0.63 to 0.67), p < 0.0001. The Brier score (validation Brier score) was 0.200 (0.219). This is equivalent to the squared error of the forecast, and can be decomposed into three elements. The uncertainty = 0.234 reflects the fact that headaches occurred about ¼ days in the observation period. The reliability (validation reliability) = 0.0016 (0.0018) reflects the close similarity between the forecasts and actual probability of the events being forecasted (0 represents a perfect forecast). The resolution (validation resolution) = 0.036 (0.017) reflects how much forecasts differ from each person’s average chance of experiencing a headache (0 represents continually forecasting the average probability for each individual). Figure 2 displays the calibration plot of the forecasted versus actual headache activity. Taken together, the various metrics can be interpreted as signifying that the forecasting model is well calibrated (i.e., when it forecasts an 80% chance of headache, there is a very close to an 80% chance of headache), but that the forecasts are not very sharp (i.e., the forecasted probabilities are close to the individual’s long-run average).

Figure 2.

The calibration and resolution of the selected AIR model is displayed using a calibration plot (large plot) and the histogram (small plot). The similarity of the forecasted (x-axis) versus actual probabilities (y-axis) indicates a well-calibrated model, while the high frequency of forecasts near the base rate of the sample indicates that the model does not have a high degree of resolution.

Discussion

The HAPRED study aimed to develop and initially validate a forecasting model of individual headache attacks within an individual headache sufferer. The primary hypothesis was that perceived stress in the form of ‘daily hassles’ could be used to forecast the let-down headache risk (i.e., increased headache activity that occurs after the experience of stress). Evidence to support the hypothesis was substantial given that the forecasting model exhibited clinically significant utility across a large sample of individual headache suffers over an extended period of time. This appears to be the first evidence that individual headache attacks can be forecasted within an individual sufferer, and this finding creates substantial opportunities for additional treatment strategies if the forecasting model can be refined. The use of a forecasting model could be used to enhanced pharmacological treatment opportunities, reduce anxiety about the unpredictability of attacks, increase locus of control beliefs, and lead to increased self-efficacy assessments about the self-management of migraine attacks.

In 2003 Giffin and colleagues [9] highlighted the idea of preemptory headache treatment. This innovative strategy was suggested after 97 headache sufferers were found to experience subtle cognitive and physical symptoms immediately prior to the initiation of an attack (i.e., premonitory symptoms). The authors argued that if the identification of these symptoms could be reliable, treatments could be delivered before the onset of actual pain onset. Indeed, this treatment strategy is highly supported by the notion that migraine attacks are more efficaciously treated early in the pain experience. [7]The current study supports the feasibility of preemptory treatment and even enhances the potential window of opportunity for treatment given that unlike the Giffin (2003) study, participants in the HAPRED study were forecasted to be at risk for a future headache before any premonitory symptoms actually began (assuming that perceived stress is not a premonitory symptom of a coming migraine attack).

The stress model in this analysis should be viewed as representing a first step in the new venture of forecasting headache attacks and not a final model for widespread clinical use. The forecasting model exhibited statistically significant utility in that the model performed better than chance. The model forecasted attacks greater than a simpler scheme that only used an individual sufferer’s long-run risk (i.e., their base rate). Further, including a daily-hassles predictor increased the utility of the model over just using the presence-absence of a current headache, and this supports the notion of the importance of the let-down phenomenon in predicting risk. [13, 14] Of note, however, is that the model utilized the predictive power of higher than typical levels of stress in predicting future attacks but did not actually specify that stress was lower in the forecasted period (i.e., the let-down was not assessed for predictive accuracy). The model predicted future headaches on days when a sufferer had an existing headache and on days when they did not, but in practice it would be intuitive to think that predicting novel attacks when a current attack is not present would be more useful. Finally, the model was very well calibrated as evidenced by the fact that forecasted probabilities were remarkably similar to observed probabilities across the full range of forecasts. However, the forecasting model exhibited several characteristics in need of refinement.

The stress forecasting model primarily forecasted probabilities near the individual sufferer’s long-run risk of an attack. That is to state, if a sufferer experienced a migraine attack on ¼ (25%) of the days during the observation period, the forecasted probabilities were likely to be near 25% for most days for that sufferer (although a broad range of forecasts, 10 – 90% were observed). While these forecasts are certainly informative, a stronger forecasting system would forecast probability that are very ‘sharp’, or very near 0% or 100%. This is akin to the idea that forecasting rain at 50% is not nearly as useful as forecasting a 0% or 100% chance of rain; the latter forecasts allow reasonable action based on assurance of the forecasted state. Improving the resolution of the stress forecasting system will almost certainly require additional predictors or conceptualizations of risk. Additionally, the use of more extended risk periods (e.g., moving from one to several days before the forecasted attack) could be very useful, especially given that previous studies have found that risk develops over the course of several days. [13]

Generalizability and Limitations

While this innovative study has several strengths, including the well-characterized migraine population and the use of electronic daily diaries, its limitations must be considered in the interpretation of the findings. The participants in this study were a community-based convenience sample, which included both those under a physician’s care and those who were not. The degree to which these individual represent the general migraine population is not known. Our sample included more women than men, which is typical of the population of migraine sufferers. However, it is not clear the extent to which the findings generalize those not directly studied.

Early on in the study, occasional failures of the dairy collection systems caused some missed entries from participants. In all of these occurrences, the participant was given a new diary unit at the earliest possible time after the failure. As the study progressed, we updated our electronic diary hardware and software to improve data capture and eventual moved to a completely web-based system, improving the integrity of data capture. However, this did create an increased rate of missing data that we considered to be missing at random (i.e., not based on patient factors).

As with any study that requires regular completion of questionnaires, there is a chance that participants became very familiar with the questions and, at times, could have been inclined to answer in a way that would allow them to complete the diary entry as quickly as possible. The DSI is also quite long, and completing the measure daily over long periods could lead to fatigue within a person. This could have led to inaccurate responses, but there is no method to measure the impact of this potential issue. Future studies will evaluate the use of abbreviated assessment tools to reduce participant burden and to examine the nature of the stress-migraine association across different subgroups of patients.

Conclusions

A simple forecasting model based on measuring the stress from daily hassles demonstrated very promising utility in predicting the future occurrence of a migraine attack in a group of migraine sufferers over an extended period of time. The performance of the model was well calibrated, but possessed only modest degrees of resolution (i.e., the forecasts centered around the sufferer’s base rate). When refined, this model has the potential to allow preemptive treatment of migraine attacks before the sufferer experiences pain or disability.