A Qualitative Study to Assess the Content Validity of the 24‐Hour Migraine Quality of Life Questionnaire in Patients with Migraine

Abstract

Objective

A concept elicitation, cognitive debriefing, and usability study was undertaken to: (1) explore migraine symptoms and day‐to‐day impacts; (2) determine the comprehensiveness and comprehensibility of the previously developed 24‐Hour Migraine Quality of Life Questionnaire electronic patient‐reported outcome (24‐Hr MQoLQ ePRO) items, and the appropriateness and understanding of the recall period, response options, and instructions; and (3) assess the usability on an electronic hand‐held device.

Methods

Eleven United States English‐speaking people with episodic migraine were recruited to participate in one‐on‐one interviews, which followed methods appropriate for concept elicitation, cognitive debriefing, and usability testing. Interviews were audio‐recorded, transcribed, and analyzed following the constant comparative method.

Results

Participants had a mean age of 42 years, and 8 were female. Through spontaneous mention or probing, all concepts of the 24‐Hr MQoLQ ePRO were endorsed by a majority of the participants. Cognitive interviewing confirmed the 24‐Hr MQoLQ ePRO instructions were clear, meaningful, and important to assess as symptoms and day‐to‐day impacts experienced as a result of migraine. Overall impressions of the ePRO device were overwhelmingly favorable, and the ePRO device was preferred to paper and pencil by all participants. Participant responses regarding the level of headache pain that would be acceptable in order to continue to go about daily activities ranged from 3 to 6, on a scale of 0 to 10, with 0 being “no headache” and 10 being “the worst headache.”

Conclusions

The 24‐Hr MQoLQ ePRO is content‐valid and appropriate for inclusion in future acute treatment for migraine studies designed to measure the symptoms and health‐related quality of life of migraine.

Abbreviations

24‐Hr MQoLQ ePRO

24‐Hour Migraine Quality of Life Questionnaire electronic patient‐reported outcome

U.S.

United States

FDA

Food and Drug Administration

HRQoL

health‐related quality of life

ePRO

electronic patient‐reported outcome

PFDD

Patient‐Focused Drug Development

Background

Migraine is a common disorder, often associated with debilitating symptoms. It is estimated that worldwide over 1 billion people have migraine, the global age‐standardized prevalence being 18.9% for women and 9.8% for men. ¹ Migraine has a negative impact on patients’ quality of life (QoL), including physical and social aspects of daily living. Work productivity is particularly impaired, with migraine recognized as the leading cause of years lived with disability in individuals under 50 years of age. ²

While the primary endpoints in clinical trials for acute treatment of migraine are pain freedom and most bothersome symptom freedom at 2 hours post dose, ³ , ⁴ patient‐reported outcome (PRO) measures should be included in clinical trials. One of the PROs that could be included for measuring the health‐related quality of life (HRQoL) of patients with migraine is the 24‐Hour Migraine Quality of Life Questionnaire (24‐Hr MQoLQ).

The 24‐Hr MQoLQ was developed in the mid‐1990s, ⁵ , ⁶ prior to release of the Food and Drug Administration (FDA) PRO Guidance regarding the need for evidence to support the content validity of PROs. ⁷ While qualitative interviews were conducted with patients during the instrument development phase, ⁵ the sample included only 6 subjects, the only criterion for their selection was willingness to speak to how their life was affected by migraine, and the sociodemographic and clinical characteristics of the sample has not been reported. Therefore, the evidence necessary to demonstrate that the 24‐Hr MQoLQ is content valid, a component of demonstrating an instrument is fit‐for‐purpose, ⁸ is lacking and further interviews needed among patients with migraine to facilitate its use in future acute treatment clinical trials. In addition, the original 24‐Hr MQoLQ was developed for administration via paper‐and‐pencil. It was converted, with no changes to the content, to an electronic PRO (ePRO) form consistent with data collection practices as recommended in the FDA Guidance for Industry. ⁹ Though there is a body of evidence supporting the equivalence of electronic and paper PRO administration, ¹⁰ there was a desire to obtain instrument‐specific evidence to support the usability of the 24‐Hr MQoLQ. For these key reasons, a concept elicitation, cognitive debriefing, and usability study was undertaken to: (1) ascertain the migraine‐related symptoms and impacts; (2) determine the comprehensiveness and comprehensibility of the 24‐Hr MQoLQ items, as well as the appropriateness and understanding of the recall period, response options, and instructions; and (3) assess the usability of the 24‐Hr MQoLQ in ePRO form on a hand‐held electronic device (ie, tablet).

Methods

Instrument

The 24‐Hr MQoLQ used in this study was an electronic version of the original instrument, which is a reliable, valid, self‐administered measure specifically developed to capture the HRQoL of patients with migraine within 24 hours of taking migraine medication. Characteristics that motivated the development of the 24‐Hr MQoLQ were that it: (1) reflect areas of health and functioning important to adults with migraine, (2) reflect areas of health and functioning identified through statistical modeling, and (3) be responsive to change in HRQoL in the 24‐hour period following migraine onset. ⁶ The 24‐Hr MQoLQ contains 15 items that cover the following domains: work functioning, social functioning, energy and vitality, feelings and concerns, and migraine symptoms. ⁵ , ⁶ Each domain consists of three items answered on a 7‐point scale, where 1 indicates maximum impairment and 7 indicates no impairment. Domain scores range from 3 to 21 and are calculated by summing the responses to the three items, with lower scores indicating lower HRQoL.

The process for development of the 24‐Hr MQoLQ is described by Hartimer and colleagues. ⁵ Item‐generation of the instrument was completed with input from 6 patients with migraine and 101 items were generated. The item reduction process involved completion of the items by 76 subjects who responded to a public announcement advertising for patients with migraine. The subjects also ranked the importance of each item. Principal components factor analysis with varimax rotation was performed to organize domains and reduce the items. The final 15‐items were pretested with 2 groups of 5 people with migraine and they were debriefed on the items with investigators. The instrument was modified based on subject feedback and then pretested again with another 2 groups of 5 people with migraine.

Psychometric measurement properties of the instrument were examined among 107 subjects. ⁶ Internal consistency of the 24‐Hr MQoLQ, as assessed by Cronbach's alpha was .90 for the work functioning domain; .91 for social functioning; .86 for the energy domain; .74 for migraine symptoms; and .85 for the feelings/concerns domain. Construct validity was demonstrated by moderate correlations with migraine duration (work, r = −.31; social, r = −.29; energy, r = .29; symptoms, r = −.44; concerns, r = −.25) and global rating of symptoms (work, r = .35; social, r = .42; energy, r = .53; symptoms, r = .43; concerns, r = .35). Responsiveness was demonstrated through significant mean score differences between migraine‐free and with‐migraine scores.

Participants

All interviews were conducted in a U.S. English‐speaking sample of people with episodic migraine recruited from 2 clinics that specialize in the treatment of migraine. Participant inclusion criteria were similar to those used in recent Phase 3 clinical trials. ³ , ⁴ Key criteria for the qualitative research included participants’ being ≥18 years old, having a diagnosis of migraine as defined by the International Headache Society (IHS) 1.1 or 1.2 criteria, with history of migraine for at least one year, onset prior to age 50, and having a history of 3 to 8 migraine attacks per month and <15 headache days per month during the past 3 months, and a Migraine Disability Assessment score ≥11. Exclusion criteria included history of stroke, epilepsy, vertigo, vestibular migraine, or diabetes with complications; actively suicidal or significant risk for suicide; known hepatitis B or C or HIV infection; chronic migraine or other form of primary or secondary chronic headache disorder; pregnant or breastfeeding; drug or alcohol dependence; acute, serious, or unstable medical condition (autoimmune disease, cardiovascular, hepatic, respiratory, hematological, endocrine, psychiatric or neurological disease, or any clinically significant laboratory abnormality, that in the judgment of the investigator, indicates a medical problem that would preclude study participation); direct affiliation with study or immediate affiliation with study; and Eli Lilly employees.

All potentially eligible participants were screened for study eligibility using patient medical charts and were subsequently contacted by site staff using a standardized screening script, either over the telephone or at their regularly scheduled office visits. Eligible participants were invited to participate in a face‐to‐face interview. This study was approved by the Advarra Institutional Review Board and all participants provided written informed consent.

Data Collection

Experienced interviewers (RMS and EMC) followed a semi‐structured interview guide during the one‐on‐one interviews, which included content appropriate for concept elicitation, cognitive debriefing, and usability testing. A single, in‐person, one‐on‐one interview was conducted per participant in a private office at the clinic site, and each interview took approximately 90 minutes to complete. The objective of the open‐ended concept elicitation section of the interview was to understand participants’ migraine symptoms and the day‐to‐day impacts they experience. Participants were probed on known symptoms and impacts of migraine, ¹¹ , ¹² if they were not spontaneously reported by the participant. Participants were also asked what they thought would be a meaningful change or improvement in their day‐to‐day effects or functioning and what level of headache pain on a scale of 0‐10, with 0 being “no headache” and 10 being “the worst headache,” would be acceptable to them in order to still perform their day‐to‐day activities.

Following the open‐ended concept elicitation discussion, participants were given the ePRO device and asked to complete the 24‐Hr MQoLQ ePRO and then, through structured questioning provide feedback on the overall comprehension, relevance, and content validity of the instrument. For example, participants were asked to state in their own words the meaning of each item, what answer they selected and why, and whether they personally found the item to be relevant. After debriefing of the 24‐Hr MQoLQ ePRO and cognitive debriefing of the items, a set of questions about device usability were asked. Participants were asked about their experience and familiarity with ePRO devices, what if any difficulty they had using the device, as well as the ease of screen transitions and selecting response options, and the appearance of font. The interviews were audio‐recorded and transcribed by a third‐party transcription vendor. Following completion of the interview participants completed a brief sociodemographic form and clinical sites completed a clinical report form regarding migraine diagnosis, migraine treatment, and comorbidities. Each participant was remunerated $150 via ClinCard for their time.

Data Analysis

Qualitative data were analyzed using ATLAS.ti qualitative data analysis software version 8.2, ¹³ which was developed for the analysis of textual, graphical, audio, and video data. Within ATLAS.ti researchers create “codes,” to be used for conceptualizing and organizing the qualitative interview content. The analysis followed the constant comparative method, which is an iterative coding approach that involves reviewing consecutive transcripts and allowing new codes to emerge. ¹⁴ All transcripts were coded by one study team member and reviewed by the investigator for agreement. For each utilized code, coding outputs were generated from ATLAS.ti. Codes from the concept elicitation portion of the interview were entered into a saturation grid. Saturation is defined as the point at which no new themes, descriptions of a concept, or terms are introduced as additional interviews are conducted. ¹⁵ , ¹⁶ For the cognitive debriefing and usability portions of the interview, data were evaluated by 24‐Hr MQoLQ item or usability concept and summarized for the study sample. Descriptive statistics (mean, standard deviation, frequency, and percentage) were calculated for sociodemographic and clinical report form data.

Results

Participant Descriptors

A total of 31 patients were screened for participation in this study, of whom 19 were deemed eligible. Of the 19 eligible, 4 were not interested in participating (reasons not recorded) and 15 were scheduled for their in‐person interview visit. Four potential participants did not show up for their visit, which resulted in 11 total participants across two U.S. clinical sites. The 11 participants had a mean age of 42 (standard deviation [SD] = 14) years, and 8 (73%) were female. The majority of the sample identified as white (n = 10, 91%). The mean length of migraine diagnosis for the study sample was 20.8 years (SD = 11.4, range = 3‐38). The most common comorbidities were depressive disorder (n = 7, 64%) and anxiety (n = 5, 46%). Two study participants had no current comorbidities. Additional demographic characteristics, including living situation, employment status, highest education achieved, and household income are found in Table 1.

Table 1.

Participant Demographic and Clinical Characteristics

Characteristic	Total N = 11
Age
Mean (SD)	42 (14)
Median	44
Range (min‐max)	24‐66
Gender (n, % male)
Male	3 (27.3%)
Female	8 (72.7%)
Ethnic background (n, %)
Hispanic or Latino	1 (9.1%)
Not Hispanic or Latino	10 (90.9%)
Racial background (n, %)
Black or African American	1 (9.1%)
White	10 (90.9%)
Current living/domestic situation (n, %)
Living alone	1 (9.1%)
Living with spouse, partner	9 (81.8%)
Other‡	1 (9.1%)
Highest education level (n, %)
Secondary/high school	2 (18.2%)
Some college	4 (36.4%)
College degree	2 (18.2%)
Postgraduate degree	2 (18.2%)
Technical or vocational degree	1 (9.1%)
Employment status (n, %)
Work full‐time	4 (36.4%)
Work part time	1 (9.1%)
Student	2 (18.2%)
Retired	2 (18.2%)
Disabled	2 (18.2%)
Current annual household income (n, %)
Less than $25,000	2 (18.2%)
$25,000‐$44,000	2 (18.2%)
$45,000‐$75,000	4 (36.4%)
More than $75,000	3 (27.3%)
Years since first diagnosed with migraines (n, %)
More than 1 year and up to 5 years	2 (18.2%)
More than 5 years and up to 10 years	1 (9.1%)
More than 10 years	8 (72.7%)
Years since migraine diagnosis
Mean (SD)	20.8 (11.4)
Median	22.5
Range (min‐max)	3.0‐38.0
Missing, n (%)	1 (9.1%)
Current comorbid conditions† (n, %)
Anxiety	5 (45.5%)
Depressive disorder	7 (63.6%)
Diabetes	1 (9.1%)
Sleep apnea	3 (27.3%)
Other§	5 (45.5%)
None	2 (18.2%)

^†Not mutually exclusive.

^‡Other living situation includes: “Dorm living” (n = 1).

^§Other comorbid conditions included: “Attention‐deficit/hyperactivity disorder combined” (n = 1), “Cervicalgia, tension headache, restless leg syndrome” (n = 1), “Degenerative disc disease” (n = 1), “Fibromyalgia, attention deficit disorder, lumbago, degenerative disc disease” (n = 1), “Obesity, anemia, arthritis, insomnia, chronic pain, bilateral hip replacements” (n = 1).

Concept Elicitation

During the concept elicitation interviews, the migraine symptoms reported either spontaneously or using probing techniques included: head pain (front of head, back of head, and sides of head), dizziness or light‐headed, blurred vision, involvement of body parts other than the head (e.g., neck, shoulders), sensitivity to light, sensitivity to sound, sensitivity to smell, nausea, loss of appetite, insomnia, and aura. For these migraine symptoms, saturation was reached within the study population (Table 2).

Table 2.

Saturation Grid – Migraine Symptoms

Migraine Symptoms	Participant ID
	002‐001	002‐019	002‐018	002‐020	002‐006	003‐001	003‐006	003‐010	003‐009	003‐007	003‐003
Dizziness or light‐headed	S	S		S					S	S	S
Blurred vision	S	S			S	P
Pain (Headache)	S	S	S	S	S	S	S	S	S	S	S
Involvement of body parts other than the head	P	P			S	S	P	S	P
Sensitivity to light	S	S	P	S	S	P	P	S	S	P	S
Sensitivity to sound	P	P		P	S	P	P	S	S	P	P
Feeling nauseous		P		S	S	P	S	P	S	S	P
Loss of appetite			S				P		S
Sensitivity to smell					S	P		P	S
Insomnia								S
Experiencing Aura									S		S

P = probing; S = spontaneous.

When queried about the effects that migraine symptoms have on day‐to‐day activities, the impacts/impairments reported either spontaneously or via probing were: relationships/interactions with others, physical activity, work/employment, meal preparation/cooking, housework, sleep (ability, quality), ability to drive, concentration or focus, energy level, ability to have conversations, mental health, going outside the house, and enjoyment of life. A concept grid that summarizes the responses that participants provided to open‐ended questioning regarding the ability limitations and negative effects reflected in the 24‐Hr MQoLQ content (Items 7 to 15) is shown in Table 3.

Table 3.

Concept Grid – Patients Mentioning 24‐Hr MQoLQ Ability Limitation and Negative Effect Item Concepts

	002‐001	002‐019	002‐018	002‐020	002‐006	003‐001	006‐006	003‐010	003‐009	003‐007	003‐003
Do normal everyday work (job outside the home, schoolwork, housework)	S	S	S	S	S	S	S	S	S	S	S
Stay alert		P	P	P	S	P	S	P	S	P	P
Operate machinery or a motor vehicle (including home appliances and office equipment)	S	S	S	S			S
Enjoy life		P	S	P	P	S	S	P	S	P	P
Interactions with people who are close to you	P	P	P	P	P	P	S	P	P	S	S
Interactions with other people	P	P	P	P	P	P	S	P	P	S	S
Energy level		P	P	P	P	P	P	P	P	P	P
Ability to have a good night’s sleep	P	P	P	P		P	P	P		P	P
Mood	P	P	P	P	P	S	S	P	S	P	P

P = probing; S = spontaneous.

Results demonstrated that the content of the 24‐Hr MQoLQ covered key symptoms and impacts/impairments that were important and relevant to patients with migraine. Importantly, whether through spontaneous mention in the open‐ended questioning or through probing, all concepts of the 24‐Hr MQoLQ items were endorsed by the majority of participants. Illustrative participant quotes that describe how their day‐to‐day activities are affected by migraine attacks are displayed in Table 4.

Table 4.

Qualitative Support for the 24‐Hr MQoLQ Items and Domains

24‐Hr MQoLQ Domains	24‐Hr MQoLQ Items	Example Quotes
Migraine symptoms	Have increased sensitivity to light or sound	002‐006: I would say it is, um, over – an overload on your senses, um, light hurts, sound hurts
		002‐018: It’s beyond light sensitivity, it’s even almost having your eyes open at that point that it hurts. Yeah, almost any kind of light, but bright light is almost is in – is intolerable
	Have nausea	002‐019: The ones that last longer, those are the ones where I’m nauseated
		003‐006: And, of course, they make me feel nauseated
	Have throbbing head pain	002‐019: Just the pounding on my head, my vision gets kind of blurry when it’s really bad
		003‐010: The pain, excruciating painful throbbing in the forehead region
Feelings/concerns	Feel upset about having migraine headaches	003‐001: I think when I feel one coming on I get so anxious and stressed, because I start thinking of all the things I need to do and how am I going to do this if I don’t feel good
		003‐006: I just kind of want to cry sometimes, because it does – I mean if they’re bad enough it takes me out of life
	Feel physically uncomfortable	003‐006: I just feel for 2 or 3 days after I have a really bad one – and I don’t have strength, I mean I just feel so tired. My eyes hurt, you know, and everything is just – you feel drained, I guess, like you’ve been sick, but it just takes everything out of you
		003‐010: I’m feeling like I’m – feel like the – the hangover after drinking to where you just can’t keep nothing down. You do not want to look at food, you do not want to look at nothing besides the blackness of your eyelids because it’s like you feel like you’re on a rollercoaster, you’re going up and then you go down
	Feel concerned that your migraine medication wouldn’t relieve your migraine headache symptoms	003‐006: I can take a whole lot of things sometimes, and I don't get relief. So, I think that the whole time that I have a headache I just am waiting on the relief of it and it’s not very often that I get it
Work functioning	Do normal everyday work (job outside the home, schoolwork, housework)	002‐001: Well, for my job, I’m on the computer and phone all day, so if I have a migraine I’m not going to work
		003‐009: Sometimes there’s been occasions where I’ve had to miss school, work, whatever it was, um, because of the pain
	Stay alert	002‐006: If someone is asking me a question, uh, I have a difficult time putting the information together, and I’ll give you an example. Uh, so, uh, after work I’m going to go here, can you pick me up and then can we go to the store? To put those answers together should be a very, very simple and brief. But to think about that with the pain going on and the sensors being overloaded, it’s – it’s almost like I can’t say it, can’t do it
		003‐003: You can’t really even get yourself to think straight sometimes because you’re just so absorbed with the throbbing and all of it
		003‐006: I can’t – you could be asking me a question and I wouldn’t – I’d be like – you know, my eyes hurt, like I can’t – I don’t even know what you’re saying. I mean, but I do know what you’re saying, it’s just trying to find the thoughts
	Operate machinery or a motor vehicle (including home appliances and office equipment).	002‐018: I wouldn’t be able to go outside, um, or drive on a day like this and the sun like this and actually driving it any – at any point it’s just excruciating, so
		002‐020: It’s really hard to do, I mean I was driving home the other day and it was really hard to still like even open my eyes and drive. Um, so I tend to shut down and do nothing, uh, yeah
Social functioning	Enjoy life	002‐018: During them [migraine] see, I don’t enjoy anything when I have them
		003‐006: I mean it’s the pain, but it’s also just like I said, missing out on life, I mean nobody has time to stop and take a 5‐hour nap in the middle of the day, you know? And it could be on days that my kid had a party at school or things that I want to be able to be there for, and unless I go feeling sick or with an awful headache, then I just can’t
	Interactions with people who are close to you	002‐006: My son comes in and he’s telling me all about his day and my other son will come in and he’s fixing himself something to eat and then, uh, my wife would come home, talk about her day and the dog barks in the middle. Uh, all of those perfectly normal things happen, uh, but all those together it’s too much for me to, uh, process out, so that all contributes to. I’ll either – my family will pick up on when I have a headache and they will keep it – keep things down or I’ll just remove myself to – to get away from it
		003‐009: When you don’t feel well and you have a – you know, we’re going out to dinner or something like that with your husband or friends or whoever and then it’s, well, I don’t feel well, so I’m just going to sit there or [laughs], you know, I don’t feel up to it, so I don’t want to go. I think that does impact your relationships with people when – when you’re not able to be yourself or even go to an event with them
	Interactions with other people	002‐019: It’s hard for me to concentrate and to have like a conversation, you know, with someone else
		003‐003: You don’t want to go out. You don’t want to converse with people
Energy/vitality	Energy level	002‐019: Definitely, when I have a migraine it’s like I have no energy. Like I’ll just lay on the couch or, you know, like I can’t do anything
		003‐001: Typically I’m a pretty energetic person if I feel okay, but if I have a migraine then it’s just ugh, you just feel like a sloth
	Ability to have a good night’s sleep	002‐001: 002‐001: It can, if I have one going to bed, uh, I can have a hard time getting to sleep. And sometimes I’ll wake up because of them, like when I wake up in the morning, I’ll wake up like an hour earlier than my alarm goes off just because my head is killing me
		003‐010: Insomnia, I’m not able to sleep, I don’t get – because I’m not having to sleep, you know, it messes with your sleep pattern because you’re in pain
	Mood	002‐006: It does very much affect my mood. To take me from being in, uh, a happy, outgoing mood to being very critical, very short‐tempered, very – I, uh, feel like I need to be secluded or moved away from everybody. 003‐001: Obviously, it makes you cranky, and that’s just not my nature, so I hate when I’m like that, you know, it just kind of pisses me off
		003‐009: I mean I’m just not the same person, I’m just not – um, well, you’re just not happy, you’re just – you don’t feel well

Participants were asked about benefits they would expect or prefer to see if they received an effective migraine treatment. All 11 participants answered the question, and some provided more than one benefit. Ten participants (91%) stated they would like to experience less pain. Other expectations or benefits included no side effects (n = 1), resolution of light sensitivity (n = 2), resolution of nausea (n = 2), and vision clarity (n = 1). Participants were also asked what level of headache pain, on a scale of 0 to 10, would be an acceptable level in order to continue to go about their daily activities. One participant said a pain level of 6 would be acceptable to continue to go about their daily activities, 5 participants stated a 5 would be acceptable, 4 participants said a 4 would be acceptable, and 1 participant stated a 3 would be acceptable.

Cognitive Debriefing

Cognitive interviews confirmed that the 24‐Hr MQoLQ ePRO instructions were clear, and the items comprising the 15‐item 24‐Hr MQoLQ ePRO captured the key symptoms and their impact as experienced by the participants. For Item #1 (how much of the time did you have increased sensitivity to light and/or noise?), all 11 participants demonstrated understanding the intended meaning of the item, reported they were impaired in this area, and were able to relate their reasons behind their answer choice, demonstrating the item was meaningful. For Item #2 (how much of the time did you have nausea?), all participants demonstrated understanding the intended meaning of the item. Some participant responses indicated they do not always experience nausea as a migraine symptom, while others described frequently feeling nauseous related to a migraine attack. For Item #3 (how much of the time did you have throbbing head pain?), all participants demonstrated understanding the intended meaning of the item and reported it was meaningful and important to assess for their experience of migraine. For Item #4 (how much of the time did you feel upset about having migraine headaches?), all participants demonstrated understanding the intended meaning of the item. When debriefed on responses, more than half of the participants responded they felt upset about having migraine headaches “all of the time” (n = 5, 46%) or “most of the time” (n = 2, 18%). The importance of the item is reflected in the following participant quote:

003‐001: Actually, a little relieved when I saw that question, because remember how I was telling you earlier how I just kind of get mad at my head or at myself, and I thought I probably was the only person that felt like that, so I was like, oh, other people must think like that, too, or they wouldn’t have put it in the question.

For Item #5 (how much of the time did you feel physically uncomfortable?), all participants demonstrated understanding the intended meaning of the item and the majority described feeling uncomfortable during most if not the entirety of the migraine attack, demonstrating a strong relevance from this item. For Item #6 (how much of the time did you feel concerned that your migraine medication would not relieve your migraine headache symptoms?), all participants demonstrated understanding the intended meaning of the item.

For Item #7 (how much of the time did your migraine headache and accompanying symptoms limit your ability to do normal everyday work [job outside the home, schoolwork, housework]?”), all participants demonstrated understanding the intended meaning of the item. Participants described everyday work as either at their job or doing household chores, such as doing the laundry or washing dishes. While some participants described not being able to perform such tasks while experiencing a migraine, 2 participants described having to perform everyday household task despite the migraine pain. For Item #8 (how much of the time did your migraine headache and accompanying symptoms limit your ability to stay alert?), all participants demonstrated understanding of the item.

For Item #9 (how much of the time did your migraine headache and accompanying symptoms limit your ability to operate machinery or a motor vehicle [including home appliances and office equipment]?), all participants demonstrated a correct interpretation of the meaning. Many of the participants (n = 8, 73%) described either not wanting or being unable to operate a motor vehicle during a migraine attack. All of these same participants indicated they were still able to operate office equipment and home appliances; however, two participants mentioned a desire not to do so. One participant did make a comment concerning the inclusion of both machinery or a motor vehicle and home office equipment in the question:

002‐006: What it means to me is the ability to drive a car, operate tools, handsaw or, um, tools of any nature. And it says including home appliances and office equipment, I wouldn’t exactly consider, uh, a fax machine in the same classification as a – as a back loader … .

For Item #10 (how much of the time did your migraine headache and accompanying symptoms limit your ability to enjoy life?”), all participants demonstrated a correct interpretation of the meaning. Participants described what “enjoy life” meant to them in variable ways, including activities like “hiking”, “hunting”, “bicycling”, “reading”, “watching TV”, “cooking”, “socializing with…friends”, and spending time with family. For Item #11 (how much of the time did your migraine headache and accompanying symptoms negatively affect your interactions with people who are close to you?”), all participants demonstrated understanding of the item. Participants were asked to specify who they defined as people close to them. Most participants (n = 10, 91%) indicated they thought about family, including children, spouses/significate others, siblings, parents, and an in‐law. Four (36%) participants included friends and 2 (18%) participants included co‐workers as people who are close to them. For Item #12 (how much of the time did your migraine headache and accompanying symptoms negatively affect your interactions with other people?”), all participants demonstrated understanding of the item. Participants were also asked to specify who they defined as “other people.” Four (36%) out of the 11 participants mentioned work clients or customers as other people. Five (45%) participants mentioned general acquaintances and 8 (73%) participants described strangers as being other people when answering this question. For Item #13 (how much of the time did your migraine headache and accompanying symptoms negatively affect your energy level?”), all participants demonstrated correct interpretation of the meaning.

For Item #14 (how much of the time did your migraine headache and accompanying symptoms negatively affect your ability to have a good night’s sleep?”), all participants demonstrated understanding. The majority of participants (n = 7, 64%) indicated that their quality of sleep was negatively affected by their migraine symptoms. Two participants stated their migraine medication aids them in getting some sleep. Another participant said that sleep often helps alleviate her migraine symptoms. A third participant described suffering from insomnia and suggested a change to the question’s wording:

002‐006: I would have been more comfortable with the question had it said ability to sleep, instead of a good night’s sleep.

For Item #15 (how much of the time did your migraine headache and accompanying symptoms negatively affect your mood?”), all participants demonstrated correct interpretation of the meaning. All participants described their mood being affected to some degree throughout the 24‐hour period.

Participants were debriefed on the response options for each of the 24‐Hr MQoLQ ePRO items. All 11 participants reported that the response options were sufficient and easy to understand. Two participants preferred having fewer response options for all questions. One participant suggested reducing the number of response options from 7 to 5, by omitting “A good bit of the time” and “Hardly any of the time” for questions 1 to 10; and taking out the response options of “A very great deal” and “Very little” for questions 11 to 15. All participants were also debriefed on the instrument’s 24‐hour recall period. Specifically, they were asked how they incorporated their past 24‐hour period since taking a medication for their migraine into selecting their item response option when completing the 24‐Hr MQoLQ ePRO. Participants reported that they were thinking of specific events or instances over a 24‐hour period when they were responding to each question, such as going to bed, waking up, or sitting at work. Participants’ responses in describing the timeframe for selecting a response option varied by participant and by item. For some items, the majority of the participants reported an entire 24‐hour period (eg, stay alert and enjoy life), while other questions prompted a more specific point in time during the 24‐hour recall period (the majority of light sensitivity and pain sufferers chose earlier within the 24 hours after taking medication). Overall, all 11 participants demonstrated an understanding of the 24‐hour period per ePRO item.

Usability Testing

Seven (64%) of the 11 participants had previously completed a survey on a computer, tablet, or handheld device either in a medical setting, for work or school, or for marketing purposes. Participants’ impressions of the ePRO device were overwhelmingly favorable. Most participants (n = 10, 91%) commented that it was easy to use. No difficulties were reported with regard to screen transitions. Participants were asked to describe any difficulty in physically selecting their response option for any of the items on the device. Two participants (18%) mentioned two difficulties and offered suggestions for a check mark to be placed in the answer boxes of the ePRO. Nine of the 11 participants found the font size appropriate and clearly visible. The remaining 2 participants mentioned the font size being too small to read comfortably (n = 1) and an issue with the device screen being too bright (n = 1). All 11 participants stated a preference of the ePRO version of the 24‐Hr MQoLQ over the paper version, stating it was faster, easier, less likely to make an error, and conforms with today’s technology.

Discussion

The results of this concept elicitation, cognitive debriefing, and usability study provide supportive evidence for the content validity of the 24‐Hr MQoLQ ePRO in the population of patients with migraine. To our knowledge, this is the only qualitative study to report on the content validity of the 24‐Hr MQoLQ and the only qualitative work done with the instrument other than initial interviews conducted by the developers to support the generation of items. In the concept elicitation phase of the interviews, saturation of migraine symptoms was reached in the study population. The migraine symptoms reported included head pain (front of head, back of head, and sides of head), dizziness or feeling light‐headed, blurred vision, involvement of body parts other than the head (eg, neck, shoulders), sensitivity to light, sensitivity to sound, sensitivity to smell, nausea, loss of appetite, insomnia, and aura. The effects that migraine symptoms have on day‐to‐day activities and the impacts/impairments on life that were reported by participants were consistent with those included within the 24‐Hr MQoLQ items. During the cognitive debriefing of the 24‐Hr MQoLQ, all participants demonstrated understanding of all of the items. Overall impressions of the ePRO device were positive, and the usability was good, with no revisions needed for screen transitions, response option selection, or screen clarity.

The usability of the ePRO was reported to be straightforward and the preferred mode. Most participants (n = 10, 91%) commented that it was easy to use. Some of the dislikes for the paper‐based 24‐Hr MQoLQ was that it looked “smooshed” or difficult to read and that viewing the horizontal scales at once was prone to making an error when checking a response box. These findings are consistent with prior work that suggests that an ePRO platform results in more accurate and complete data, improved compliance, and high respondent acceptance. ¹⁷ In addition, the use of ePRO data collection is consistent with the FDA Guidance for Industry. ⁹

In addition to the comprehensive interviews among a sample of patients with episodic migraine, a strength of this study is the inclusion of questions about participants’ perceptions of treatment benefit and meaningful change. Establishing individual‐level treatment benefit and meaningful change (ie, responder definition) as opposed to differences between groups, is the recent focus of the Patient‐Focused Drug Development (PFDD) draft guidance series. ⁸ , ¹⁸ , ¹⁹ Semi‐structured interviews conducted independently from a clinical trial, such as those completed for this study, are among other qualitative approaches, such as exit interviews conducted in the context of a clinical trial, focus groups, vignettes, and the Delphi panel method, that can be used to obtain data regarding meaningful change thresholds. ²⁰ In the present study, participants were asked about benefits they would expect or prefer to see if they received an effective migraine treatment and what level of pain, on a scale of 0 to 10, with 0 being no pain and 10 being the worst imaginable pain, would be an acceptable level in order to continue to go about their daily activities. The experience of less pain was the clear benefit expected by participants. The level of pain that would be acceptable ranged from 3 to 6 in this study population.

One potential limitation of this study is response bias. A form of response bias may have occurred during concept elicitation because of the use of probing on migraine‐related symptoms and impacts. It is possible that some symptoms and impacts were endorsed by participants because they were introduced through conversation. Another limitation of this study is the make‐up of the study sample. There was a lack of racial and ethnic diversity among the participants, with 90.9% being white. This limitation may limit the generalizability of the findings to racially and ethnically diverse populations. That said, the racial make‐up of this study is consistent with that of the recent SAMURAI and SPARTAN Phase 3 trials, which was roughly 80% white, ²¹ and the American Migraine Prevalence and Prevention (AMPP) study which was 87.3% white for patients with episodic migraine. ²² The full‐time employment of the study sample was only 36.4% and the disability rate 18.2%. In the AMPP study, the full‐time employment rate and occupational disability rate was 52.3% and 11.1%, respectively, for individuals with episodic migraine. ²² The proportion of study participants whose highest level of education was beyond high school was 82%, whereas it was 70.4% in the AMPP study. ²² That said, the education level of our study was similar to that of the patients involved in the development of the 24‐Hr MQoLQ. Of the 76 participants involved in the pretesting of the instrument, 81.5% had an education beyond the high school level and 52.6% had a college degree, some graduate school, or a graduate school degree. ⁵ Another limitation is that the majority of participants had been diagnosed with migraine for more than 10 years, the mean being just under 21 years. It is possible that people who have not been diagnosed with migraine for that length of time experience different impacts/impairments on HRQoL. Further, given that both the 24‐Hr MQoLQ was developed and the current study conducted in the US, future research among patient populations in other countries would be beneficial.

Conclusions

This study provided evidence demonstrating the content validity of the 24‐Hr MQoLQ ePRO as a measure of acute impact on HRQoL in patients with migraine. The items in the 24‐Hr MQoLQ are well understood and represent the range of symptoms and impacts/impairments experienced by patients with migraine. The 24‐Hr MQoLQ ePRO is a content valid, appropriate measure of impact on HRQoL for inclusion in clinical trials on the acute treatment of migraine.

Statement of Authorship

Category 1

(a) Conception and Design

Rebecca M. Speck, Ethan M. Collins, Louise Lombard, David W. Ayer

(b) Acquisition of Data

Rebecca M. Speck, Ethan M. Collins

(c) Analysis and Interpretation of Data

Rebecca M. Speck, Ethan M. Collins, Louise Lombard, David W. Ayer

Category 2

(a) Drafting the Manuscript

Rebecca M. Speck

(b) Revising It for Intellectual Content

Rebecca M. Speck, Ethan M. Collins, Louise Lombard, David W. Ayer

Category 3

(a) Final Approval of the Completed Manuscript

Rebecca M. Speck, Ethan M. Collins, Louise Lombard, David W. Ayer