Establishing the content validity of the Epworth Sleepiness Scale for Children and Adolescents in Prader-Willi syndrome

Abstract

Study Objectives:

Excessive daytime sleepiness is common in Prader-Willi syndrome (PWS), with prevalence ranging from 52% to 100%. The goal of this study was to establish the content validity (ie, evidence that an instrument measures an intended concept of interest) of the parent/caregiver version of the Epworth Sleepiness Scale for Children and Adolescents (ESS-CHAD), a measure of daytime sleepiness, in PWS.

Methods:

Qualitative, dyadic semistructured video interviews were conducted with 18 caregivers and their children with PWS from April to June 2020. Concept elicitation and cognitive interview techniques were implemented. Thematic analyses allowed for examination of themes and data patterns.

Results:

All caregivers (mean age 49 years) were mothers of individuals with PWS who experienced troublesome daytime sleepiness (mean age 14 years). The most prevalent observable signs/symptoms of daytime sleepiness were sleepy/sleepiness (n = 17; 94.4%), tired/tiredness (n = 16; 88.9%), exhaustion/exhausted (n = 5; 27.8%), anxious/stressed (n = 5; 27.8%), irritable/frustrated (n = 5; 27.8%), having tantrums/outbursts (n = 5; 27.8%), and lethargy (n = 4; 22.2%). Daytime sleepiness impacted various aspects of health including mental, emotional, physical, and social well-being. When caregivers were asked about the activities associated with daytime sleepiness, all salient concepts elicited mapped to the ESS-CHAD; saturation was met after the first 4 interviews. Only 2 concepts, after physical exertion and while inactive/bored, did not map. Caregiver statements indicated that these concepts, although related to daytime activities, were atypical of daily routines. The ESS-CHAD was well understood and relevant to caregivers.

Conclusions:

This study supports the content validity of the ESS-CHAD and its appropriateness for evaluating treatment efficacy of daytime sleepiness in PWS.

Citation:

Patel VP, Patroneva A, Glaze DG, Davis K, Merikle E, Revana A. Establishing the content validity of the Epworth Sleepiness Scale for Children and Adolescents in Prader-Willi syndrome. J Clin Sleep Med. 2022;18(2):485–496.

BRIEF SUMMARY

Current Knowledge/Study Rationale: No qualitative evidence is available to support the content validity of the parent/caregiver version of the Epworth Sleepiness Scale for Children and Adolescents for the measurement of daytime sleepiness in Prader-Willi syndrome. The goal of this qualitative study was to evaluate if this instrument captures the most important and relevant concepts associated with daytime sleepiness in Prader-Willi syndrome from the caregiver’s perspective.

Study Impact: This novel study demonstrated the importance of measuring daytime sleepiness in PWS from the caregiver perspective and established the content validity of the Epworth Sleepiness Scale for Children and Adolescents in this patient population. The Epworth Sleepiness Scale for Children and Adolescents merits consideration for the assessment of therapeutic benefit in clinical trials and in patient care where the impact of therapy on daytime sleepiness in individuals with Prader-Willi syndrome is of importance.

INTRODUCTION

Prader-Willi syndrome (PWS) is a rare neurogenetic condition caused by the inactivity, deletion, or translocation of genes on chromosome 15 and is estimated to occur in approximately 1 in 10,000 to 1 in 25,000 live births.^1,2 Decreased muscle tone, feeding problems, an inability to thrive, atypical body composition, and developmental issues are common characteristics of this condition in infancy and early childhood.² Measurable changes in the distribution of body fat occur between 2 and 3 years of age followed by an increase in appetite and food consumption, which often results in severe obesity.³ Impaired cognitive abilities, learning problems, and behavioral issues emerge early on and worsen with age. Although advances in the treatment of PWS have lengthened life spans and improved the quality of life of individuals afflicted with this condition, the estimated annual mortality rate of 3% continues to exceed that of intellectually disabled individuals without PWS.^2,4,5 More importantly, no cure for this condition exists; as a result, many individuals with PWS live well into adulthood, forced to manage several persistent and chronic health issues including sleep abnormalities.

Individuals with PWS commonly experience sleep irregularities including excessive daytime sleepiness (EDS), a hallmark of the condition.⁶ Reporting by parents and caregivers suggests that the estimated prevalence of EDS in PWS is between 52% and 100%.^2,7 EDS is highly disruptive to the daily routines of individuals with PWS and is typically seen as problematic by parents and caregivers of children with this condition.⁶ Although the underlying mechanism of EDS is unclear, it is thought that a central pathway characterized by hypothalamic dysfunction or hypocretin deficiency is present.^8,9

Proper measurement of daytime sleepiness among individuals with PWS is needed to accurately demonstrate clinical benefit derived from medical interventions. The Epworth Sleepiness Scale was developed in 1990 for the measurement of daytime sleepiness in adults with and without sleep disorders; daytime sleepiness represents a concept independent of transient variations in sleepiness and is defined as the individual’s average propensity for sleep during everyday life.^10,11 This concept differentiates from fatigue and drowsiness in that daytime sleepiness as measured by the Epworth Sleepiness Scale denotes the likelihood of transitioning from alert wakefulness to sleep under certain scenarios.¹² Since its development, this measure has been used in numerous investigations across different therapeutic areas worldwide. The Epworth Sleepiness Scale for Children and Adolescents (ESS-CHAD) is the version of the Epworth Sleepiness Scale adapted for use in pediatric populations via the modification of items to allow for better applicability (eg, removal of the reference to alcohol in item 7 to “Sitting quietly by yourself after lunch”).¹² Both child (self-report) and parent/caregiver versions of the ESS-CHAD are available (https://epworthsleepinessscale.com/about-the-ess-chad/).

Previous research evaluated the measurement properties of the child version of the ESS-CHAD in adolescents without sleep disorders aged 12 to 18 years old using exploratory factor and Rasch analyses and found this instrument to be unidimensional with good model fit,¹³ indicating that the ESS-CHAD measures a single underlying construct. In a separate study,¹⁴ the content validity of a provisional, child version of the ESS-CHAD was examined for use in narcolepsy type I from the caregiver and patient perspectives; the findings resulted in the recommendation of 2 additional child versions with different recall periods and content presentation specific to narcolepsy type I. To the best of our knowledge, these additional child versions of the ESS-CHAD have not been re-examined for suitability among patients with narcolepsy type I.

The Patient Focused Drug Development initiative by the U.S. Food & Drug Administration (FDA) began in 2012 under the fifth authorization of the Prescription Drug User Fee Act. Through patient engagement, this initiative has demonstrated that patients are the experts on what it is like to live with their condition and that their chief complaints may not be factored explicitly into measures of treatment benefit in clinical trials. These findings have led the FDA to systematically evaluate clinical outcome assessments (COAs) using established guidance¹⁵ to assess whether these instruments capture what matters most to patients particularly if meant to inform regulatory decision-making on clinical benefit. The guidance structures the review criteria for determining if a COA is fit-for-purpose, which includes a review of the instrument’s content validity in the intended population and context of use. Fit-for-purpose is a regulatory conclusion that the level of validation associated with a COA is adequate to support measurement of a planned clinical trial endpoint within an intended context of use (ie, clinical trial objective and population). Content validity, defined as qualitative evidence that an instrument captures all of the most important concepts to patients and that the items are relevant and well understood, thus demonstrating that the instrument measures an intended concept of interest within a particular context of use,¹⁵ is the primary evidence reviewed by the FDA when determining if an instrument is fit-for-purpose. From an FDA perspective, content validity must be demonstrated prior to the evaluation of any other type of validity (ie, construct validity [convergent, divergent, known-groups validity]), reliability, and the ability to detect clinically meaningful individual-level change; demonstration of these other aspects of an instrument’s measurement properties does not, from a regulatory perspective, circumvent any issues identified based on the content validity evidence.

To date, no content validity assessment of the parent/caregiver version of the ESS-CHAD has been conducted to establish if this instrument adequately measures daytime sleepiness in individuals with PWS. Consequently, this qualitative study aimed to examine the content validity of the parent/caregiver version of the ESS-CHAD to determine if this instrument adequately measures daytime sleepiness in PWS and has the potential to characterize clinical benefit in ongoing and planned clinical trials of this therapeutic target. This study did not set out to assess other measurement properties (construct validity, reliability, ability to detect individual-level change) which, from a regulatory perspective, are evaluated only after content validity has been established.

METHODS

Study design and population

This cross-sectional, qualitative, dyadic interview study was conducted in the United States with caregivers and their care recipients. Recruitment advertisements with contact information for the research team were disseminated via the Foundation for Prader-Willi Research. Each eligible, consenting dyad participated in a single 90-minute semistructured video interview between April 27, 2020 and June 1, 2020. This study was reviewed and approved by the Advarra Institutional Review Board in accordance with the 45 CFR Part 46, Subpart D federal regulations which provide for additional protections for children as research participants.

Eligibility criteria

Dyads were eligible if both caregiver and care recipient met the study criteria. Caregivers met the inclusion criteria if they were the primary caregiver of a person with genetically confirmed PWS; were 18 years of age or older at the time of the interview; could speak, read, write, and understand English; and be able to provide voluntary written informed consent. Care recipients were eligible if they had received a genetic diagnosis of PWS, were at least 6 years of age at the time of the interview, had at least moderately troublesome daytime sleepiness based on caregiver report, and were able to provide voluntary written informed consent, or assent, as appropriate. Both caregivers and care recipients must have been willing and able to participate in a single 90-minute dyadic video interview, agreed to have the interview recorded, and had access to the internet. Dyads were ineligible if they were unwilling or unable to comply with any one of the study requirements; if the care recipient had a diagnosis of another genetic, hormonal, or chromosomal disorder distinct from PWS; or if the care recipient was or had been enrolled in a pitolisant clinical trial.

The ESS-CHAD

The parent/caregiver version of the ESS-CHAD is presented in Figure 1. The instrument measures daytime sleepiness defined as sleep propensity, ie, the likelihood of making the transition from alert wakefulness to sleep under a certain set of circumstances, and is to be completed by the parent or caregiver.¹² The parent or caregiver rates the likelihood of their child or care recipient falling asleep while engaging in each of the 8 activities listed over the past month using a 4-point scale: 0, would never fall asleep; 1, slight chance of falling asleep; 2, moderate chance of falling asleep; and 3, high chance of falling asleep. If the child or care recipient has not engaged in an activity listed, parents/caregivers are asked to imagine how the activity would have affected the care recipient’s likelihood of falling asleep. All activities are weighted equally and scored as 0, 1, 2, or 3, and then summed. The total ESS-CHAD score ranges from 0 to 24, with higher scores representing greater daytime sleepiness as perceived by parents/caregivers.

Figure 1. The Epworth Sleepiness Scale for Children and Adolescents (ESS-CHAD), as rated by parents/caregivers.

“Epworth Sleepiness Scale for Children and Adolescents (ESS-CHAD), parent/carer to answer” copyright © MW Johns. Reproduced with permission. Contact information and permission to use: Mapi Research Trust, Lyon, France, https://eprovide.mapi-trust.org.

Interview procedures

The qualitative interviews were conducted by 2 doctoral-level researchers with mixed-methods expertise at Labcorp Drug Development. For the concept elicitation portion of each interview, open-ended questions were used to characterize general experiences with daytime sleepiness as reported by caregivers and their care recipients. This portion of the interview further served as a basis for determining if care recipients could reliably self-report their experiences. Once care recipients completed this portion of the interview, they were allowed to leave the area where the interview was taking place or stay with their caregiver for the remainder of the interview.

Following concept elicitation, caregivers were cognitively interviewed on the ESS-CHAD; care recipients did not participate in this part of the interview. Specifically, caregivers were asked to complete the ESS-CHAD and to describe their own interpretations of the content. Probes were used to uncover any problems with comprehension or interpretation.

Following the cognitive interview, caregivers completed a sociodemographic form. Dyads were remunerated $100 in the form of gift cards upon study completion.

Data analysis

Characteristics of the study sample were quantitatively summarized. Continuous data were reported as means and ranges, and categorical data were presented as percentages.

Each interview was professionally transcribed for qualitative analysis. Coding and analysis conformed to best practices^16,17 and followed principles in line with qualitative thematic analysis¹⁸ with additional features drawn from grounded theory.^19,20 To aid in the content validity assessment of the ESS-CHAD, saturation of concepts, namely the point at which no new concepts are identified through analysis of interview transcripts,^17,21,22 was evaluated. For this study, the activities described by caregivers as being associated with their care recipients’ likelihood for daytime sleepiness were assessed for saturation. This assessment was specific to activities and not to the elicited symptoms and impacts, as the goal of this study was to assess the conceptual coverage of the ESS-CHAD and consequently make a determination regarding the extent to which this instrument measures daytime sleepiness in PWS.

For the cognitive interview component of this study, a formalized system of assessment was employed to evaluate 3 aspects of caregiver comprehension (determined a priori): literal, inferential, and evaluative.²³ Each caregiver needed to demonstrate a minimum of 2 of the 3 comprehension components to show adequate understanding of that question or set of instructions. Adequate comprehension in the overall caregiver sample was determined if ≥ 80% of caregivers showed comprehension in each of the 3 aspects.²⁴

Conceptual framework for daytime sleepiness in Prader-Willi syndrome

Per FDA guidance,¹⁵ the adequacy of an instrument for measuring an intended concept within a specific population and context of use is demonstrated by the conceptual framework, which illustrates via a diagram the relationships between the items in a COA instrument and the concept(s) measured. For this study, a conceptual framework was developed based on the qualitative evidence generated from the interviews to assess the extent to which the items and concept of interest, ie, daytime sleepiness, of the ESS-CHAD are appropriate, relevant, and comprehensive from the caregiver perspective.

RESULTS

Of the 31 dyads who contacted the research team for potential inclusion, 18 dyads (58.1%) met eligibility criteria and were interviewed (Figure 2). Characteristics of the sample are shown in Table 1. All caregivers (mean age 49.0 years, range 30–61) were White, non-Hispanic/-Latino mothers of care recipients diagnosed with PWS (mean age 14.4 years, range 6–36) who experienced moderate to extremely troublesome daytime sleepiness. The mean ESS-CHAD score was 12.7 (range 9–21).

Figure 2. Dyad selection process.

Table 1.

Sample characteristics.

Characteristic	Caregiver (n = 18)	Care Recipient (n = 18)
Age, y, mean (range)	49.0 (30–61)	14.4 (6–36)
Sex, n (%)
Female	18 (100.0)	8 (44.4)
Male	0 (0.0)	10 (55.6)
Race, n (%)
White	18 (100.0)	18 (100.0)
Ethnicity, n (%)
Not Hispanic or Latino	18 (100.0)	18 (100.0)
Relationship to care recipient, n (%)
Mother	18 (100.0)
Caregiver highest level of education, n (%)
Graduate degree	8 (44.4)
College degree	7 (38.9)
Some college/associate’s degree	3 (16.7)
Care recipient highest level of education, n (%)
High school graduate		5 (27.8)
High school		2 (11.1)
Elementary school		9 (50.0)
Pre/kindergarten		2 (11.1)
Caregiver-reported severity of daytime sleepiness, n (%)a
Moderately troublesome		9 (50.0)
Very troublesome		7 (38.9)
Extremely troublesome		2 (11.1)
ESS-CHAD score, mean (range)b		12.7 (9–21)

^aBased on caregiver report at screening. ^bBased on completion in the cognitive interview (standard deviation = 3.50, median score = 12.0); the total possible ESS-CHAD score ranges from 0 to 24, with higher scores representing greater daytime sleepiness as perceived by parents/caregivers. ESS-CHAD = Epworth Sleepiness Scale for Children and Adolescents.

Signs, symptoms, and impacts of daytime sleepiness, as reported by care recipients

Nine care recipients (50.0%) were able to self-report on their signs and symptoms of daytime sleepiness (Table 2). The most prevalent (≥ 20.0%) self-reported signs and symptoms were drowsiness/grogginess (n = 3; 33.3%), tiredness (n = 3; 33.3%), desire to lie down (n = 2; 22.2%), and exhaustion (n = 2; 22.2%). Less common were achiness, feeling unwell, and yawning. The remaining 9 care recipients (50.0%) were either unable or unwilling to describe the signs and symptoms associated with their daytime sleepiness despite probing by the interviewer or prompting by their caregiver.

Table 2.

Signs, symptoms, and impacts of daytime sleepiness, as reported by care recipients.

Self-Reported Signs and Symptoms	n (%)
Drowsiness/grogginess	3 (33.3)
Tiredness	3 (33.3)
Desire to lie down	2 (22.2)
Exhaustion	2 (22.2)
Achiness	1 (11.1)
Feeling unwell	1 (11.1)
Yawning	1 (11.1)
Self-Reported Impacts
Irritability and crankiness	4 (40.0)
Inability to focus or follow instructions	2 (20.0)
Feeling happy	2 (20.0)
Feeling “weird” or not normal	1 (10.0)
Feeling sad	1 (10.0)
“It’s boring”	1 (10.0)

Nine care recipients (50.0%) were either unable or unwilling to describe their experienced signs and symptoms of daytime sleepiness despite being probed and therefore are excluded from the estimates shown; 8 care recipients (44.4%) were either unable or unwilling to describe the impacts experienced despite probing and are excluded from the estimates presented.

When care recipients were asked about how being sleepy during the day made them feel, 10 (55.6%) could report on the various impacts of their daytime sleepiness (Table 2). Impacts included feelings of irritability and crankiness (n = 4; 40.0%), an inability to focus or follow instructions (n = 2; 20.0%), and feelings of happiness (n = 2; 20.0%). Less common impacts included feeling “weird” or not normal (n = 1; 10.0%), feeling sad (n = 1; 10.0%), and feeling bored (n = 1; 10.0%). Eight care recipients (44.4%) were either unable or unwilling to describe the impacts associated with their daytime sleepiness despite probing by the interviewer or prompting by their caregiver.

Signs, symptoms, and experienced impacts of daytime sleepiness, as reported by caregivers

Interviewers asked caregivers to describe the signs and symptoms they observed in relation to their care recipient’s daytime sleepiness as well as the associated impacts experienced; these results are shown in Table 3. The most prevalent (≥ 20.0%) and observable signs and symptoms were being or experiencing the following: sleepy/sleepiness (n = 17; 94.4%), tired/tiredness (n = 16; 88.9%), exhaustion/exhausted (n = 5; 27.8%), exhibiting anxiety or stress (n = 5; 27.8%), being irritable or frustrated (n = 5; 27.8%), having tantrums or outbursts (n = 5; 27.8%), and lethargy (n = 4; 22.2%). Less common signs and symptoms spanned a range of descriptions from picking at skin/pulling hair (n = 3; 16.7%) to having low energy (n = 1; 5.6%) and being wiped out (n = 1; 5.6%).

Table 3.

Observable signs, symptoms, and experienced impacts of daytime sleepiness, as reported by caregivers (n = 18).

Observable Signs and Symptoms	n (%)
Sleepy/sleepinessa	17 (94.4)
Tired/tirednessa	16 (88.9)
Exhaustion/exhausted	5 (27.8)
Anxious/stressed	5 (27.8)
Irritable/frustrated	5 (27.8)
Tantrums/outbursts	5 (27.8)
Lethargy/lethargic	4 (22.2)
Picking at skin/pulling hair	3 (16.7)
Afternoon slump	2 (11.1)
Fatigue	2 (11.1)
Hitting a wall	2 (11.1)
Poor behavior	2 (11.1)
Stupor/in a fog	2 (11.1)
Zoning/zoned out	2 (11.1)
Yawning	2 (11.1)
Aggressively food-driven	1 (5.6)
Cries easily	1 (5.6)
Low energy	1 (5.6)
Seizure-like fit before falling asleep	1 (5.6)
Sluggish	1 (5.6)
Wiped out	1 (5.6)
Impactsb
Mental health
Focus problems	9 (50.0)
Comprehension problems	5 (27.8)
Retention problems	2 (11.1)
Emotional health
Irritability	5 (27.8)
Tantrums and outbursts	5 (27.8)
Anxiety and stress	3 (16.7)
Embarrassment	1 (5.6)
Physical health
Reduced physical activity	6 (33.3)
Problems with nighttime sleep	6 (33.3)
Association with illness	1 (5.6)
Social health
Family function	7 (38.9)
Problems with peer relationships/engagements with others	7 (38.9)
Missed educational opportunities	5 (27.8)
Missed social opportunities	4 (22.2)

^aMost caregivers (n = 13; 72.2%) used “sleepy” and “tired” interchangeably. ^bImpacts are not mutually exclusive (ie, a caregiver could have reported multiple impacts).

Caregivers reported a variety of impacts due to their care recipient’s daytime sleepiness (Table 3). The most common (≥ 20.0%) mental health impacts were difficulty in maintaining focus (n = 9; 50.0%) and difficulty understanding new information (n = 5; 27.8%). Increased irritability (n = 5; 27.8%) and tantrums and outbursts (n = 5; 27.8%), which were also perceived as signs of daytime sleepiness by caregivers, were reported as common emotional health impacts. The most frequent physical health impacts were reduced physical activity (n = 6; 33.3%) and problems with nighttime sleep (n = 6; 33.3%). Social health impacts consisted of negative effects on family function (n = 7; 38.9%), problems with peer relationships/engagement with others (n = 7; 38.9%), missed educational opportunities (n = 5; 27.8%), and missed social opportunities (n = 4; 22.2%).

Activities associated with daytime sleepiness, as reported by caregivers

Caregivers were also asked about the daytime activities associated with their care recipient’s propensity to fall asleep. Saturation, ie, the point at which no new concepts were identified through analysis of interview transcripts, was reached after the first 4 interviews (Table 4). Several activities were associated with the propensity for sleep: riding in the car; quiet activities, including watching a screen, talking with another person, playing quietly, and playing video games or using a computer; mental work or focus, such as school work; lying down; physical exertion; eating, during or after; and inactivity or boredom. Representative caregiver statements were as follows:

Table 4.

Saturation of daytime activities associated with care recipient sleep propensity, as reported by caregivers (n = 18).

	Transcript Group Where Activity First Appeareda
	Group 1 (n = 4)	Group 2 (n = 4)	Group 3 (n = 4)	Group 4 (n = 6)
Activity
Riding in a car	✓
Watching a screenb	✓
Readingb	✓
Talking with another personb	✓
Playing quietlyb	✓
Playing video games or using computerb	✓
Mental work or focus	✓
When lying down	✓
After physical exertion	✓
During or after eating	✓
Inactivity or boredom	✓
Number of new activities appearing in each transcript group	11	0	0	0
% of total new concept codes	100	0	0	0

^aSaturation, meaning the point at which no new concepts are identified through analysis of interview transcripts, was achieved after the first four transcripts. ^bDiscussed by caregivers as a quiet activity.

Riding in the car

In the car. 100%, in the car every time he’ll fall asleep in three minutes. [H002]

Yeah, she does fall asleep in the car, always. Even if it’s short rides to the grocery store, she’ll fall asleep. [H013]

Quiet activities

Yes, if it’s quiet around here, if he’s watching TV, if he’s playing a quiet activity by himself, I know like he said if he watches YouTube, anything where he’s sedentary for a certain amount of time is when he’ll get overly tired. [H008]

…if she’s just sitting and watching a show and she’s not moving, she’ll fall asleep. [H017]

Mental work or focus

When he is at school, they tell me he’s very sleepy, and his teacher says he falls asleep in class. [H002]

In school he would fall asleep somewhat regularly. [H004]

When lying down

He lays down, he’s asleep within 15 seconds, he sleeps, and then wakes up kind of a new kid. [H014]

She’ll go climb in bed, and then she’ll be asleep. Like instantly asleep. [H016]

After physical exertion

… little kids’ league basketball at the school, he was really good for 15 minutes, and after all the running, and after all the instructions, then all of a sudden with all the instructions from the coach, he was just overwhelmed, and he was done. He said, “I’m tired, and I don’t want to do it anymore.” [H002]

And then just anytime we have a lot of physical activity. That usually takes it out of her. [H017]

During or after eating

He will lay down on the chair and actually fall asleep while eating. [H008]

…we still have days where she’ll be eating, and you see her eyes kind of roll, and you see her trying to kind of stay awake. [H017]

Inactivity or boredom

I think it’s when it’s the activity that he’s not chosen or isn’t a high-interest activity. So it goes with the boredom. [H004]

Basically any time that she’s sitting very still. [H017]

Mitigation strategies for daytime sleepiness

When asked about the mitigation strategies used for their care recipients’ daytime sleepiness, all caregivers reported attempting to mitigate their care recipient’s daytime sleepiness using one or more interventions. The most commonly reported intervention was the use of prescription medications or stimulants (n = 11; 61.1%); questions on which specific medications used were not asked. Other reported methods included physical activity to distract from sleepiness (n = 8; 44.4%), having a flexible daytime schedule (n = 6; 33.3%), and adhering to a strict nighttime sleeping schedule (n = 4; 22.2%).

Cognitive debrief of the ESS-CHAD

All caregivers reported that the instructions for the ESS-CHAD (Figure 1) were easy to read and understand, with one caregiver stating: “I was able to read through the instructions and understand what to do without getting bogged down.” [H018] All caregivers demonstrated comprehension of the instructions and reported that the ESS-CHAD was easy to answer.

All caregivers stated that the ESS-CHAD captured relevant activities in which their care recipient was likely to fall asleep and that the response options were distinct and easy to differentiate. No activity was reported by caregivers to be more relevant than any other included in the ESS-CHAD. Most caregivers (n = 14; 77.8%) found the items to be distinct and nonrepetitive, with 2 caregivers stating that the word sitting was repetitive across items. Caregivers demonstrated comprehension of the ESS-CHAD items; overall, ≥ 90% of caregivers showed comprehension for each item.

Caregivers were asked to reflect on the 1-month recall period of the ESS-CHAD. Over half of the caregivers (n = 10; 55.6%) found the past month to be appropriate. Of the remaining caregivers, 2 (11.1%) reported that a shorter time period would be easier to remember. The other 6 caregivers (33.3%) stated that a longer recall period would allow for a more accurate picture as behavior can vary month to month.

Conceptual framework for daytime sleepiness in Prader-Willi syndrome

The conceptual framework for daytime sleepiness in PWS is shown in Figure 3 and demonstrates that the items in the ESS-CHAD map to the activities in which care recipients are likely to experience daytime sleepiness, as reported by caregivers. Only 2 concepts, “after physical exertion” (n = 6, 33.3%) and “while inactive or bored” (n = 11, 61.1%), did not map to the ESS-CHAD. Caregiver statements, however, indicate that these concepts are closely linked to daytime activities but are atypical of daytime routines. The caregivers who reported daytime sleepiness after physical exertion characterized this as resulting in an increased sleepiness later in the day, often occurring during another activity:

Figure 3. Conceptual framework for daytime sleepiness in Prader-Willi syndrome.

Concepts elicited from caregivers that are highlighted in green map to the items in the ESS-CHAD. The concepts highlighted in purple do not map directly onto the ESS-CHAD; these concepts, however, were closely linked to care recipients’ daytime activities but were atypical of daily routines, based on caregiver statements. ESS-CHAD = Epworth Sleepiness Scale for Children and Adolescents.

Yeah, I mean, if she does a lot of physical activity, she definitely gets more sleepy during the day. If we go out and do a walk or a hike or something or kind of earlier in the day, later on like that 3 or 4 o’clock is the sleepy time for her, especially if there’s been physical activity. I see it mostly if she’s kind of sitting down, curling down, like watching something on her computer or on the TV. [H011]

…there are times when we’ve had hikes, he fell asleep eating, just I think because of—the physical activity was a little more strenuous. But that’s rare. [H015]

Inactivity was described by caregivers in association with sitting still or any sedentary activity:

…anything where he’s sedentary for a certain amount of time is when he’ll get overly tired. [H008]

So, sitting pretty much equates to sleeping. [H015]

Boredom was described as occurring during any low-interest activity, which consisted of some subjects at school, listening to someone speak, and watching something on a device where the topic was not of the care recipient’s choosing:

…there’s other times that she just doesn’t have enough to do, her default would be to take a nap. [H007]

…I don’t know if boredom is the word but, you know, not being stimulated in a way that connects with her. Especially boredom because it’s not connecting with her, whether that’s school or someone speaking or a movie. It could be any of those things, but it’s something that she feels a little bored and she’ll go to sleep. [H018]

DISCUSSION

In this qualitative interview study conducted with 18 caregivers and their care recipients with genetically confirmed PWS who experienced daytime sleepiness, we aimed to establish the content validity of the parent/caregiver version of the ESS-CHAD for the measurement of daytime sleepiness in PWS. This novel study found that the caregiver perspective on daytime sleepiness in PWS is essential, particularly given that only approximately half of the care recipients could self-report their symptoms and impacts. Furthermore, this study showed that the ESS-CHAD is fit-for-purpose within the context of daytime sleepiness in PWS, and that it is well understood and interpreted by caregivers. Consequently, the ESS-CHAD merits consideration for the assessment of therapeutic benefit in clinical trials in which the impact of therapy on daytime sleepiness in individuals with PWS is of interest.

This study found that only half of the care recipients could self-report the signs and symptoms of their daytime sleepiness; the remaining half were either unable or unwilling to describe their symptoms despite the interviewer probing and, in some cases, prompting by their caregiver. Similar observations were made regarding the ability to self-report on impacts. On the other hand, caregivers in this study reliably reported on observable signs, symptoms, and impacts of their care recipient’s daytime sleepiness. The most common caregiver-reported signs and symptoms of daytime sleepiness included being visibly sleepy, tired, and exhausted; exhibiting anxiety and stress; being irritable and visibly frustrated; having temper tantrums and outbursts; and being noticeably lethargic. Several mental, emotional, social, and physical health impacts were characterized by caregivers. Although caregivers were not asked to rate the level of importance of each reported symptom and impact, the findings highlight the notable burden of daytime sleepiness in PWS and show that the measurement of patient-centric outcomes in this condition are best captured by the caregiver perspective. Studies involving individuals with intellectual disabilities, and particularly clinical trials evaluating the effects of treatment on patient populations where intellectual disability is central to the condition under study, should consider the inclusion of caregiver-reported outcome measures to best characterize perceived treatment efficacy.

This study showed that the concepts elicited by caregivers with respect to the activities in which care recipients were likely to experience daytime sleepiness mapped well to the content in the ESS-CHAD; saturation of concepts was achieved after the first 4 interviews. Only 2 concepts, “after physical exertion” and “while inactive or bored,” did not map to the ESS-CHAD. Caregiver reports revealed that daytime sleepiness after physical exertion typically led to an increased tiredness later in the day, frequently occurring during another activity, and that the amount of sleepiness experienced following physical activity was atypical. Physical activity was also referred to as a rare event by some of the caregivers, and was typically described as resulting in an immediate “wake up.” For inactivity and boredom, caregivers did not describe these instances as an activity, but instead as a state of physical or mental being that would occur during other activities throughout the day. These concepts were described within the context of any low-interest activity (boredom) or sitting still (inactivity), the latter of which is explicitly specified in all but one item contained in the ESS-CHAD. Overall, caregiver statements showed that these concepts, albeit intricately associated, were inherently different from the core activity-related concepts where care recipients would exhibit a propensity for daytime sleepiness.

The ESS-CHAD was well understood, deemed relevant, and had distinct and interpretable response options; no one item or set of items was reported to be more important than any other. Taken together, equally weighted items are deemed acceptable.¹⁵ The instrument’s recall period (1 month) was found to be appropriate by over half of the caregivers; the remaining caregivers either preferred a longer (33.3%) or shorter (11.1%) recall period. Previous research evaluated the accuracy of 3-, 7-, and 28-day recall periods on the ESS in a large sample of primary care clinic patients and found that the ability to remember was similar across the varying recall periods.²⁵ Hence the use of the ESS-CHAD within a clinical trial setting with longitudinal administration of the instrument is considered appropriate.

In this study, all caregivers were White, non-Hispanic/-Latino, and mothers between 30 and 61 years of age, inclusive. Although this sample may not generalize to the broader population of caregivers of individuals with PWS, the demographic profile of the participants in this study is similar to that of previous research.^26,27 This observation also holds in the Global PWS Registry (Foundation for Prader-Willi Research, Walnut, CA), whereby most who have enrolled to date have been Caucasian (85.1%) and non-Hispanic/-Latino (82.0%).²⁸ The lack of male caregivers in this study is not surprising within the context of current research reporting on notably higher proportions of female caregivers than male caregivers in PWS.^26,27 Differences between male and female caregivers in the reporting of observed signs/symptoms, events, and behaviors like the propensity for daytime sleepiness during certain activities are likely negligible as these are direct observations rather than perceived experiences specific to one’s self. In contrast, broader impacts felt by the caregiver and family as a result of caring for an individual with PWS could reasonably be perceived differently between male and female caregivers; future research specific to PWS in this area would be informative. In our study, 83.3% of the caregivers had some form of an advanced degree (college or graduate level). This estimate exceeds the percentage of the general adult population with a bachelor’s degree or higher based on 2018 census data (32.3%);²⁹ the prevalence of women between the ages of 30 and 59, inclusive, in the 2018 census who attained these advanced degrees is also lower (39.5%) than what was reported in the current sample. Specific to PWS, however, previous research shows that socioeconomic factors like education and income tend to skew toward the higher end of the spectrum.^26–28 Health literacy in particular appears to be quite high among caregivers of individuals with PWS, with 78.1% of the caregivers in 1 study stating that they never needed help reading medical material and 61.5% always having confidence in filling out medical forms.²⁷ These observations may be a direct result of the recruitment strategies employed in studies involving caregivers of individuals with PWS, where individuals oftentimes have self-selected into a study.

Although achieving greater diversity of individuals with PWS and their caregivers in research remains a priority, the external validity of the present study findings is deemed good within the context of the existing literature. Our study did not explore the presence of comorbidities specific to other complex sleep disorders or irregularities of sleep-related breathing (eg, obstructive sleep apnea). Eligibility criteria were intentionally broad given challenges in recruiting in a rare disease and a desire to increase generalizability of the study findings. Certain comorbid conditions common in PWS require objectively measured assessments for accurate diagnosis (eg, overnight polysomnogram for obstructive sleep apnea), which do not correlate strongly with perceived abnormalities.² Consequently, caregivers were not asked about the presence of these other conditions or their treatment nor were they asked to attribute the perceived cause(s) of their care recipient’s daytime sleepiness. As the target population for this study was caregivers and their care recipients with genetically confirmed PWS who experienced daytime sleepiness, the potential presence of these comorbid conditions and their possible impact on the qualitative data generated were not considered. Indeed, an assessment of confounding requires a quantitative approach assuming the pertinent variables are accurately measured and able to be analyzed; such an analysis cannot be meaningfully achieved using qualitative data. Caregivers were not asked if they had ever completed the ESS-CHAD prior to becoming aware of the present study, thus potentially affecting the timing in which saturation was achieved. Although the lack of data on reported comorbid diagnoses and the use of stimulants or wakefulness-promoting medication limits our understanding of the clinical profile in the current study, and therefore may be considered a possible limitation with respect to external validity, it does not affect the ability to properly characterize the care recipient or caregiver experience with daytime sleepiness in PWS.

Although 58.1% of the dyads who contacted the study team were enrolled and interviewed in the current study, we did not examine potential differences in sociodemographic or clinical factors between those included and excluded. The sample size in both groups would have precluded a meaningful between-group assessment. Nonetheless, the implementation of video interviews likely enabled the inclusion of dyads who otherwise would not have been able to attend an in-person interview and therefore is considered a strength of the study design.

In conclusion, the findings from this study emphasize the importance of measuring daytime sleepiness in PWS from the caregiver perspective and establishes the content validity of the ESS-CHAD in this population, thus confirming the appropriateness of this measure for evaluating treatment efficacy in clinical settings. Future studies performing quantitative analyses of the psychometric measurement properties of the ESS-CHAD in PWS are needed; demonstration of this measure’s reliability, construct validity (convergent, divergent, and known-groups validity), and ability to detect clinically meaningful change is warranted. Research facilitating the interpretation of ESS-CHAD scores (cut-off points) in relation to level of daytime sleepiness would also represent a significant contribution to our scientific understanding of daytime sleepiness in PWS.

DISCLOSURE STATEMENT

All authors have reviewed and approved the final, submitted version of this manuscript. Work for this study was performed at Labcorp Drug Development, Gaithersburg, Maryland. This study was funded by Harmony Biosciences. A.P. and K.D. are employees of Harmony Biosciences. E.M. is an employee of Labcorp Drug Development (formerly Covance by Labcorp), which received consultancy fees from Harmony Biosciences. D.G.G. and A.R. are employees of the Baylor College of Medicine, are consultants to and participate in the Harmony Phase 2 clinical trial of pitolisant, and are involved with the TREND community. At the time this research was conducted and completed, V.P.P. was an employee of Labcorp Drug Development.

ABBREVIATIONS

COA,

clinical outcome assessment

EDS,

excessive daytime sleepiness

ESS-CHAD,

Epworth Sleepiness Scale for Children and Adolescents

FDA,

U.S. Food & Drug Administration

PWS,

Prader-Willi syndrome