Clinical Cut Point for the Pittsburgh Sleep Quality Index After Adolescent Concussion

Abstract

Background:

Concussions can have negative implications for sleep quality. Self-report measures, such as the Pittsburgh Sleep Quality Index (PSQI), have been used in clinical and research settings to identify individuals with sleep impairments. However, the accuracy/applicability of historically established PSQI scoring criteria for differentiating good versus poor sleep quality has not been critically examined in adolescents with concussion.

Purpose:

To establish a relevant PSQI clinical cut point for adolescents with a recent concussion.

Study Design:

Cross-sectional study; Level of evidence, 3.

Methods:

Adolescents within 16 days of concussion and uninjured controls completed the PSQI, and a global score of 0 to 21 was calculated. Independent-samples t tests were used to compare PSQI global scores, and logistic regression was used to calculate odds ratios (outcome = group; predictors = PSQI, covariates). A receiver operating characteristic curve was used to evaluate the area under the curve (AUC) and determine the optimal cut point to distinguish between adolescents with and without a concussion.

Results:

A total of 110 adolescents with a concussion (mean age, 14.9 ± 1.6 years; 53% female; 9.8 ± 3.6 days since injury) and 129 uninjured controls (mean age, 15.6 ± 1.1 years; 86% female) were included for analysis. The concussion group had significantly worse (higher) PSQI scores than controls (mean, 7.41 ± 3.62 vs 2.26 ± 1.97; P < .001; Cohen d = 1.8). Both the univariable model and multivariable model (controlling for age, sex, concussion history, history of anxiety and/or depression, and self-reported use of sleep medication) had excellent diagnostic accuracy (univariable AUC, 0.90; multivariable AUC, 0.99). Within the multivariable model, a cut point of 4 correctly classified 81% of participants as concussed or control (sensitivity, 87%; specificity, 74%).

Conclusion:

Adolescents with a concussion demonstrated worse sleep quality than uninjured controls. The results suggest that sleep quality, as measured by the PSQI, can distinguish between adolescents with and without a concussion, using a cut point of 4.

Concussion is prevalent in the United States, affecting approximately 1 in 5 adolescents.^44,45 After a concussion, adolescents experience a myriad of symptoms, which may include headache, dizziness, confusion, or sleep disturbances. Although many adolescents with concussion experience resolution of these symptoms within 1 month, up to 40% may experience symptoms for >1 month.^{20,24,27,43,48} Many studies have evaluated the way in which various factors (ie, specific concussion symptoms, premorbid characteristics, and initial examination findings) are associated with recovery from concussion. Of growing interest is the factor of sleep, which has been linked in recent research to concussion recovery.^{14,19,29,30,39,40}

The most recent consensus statement providing guidance for concussion management identifies poor sleep as a negative modifier of recovery. ³⁰ Sleep-related symptoms are commonly reported after a concussion, including excessive drowsiness, difficulty falling asleep, and difficulty maintaining sleep. Importantly, disruptions in sleep quality or quantity after concussion are associated with prolonged recovery. ³⁰ Adolescent patients who reported changes in sleep (ie, quality and/or quantity) in the first 3 weeks after concussion took 3 to 4 times longer to experience symptom resolution than those who did not. ⁴ Negative changes in sleep quality and quantity after concussion are also associated with worse mental health and academic engagement.^4,22 Therefore, critically evaluating and intervening to improve sleep after concussion should be a focus of clinical management after concussion.

Currently, there are limited guidelines for the assessment and management of sleep after concussion. ³⁰ Recently, questions regarding changes in sleep have been added to the newly developed Sports Concussion Office Assessment Tool (SCOAT).^30,37 The aforementioned questions included in the SCOAT are an important addition to the clinical management of concussion; however, their clinical utility is limited. The questions are adapted from the Athlete Sleep Screening Questionnaire, which is a questionnaire designed to assess sleep problems in elite (uninjured) athletes ² ; it has not yet been validated in individuals with a concussion. An additional, important contributing factor to the lack of formal concussion-specific sleep guidelines is that a concussion can be characterized or described by changes in sleep (eg, difficulty falling asleep and poor sleep quality); however, the diagnostic utility of changes in sleep when evaluating for a concussion remains unknown. Assessment of disrupted sleep in the context of a concussion evaluation is commonly conducted using standardized global concussion symptom inventories. These inventories typically include 18 to 22 commonly experienced symptoms after concussion, including somatic, cognitive, mood, balance, and/or sleep symptoms, yet only 1 to 3 of these symptoms assess specific changes in sleep. ³⁰ However, there are multiple dimensions of sleep health (eg, duration, continuity, timing, alertness, and quality), with changes in one affecting the others. ⁵ Because of this, sleep disruptions can present in a more complex fashion. As such global concussion symptom inventories may not be adequately equipped to address sleep deficits after concussion, and more targeted (ie, sleep specific) questionnaires are needed

Many questionnaires exist for assessing sleep. Among these questionnaires, the Pittsburgh Sleep Quality Index (PSQI) has been used for a variety of different conditions over the past 30 years.^6,28 The PSQI assesses 7 components of sleep quality, yielding a global score that ranges from 0 to 21. A score >5 indicates poor sleep quality with a sensitivity of 90% and specificity of 87%; however, these psychometric properties were established in a sample of young adults with depression. ⁶ Nonetheless, a score >5 to indicate poor sleep quality has been continually used in a large assortment of populations, ²⁸ including adolescents with and without a concussion.^{8,25,34,36,41,46,47} Fichtenberg et al ¹² did determine an optimal cut point >8 in adults after a traumatic brain injury; however, this score was to establish the presence of insomnia in their sample and did not include a control group for comparison. While the identification of poor sleep quality may provide clinicians insight into disruptions in sleep that have occurred after a concussion, the diagnostic value of changes in sleep after concussion remains poorly understood. Specifically, the diagnostic utility of previously reported PSQI cut points >5 and >8 has not been examined among adolescents with a recent concussion.^6,12 Therefore, the purpose of our study was to determine sleep quality differences using self-reported sleep quality between adolescents with and without a recent concussion and to establish an optimal PSQI cut point to distinguish between adolescents with and without a concussion.

Methods

Participants and Study Design

We performed a cross-sectional investigation of adolescent athletes between 2018 and 2023. Participants included adolescents who had recently sustained a concussion (<16 days) and uninjured controls. Participants in the concussion group were recruited from patients receiving treatment at a sports medicine center within a regional children's hospital. Concussion diagnosis was made by a sports medicine physician based on the definition of the most recent international concussion consensus guidelines available at the time of the study.^26,30 For the concussion group, included participants reported an initial symptom severity score ≥9 as measured via the Post-Concussion Symptom Inventory (PCSI), ³² were 13 to 18 years of age at the time of the initial evaluation, and were able to participate in the postconcussion assessment within 16 days ³² of injury. Uninjured control participants were recruited through the local community (ie, local high school and/or club athletes) and participated in an organized sport, had not sustained a concussion in the previous 6 months, and did not report any other coexisting neurological conditions. This study was reviewed and approved by the local institutional review board before commencement. Participants and a parent/guardian for participants <18 years of age provided written informed consent/assent before study participation.

Post-Concussion Symptom Inventory

The PCSI consists of 22 symptoms related to concussion. ³² For each symptom, participants rated the severity on a Likert-like scale ranging from 0 (none) to 6 (severe). We calculated the total symptom severity rating by summing all 22 severity ratings. This resulted in a total concussion symptom severity rating ranging from 0 to 132. The PCSI has been validated in an adolescent population (13-18 years of age). ³²

Pittsburgh Sleep Quality Index

The PSQI is a 19-item sleep questionnaire that assesses a variety of factors related to sleep quality, using a combination of mean time responses (eg, bed and wake times), frequency ratings (eg, never, once per week, twice per week, etc), and quality ratings (very/fairly good or very/fairly bad). The 19 items provide the ability for participants to rate aspects within 7 sleep components: subjective sleep quality, sleep latency, sleep duration, sleep efficiency, sleep disturbances, use of sleep medications, and daytime dysfunction. These 7 component scores are each equally weighted on a scale of 0 to 3 and subsequently summed to yield a PSQI global score, ranging from 0 to 21, with a score >5 indicating poor sleep quality. The concussion group rated their sleep since the night after their concussion occurred, and the control group rated their typical sleep over the past month.

Statistical Analysis

Demographic (age and biological sex), medical history (anxiety and/or depression and previous concussion), and PSQI component score variables as well as PSQI global scores between concussed and control groups were compared using independent t tests for continuous variables and Fisher exact tests or chi-square analyses for categorical variables. Additional independent t tests were performed on raw data (ie, individual question response as opposed to component score) of PSQI questions 1 to 4 (“During the past month or since your injury, what time have you usually gone to bed?”; “During the past month or since your injury, how long in minutes has it usually taken you to fall asleep each night?”; “During the past month or since your injury, what time have you usually gotten up in the morning?”; and “During the past month or since your injury how many hours of actual sleep did you get at night?”). We compared PSQI global scores between groups using independent t tests and then constructed a univariable logistic regression model to assess the association between concussion/control groups and PSQI global score (unadjusted model). We calculated the Cohen d effect size to determine between-group differences in PSQI global score and interpreted effect sizes as large (>0.8), moderate (0.50-0.79), or small (0.20-0.49). ²³ We also constructed a multivariable logistic regression model to evaluate the association between PSQI rating and group, using age, sex, concussion history, diagnosis of depression and/or anxiety, and self-reported use of sleep medication (via the PSQI) as covariates, as they have previously been observed to modify concussion outcomes. ²¹

From the logistic regression model, we generated a receiver operating characteristic (ROC) curve to identify the ability of the PSQI to distinguish between groups. We then calculated the area under the ROC curve (AUC) value, with the 95% confidence interval, to determine the diagnostic accuracy of the PSQI global score between concussion and control groups. The ability of the AUC to distinguish between concussed and control participants was defined as excellent (≥0.90), good (0.80-0.89), fair (0.70-0.79), and poor (≤0.69). ³³ The optimal PSQI global score cut point that distinguished between groups was identified by maximizing the Youden Index for the concussion and control groups. The Youden Index was calculated as sensitivity + specificity – 1, with scores ranging from 0 to 1 (score of 1 indicates perfect test). ¹³ Previous research has suggested a cut point of 8 in an adult population after sustaining a traumatic brain injury. ¹² As such, an additional univariable model was run using the PSQI global score cut point of 8. Statistical analyses were performed using R (Version 4.2.2; R Core Team) and the cutpointr package. ⁴²

Results

A total of 239 adolescents participated in the study, including 110 adolescents with a concussion (mean age, 14.9 ± 1.6 years; 53% female; 9.8 ± 3.6 days since injury) and 129 uninjured adolescent controls (mean age, 15.6 ± 1.1 years; 86% female) (Table 1.) Across both groups, 90 (38%) adolescents reported the use of sleep medication (concussion: 62; control: 28), whereas 149 (62%) reported not taking any sleep-promoting medicine (concussion: 48; control: 101). Participants with a concussion in the previous 16 days had significantly worse PSQI global scores (mean, 7.41 ± 3.62) compared with control participants (mean, 2.26 ± 1.97), with a large effect size (Cohen d = 1.8) between groups (Figure 1). Participants with a concussion also had significantly worse sleep duration and daytime dysfunction component scores and reported more sleep medication use compared with uninjured controls (Table 2). We also identified longer sleep latency times, as well as earlier bed and wake times among those with a concussion, compared with those without (Table 3).

Table 1.

Participant Characteristics Stratified by Control and Concussion Groups ^a

	Concussed (n = 110)	Control (n = 129)	P Value
Age, y, mean (SD)	14.9 (1.68)	15.6 (1.15)	<.001
Biological sex
Female	58 (53)	111 (86)	<.001
Male	52 (47)	18 (14)
Diagnosis of anxiety and/or depression
No	88 (80)	99 (77)	.637
Yes	22 (20)	30 (23)
Prior concussion history
No	60 (55)	93 (72)	.0067
Yes	50 (45)	36 (28)
Use of sleep medication
No	48 (44)	101 (78)	<.001
Yes	62 (56)	28 (22)

^aData are presented as n (%) unless otherwise indicated.

Figure 1.

Pittsburgh Sleep Quality Index (PSQI) global scores between groups (concussed: mean, 7.41 ± 3.62; control: mean, 2.26 ± 1.97).

Table 2.

PSQI Component Scores Stratified by Control and Concussion Groups ^a

	Concussed (n = 110)	Control (n = 129)	P Value
PSQI component
Subjective sleep quality	1.12 (0.700)	0.876 (0.637)	.00596
Sleep latency	1.34 (0.998)	1.36 (0.901)	.822
Sleep duration	0.318 (0.729)	0.620 (0.822)	.00289
Sleep efficiency	0.536 (0.955)	0.504 (0.858)	.784
Sleep disturbances	1.22 (0.612)	1.26 (0.577)	.627
Use of sleep medications	1.38 (1.32)	0.465 (0.969)	<.001
Daytime dysfunction	1.50 (0.896)	0.977 (0.795)	<.001
PSQI global score	7.41 (3.62)	2.26 (1.97)	<.001

^aData are presented as mean (SD). PSQI, Pittsburgh Sleep Quality Index.

Table 3.

Mean Differences Between Groups for Individual Responses (ie, Raw Data) to Questions 1 to 4 on the PSQI ^a

	Concussed	Control	P Value
Bedtime	11:09 pm	12:03 am	<.001
Wake time	7:26 am	8:19 am	<.001
Hours of sleep, mean (SD)	8.2 ± 1.65	7.7 ± 1.43	.229
Sleep latency (minutes to fall asleep), mean (SD)	29.1 ± 28.81	22.6 ± 17.74	.045

^aBold P values indicate statistical significance (P < .05). PSQI, Pittsburgh Sleep Quality Index.

In the univariable/unadjusted logistic regression model, worse PSQI global scores were significantly associated with concussion (OR, 1.89; 95% CI, 1.61-2.21; P < .001). After adjusting for age, sex, history of concussion, and diagnosis of depression and/or anxiety in the multivariable model, worse PSQI scores were still significantly associated with greater odds of being in the concussion group (OR, 1.90; 95% CI, 1.60-2.32; P < .001) (Figure 2.) The ROC analysis suggested that when analyzed independently, the PSQI global score demonstrated excellent ability to distinguish between concussion and control groups (unadjusted model: AUC, 0.90; 95% CI, 0.87-0.94) (Figure 3). After adjusting for age, sex, history of concussion, diagnosis of depression and/or anxiety, and self-reported use of sleep medication, the multivariable model AUC indicated the PSQI had an excellent ability to distinguish between the concussion and control groups (adjusted model: AUC, 0.99; 95% CI, 0.90-0.96) (Figure 3). The multivariable model correctly classified 81% of participants as a concussion or control group participant at a PSQI global score cut point >4 (sensitivity, 87%; specificity, 74%). Comparatively, using the traditionally used PSQI global score cut point >5, the model correctly classified 79% of participants as a concussion or control group participant (sensitivity, 64%; specificity, 91%). Using a cut point of 8, previously reported to identify insomnia among adults after sustaining a traumatic brain injury, ¹² our model correctly identified 70% of participants as concussed or control (sensitivity, 37%; specificity, 99%).

Figure 2.

After adjusting for age, biological sex, history of concussion, history of depression and/or anxiety, and self-reported use of sleep medication, worse (higher) Pittsburgh Sleep Quality Index (PSQI) global scores were significantly associated with greater odds of being in the concussed group. *P < .01; **P < .001.

Figure 3.

Both the univariable and multivariable models (controlling for age, sex, concussion history, history of anxiety and/or depression, and self-reported use of sleep medication), had excellent diagnostic accuracy (univariable area under the curve [AUC], 0.90; multivariable AUC, 0.99). ROC, receiver operating characteristic.

Discussion

Our results indicate that adolescents with a recent concussion report worse sleep quality than those without a concussion. As sleep quality may be negatively affected by concussion, this measure, when using a cut point of 4, may help to distinguish poor sleep quality between adolescents with and without a concussion in the weeks after injury. The primary clinical effect of our study, however, is in providing an evidence-based cut point for the PSQI, a commonly used measure of sleep, to differentiate normal variations in sleep quality among healthy/nonconcussed adolescents from the elevated sleep disturbances observed after concussion. Our cut point may provide a threshold for identifying patients after concussion who are sleeping within the anticipated range of healthy adolescents (ie, <4 points on the PSQI) and those who may require intervention to improve sleep quality and facilitate concussion recovery. Past work has indicated that disruptive changes in sleep after concussion are associated with worse symptoms and longer recovery relative to those who do not have sleep-related problems after concussion.^{4,18,29,39,40} This past work, in conjunction with our results, suggested that the assessment of sleep quality after concussion may be a clinically relevant and feasible approach to identify postconcussion sleep problems associated with a higher risk of developing persisting symptoms.

In recent years, there has been a substantial increase in studies evaluating the association between concussion and sleep deficits. ^¶ As such, consensus statements regarding concussion management have highlighted the importance of addressing sleep, given the potential for sleep-related deficits to negatively influence patient-reported outcomes and overall recovery.^16,30 However, a gap exists in knowledge to clinical practice, as the role of sleep in the diagnosis of concussion has not been addressed. Although previous research using the PSQI in a population with concussion has shown that sleep quality is negatively affected,^{12,14,18,36,40} the lack of established scoring criteria specific to concussion poses a risk regarding the interpretation of the results and their clinical utility. For example, in adult patients with mild traumatic brain injury, a cut point of 8 is clinically significant for the presence of insomnia. ¹² However, in our sample, a cut point of 8 had a 70% classification accuracy for control and concussion groups (sensitivity, 37%; specificity, 99%). Similarly, our data showed that a cut point of 5 had a 79% accuracy (sensitivity, 64%; specificity, 91%). If either score were used in adolescents with concussion, there would be a risk of potentially misclassifying adolescents with concussion who may benefit from early intervention to address sleep problems.

Adolescents may not report the extent of their sleep disturbances after concussion, and as such, health care providers are faced with a unique challenge to identify undetected sleep disturbances. Knowing how sleep patterns change in adolescents who have sustained a concussion may aid in identifying sleep problems. For example, when looking at the mean bedtimes of participants in our study, those with a concussion reported earlier bedtimes than those without a concussion. However, despite going to bed earlier (potentially in hopes of getting more sleep), adolescents with a concussion demonstrated longer sleep latency (ie, the time it takes for an individual to fall asleep). ³⁵ Difficulty falling asleep is a commonly reported symptom after a concussion and can be further appreciated when looking at the reporting of sleep medication use between groups. A greater need for sleep (as evidenced by the earlier reported bedtime) but difficulty falling asleep may contribute to the increased use of medication in the concussed group. These disturbances ultimately affect the overall PSQI global score as they each contribute to a component score and may go further undetected if clinicians choose to use the traditional clinical cut point of a PSQI global score >5 to identify sleep-related problems after adolescent concussion. This may result in missed opportunities to provide interventions that might aid in concussion recovery.

To understand why the clinical cut point for poor sleep quality in adolescents with a concussion may be lower than previously reported cut points (ie, >5), we should consider the unique factors that affect the sleep patterns of adolescents. Specifically, sleep quality may be reflective of physiological, psychosocial, and environmental factors that uniquely affect adolescents, and as such, an ideal cut point for the PSQI may be variable across age and population. Circadian phase preference refers to an individual's preference for sleep and wake timing.^1,3,9 As adolescents get older, there is a preference for later sleep and wake times. ¹⁵ Additionally, the homeostatic drive for sleep, also known as Process S, changes throughout the adolescent developmental period. ¹⁰ Process S (ie, the need to sleep) naturally builds the longer an individual is awake, with a greater accumulation resulting in a greater need for sleep. ³ In mature adolescents (ie, Tanner stage 5), there is a slower accumulation of Process S, compared with prepubertal adolescents, resulting in less drive to go to sleep and potentially contributing to a preference for later sleep times.^10,38 This slowing of Process S may support why mature adolescents take longer to fall asleep compared with prepubertal adolescents; a slower accumulation of Process S during adolescent development allows the older adolescent to stay awake longer. ¹⁰ Indeed, adolescents report going to bed later as they get older, which may further be compounded by accompanying evening academic and extra-curricular demands. While these factors align with the shift in preference to late sleep and late wake times across development, early school start times during the week do not. This results in decreased total sleep and subsequent daytime sleepiness, both of which are components of the PSQI global score. ⁷ On the weekends, however, a reversal of sleep patterns is observed in which adolescents have both later sleep and wake times, likely reflecting their natural, and preferred, circadian rhythm. This difference in weekday and weekend sleep schedules creates an inconsistent sleep pattern, ultimately affecting the overall quality of sleep and providing potential reasoning for why a clinical sleep quality cut point is different for adolescents compared with adults.

Existing sleep questionnaires that are frequently used for concussion clinical care and research were initially developed for populations unrelated to patients with concussion, with unknown validity among adolescent patients. ¹¹ This poses limitations for their clinical utility in our population of interest. Many existing questionnaires assess different elements of sleep health or sleep deficits, and as such, the definition and measurement of sleep vary. As the definition of the sleep dimension (eg, duration, variability, and timing) being measured is ultimately dictated by the type of study and population in which it is being assessed, it is important that psychometric properties of the questionnaire be examined in the population for which it is being used. While a specific questionnaire developed for assessing sleep in adolescents with a concussion does not yet exist, our results provide guidance for clinicians and researchers using the PSQI to distinguish between those with and without concussion when using a cut point of 4 in this population.

Limitations

Our study was not without limitations. Our sample consisted primarily of White, non-Hispanic, female athletes. All participants with concussion were recruited from a single institution providing specialty concussion care and may not be reflective of the general adolescent population after concussion. Additionally, participants with a concussion were assessed a mean of 9.8 days after their injury, so it is unclear if the PSQI can differentiate between those with and without a concussion in the more acute (<7 days) or chronic (>1 month) phases of the injury.

Conclusion

After concussion, adolescents reported significantly worse sleep compared with those without a concussion. Importantly, a PSQI cut point of 4 to determine poor sleep quality, compared with previously reported cut points of 5 and 8 among different populations, provides excellent accuracy for distinguishing between concussion and control groups. Using a cut point that is reflective of a specific population and age range (ie, adolescents with a concussion), researchers and clinicians can ensure that the questionnaire is appropriate for capturing sleep patterns and behaviors reflective of the patient they are treating.