Abstract
Objective:
Poor sleep patterns are common in undergraduates and may turn them prone to mood disorders, substance abuse and impaired academic performance. The aim of this study was to assess sleep disturbances among medical students, and whether associations with academic performance, depressive symptoms or substance use were present.
Methods:
Cross-sectional study in which 544 medical students of the Pontificia Universidad Javeriana in Bogota, Colombia were included. Using a computer-based survey, self-reported variables were assessed, including demographics, Pittsburgh Sleep Quality Index (PSQI), Epworth Sleepiness Scale (ESS), Berlin Questionnaire, Diagnostic and Statistical Manual of Mental Disorders (DSM) - IV depression criteria, Grade Point Average (GPA), and substance use. Associations between these variables were obtained.
Results:
Fifty-four percent of students were women. Poor sleep quality (PSQI>5) and daytime sleepiness (ESS>10) were found in 65% of the population. A higher GPA was more frequent in students with good sleep quality (OR= 2.6 [1.5-4.5]), lack of daytime sleepiness (OR= 2 [1.3-3.1]) and low risk of Obstructive Sleep Apnea Syndrome (OSAS) (OR= 3.1 [1.6-5.9]). DSM-IV depression criteria were fulfilled by 26% of the students and were associated with poor sleep patterns. Energy drinks use was associated with poor sleep quality.
Discussion:
Poor sleep quality, daytime sleepiness and depressive symptoms are frequent among medical students and are associated with lower academic performance. The identification of students at risk and the implementation of targeted interventions are warranted. Fostering adequate sleep habits and training on sleep medicine may partly counteract these issues.
Keywords: Students, Medical; Sleep; Sleep Disorders; Mood Disorders; Substance-Related Disorders; Latin America
INTRODUCTION
The pivotal role of sleep seems forsaken. An adult should sleep at least 7 hours per night for a good health, although there are short and long-sleepers1. Sleep disturbances, such as insomnia,2 obstructive sleep apnea (OSA)3, and shift work disturbances4 decrease sleep quality and may generate sleep deprivation, which is ubiquitous among health-care providers. It may be classified as acute continuous or chronic partial (i.e., insufficient recovery sleep)5. The former presents dose-dependent deterioration in motor skills, mood and cognitive abilities, equivalent to a blood alcohol concentration of 0.004% rise per hour5. The latter generates depressed mood, attention and cognitive impairments, which may be seen after less than 6 hours of sleep per night in less than 2 weeks; it is equivalent to 24 hours of sleep deprivation. Subjects perceive they become adapted to sleep loss, but impairment tends to increase5. Given the importance of good sleep in synapse consolidation in the hippocampus and neocortex6,7 memory is one of the most affected features in human health, which may diminish academic performance.
Medical students are exposed to poor sleep, due to high work and study loads, overnight shifts, and emotionally challenging duties8. Recent evidence suggests they tend to sleep less than 6 hours per night9,10. These factors may reduce academic performance, trigger mental health issues and have a negative impact on the performance of health-care systems8. Chang et al reported a prevalence of 66% of depressive symptoms and more than 50% of burnout syndrome in medical students11. Dyrbye et al.12 showed this syndrome undermines professional attitudes towards patients, facilitates dishonest behaviors, and diminishes altruistic values regarding physician's responsibility to society. Its relationship with sleep deprivation was shown by Söderström et al.13
Given the rising importance of sleep disturbances in public health, this subject has been approached in medical students all over the world with staggering results8. Sleep problems, such as insomnia and difficulty initiating and maintaining sleep, are frequent in college students14. Emotional distress has been reported to mediate these alterations15. For instance, anxiety is associated with a higher frequency of nightmares, and those students reporting a higher frequency tend to rate sleep quality as poorer16 This combination may lead to an increase risk of psychological and learning problems17,18 which may be buffered through psychological interventions that improve sleep19.
Further, reports in medical students in Saudi Arabia20 and Germany21 showed a prevalence of 36% of daytime sleepiness using the Epworth Sleepiness Scale (ESS), and 59% of poor sleep quality using the Pittsburgh Sleep Quality Index (PSQI), respectively. Similar results were described in Estonian22 and Iranian studies23. However, reports from Latin America seem worse. Poor sleep quality may reach 88%24 and daytime sleepiness may be as high as 60%25,26.
Equally important, differences in circadian rhythmic expression between individuals promote circadian preferences, known as morningness-eveningness, which further influence students sleep patterns27,28. Evening-type students appear to present a higher risk for depressive mood, suicidal ideations and substance abuse29 although its impact in academic performance is conflicting: a study in the United States reported worse performance in evening-type students27, but a Colombian study did not find differences9. Thus, sleep disturbances impact academic performance and mental health, drawing attention to the possible benefits of screening and targeted interventions for medical students with sleep disturbances, as these issues may turn our population prone to substance abuse and mental illnesses8. This information will be pivotal for curricula modifications worldwide.
The aim of our study was to determine sleep disturbances using validated questionnaires in a population of medical students in Bogota, Colombia, and its relationship with demographic characteristics, academic performance, depressive symptoms and substance use.
MATERIALS AND METHODS
Study design
Cross-sectional prevalence study in medical students of the School of Medicine at the Pontificia Universidad Javeriana, in Bogota, Colombia, during 2013. We included students with complete academic load, with academic duties within the city (i.e., preclinical lectures or clinical rotations), and who were at University campus when the survey was applied. No sample size was calculated, as the study was developed as a census. We excluded medical students on their last year of education (i.e., eleventh and twelfth semesters), on the basis that the tasks and schedules during this internship year hold more resemblance to those of the practicing physician than to the normal work dynamics of the medical student. All participants (99.9%) were over 16 years old and were considered adults, as they were in a university program. Moreover, sleep disorder questionnaires have been found to be reliable among university students30.
Applied instruments
Sleep quality was determined using PSQI Colombian-validated version31 which discriminates between good and bad sleepers. A score of more than 5 define bad sleepers.
Daytime sleepiness was assessed using ESS Colombian-validated version32, which determine the chance of falling asleep in 8 different circumstances. Although it has been considered a useful tool for OSA screening, in a previous study, we did not find a correlation with OSA severity33. Nevertheless, it is regarded as a significant symptom, which may point out to the existence of an altered sleep pattern.
The Berlin questionnaire (BQ) Colombian-validated version was used34 for the assessment of OSA risk in our population. The survey measures 3 categories: snoring, daytime sleepiness, and obesity and high blood pressure. In our scenario, the last domain was self-reported: students were asked to take their blood pressure and weight one day before the survey as an academic exercise, but no additional information was given. Positive results in two domains detect a high-risk individual. It has been shown useful in patients 15 years old or older35.
Regarding mood disorders, the Diagnostic and Statistical Manual of Mental Disorders (DSM) -IV diagnostic criteria were used. These criteria allow a major depression diagnosis when five out of nine criteria are present in the patient, provided depressed mood criterion is present36. Although DSM-V is available since May 2013, we applied its previous version as multiple controversies, particularly regarding false-positive diagnosis with DSM-V were suggested in the literature37. In addition, the study protocol was written and approved before this version was endorsed.
We selected questions concerning the frequency of use of the most often used substances during the last month, according to the “National Survey on Psychoactive Substance Use in school-age population”, developed in 201138, namely: energy drinks, alcohol, tobacco, marihuana, cocaine, stimulants (i.e., methylphenidate - Ritalin, methamphetamine - “Meth”) and tranquilizers (i.e., diazepam, lorazepam, alprazolam, clonazepam, etc). Given the economic-dependence of our target population, we considered that the economic capacity to obtain substances was similar between school-age population and medical students. Regarding alcohol consumption, individual episodes of consumption greater than one drink in the last month and total episodes of binge drinking (i.e., inebriation) in the same period were included.
Colombian grading system ranges from 1.0 to 5.0, the latter being the highest possible. Self-reported cumulative and actual grade point average (GPA) were obtained. We used a quantitative scale with five possible categories: <3.0, 3.0-3.4, 3.5-3.9, 4-4.4, >4.5.
The described tools were condensed in a questionnaire, which was applied to each semester. Students were free to either participate or not in the study. Two researchers (PH and AR) were lecturers during undergraduate studies, but had no responsibilities to grade them. A computer-based survey method was used (TurningPoint v8.0 for Windows). Each student was given a wireless response clicker to answer the questions projected on a screen. As self-reported answers were to be obtained, no personal data were associated to each clicker and, thus, the researchers were unable to identify each student's answer.
A time frame of 10 seconds was given for each question, which was considered sufficient after a pilot application was performed among the authors. Each participant had only one chance to answer the survey. To reassure that individuals were not sampled more than once, the survey was applied prior to the beginning of a lecture that was exclusive for each semester. Participants could not see their fellow students' answers.
Data analysis
The software yielded spreadsheets, which were analyzed using Graphpad Prism 7 for Mac. As variables were mainly categorical, absolute and relative frequencies were calculated. Chi square and Fisher's exact tests were used, as required. P values < 0.05 were considered significant. As multiple comparisons were performed, Bonferroni's correction was applied to determine significant findings.
Ethical considerations
The study protocol was reviewed and approved by the Research and Ethics Committee of the Pontificia Universidad Javeriana (FM-CIE-6086-13). The study was classified as a minimal risk research and conducted in agreement with Helsinki declaration, and the Resolution number 008430 of 1993 issued by the Ministry of Health of the Republic of Colombia.
RESULTS
We obtained data from 544 medical students. Table 1 shows the demographic characteristics of the surveyed medical student population. Due to the answering method (anonymous and elective), there were variations in the sub-group populations, as some of the participants were unwilling to answer some of the questions. Based on those missing answers, the number of participants changed from question to question. Fifty percent of the surveyed were between the ages of 19 and 21, and 54% were women. Sixty percent were on the preclinical study phase (i.e., first to fifth semester).
Table 1.
General characteristics of total surveyed population and by questionnaire.
| Variable [n (%)] | Total (n=544) | PQSI (n=300) * | ESS (n=471) * | BQ (n=544) * | p-value |
|---|---|---|---|---|---|
| Age | |||||
| <16 | 5 (1) | 4 (1) | 5 (1) | 5 (1) | 0.96 |
| 16-18 | 138 (25) | 80 (27) | 121 (26) | 138 (25) | |
| 19-21 | 274 (50) | 161 (54) | 244 (52) | 274 (50) | |
| 22-24 | 103 (19) | 48 (16) | 85 (18) | 103 (19) | |
| >25 | 8 (2) | 2 (1) | 8 (2) | 8 (2) | |
| ND | 16 (3) | 5 (2) | 8 (2) | 16 (3) | |
| Gender | |||||
| Male | 197 (36) | 110 (37) | 173 (37) | 197 (36) | 0.88 |
| Female | 296 (54) | 170 (57) | 259 (55) | 296 (54) | |
| ND | 51 (9) | 20 (7) | 39 (8) | 51 (9) | |
| Currently working | |||||
| Yes | 15 (3) | 10 (3) | 10 (2) | 15 (3) | 0.25 |
| No | 513 (94) | 287 (96) | 455 (97) | 513 (94) | |
| ND | 16 (3) | 3 (1) | 6 (1) | 16 (3) | |
| Study stage | |||||
| Preclinical | 325 (60) | 185 (62) | 281 (60) | 325 (60) | 0.94 |
| Clinical | 219 (40) | 115 (38) | 190 (40) | 219 (40) | |
| Fulfilled DSM-IV Depression Criteria (n=529) * | |||||
| Yes | 139 (26) | - | - | - | - |
| No | 390 (74) | - | - | - | |
| GPA (n=450) * | |||||
| <4 | 227 (50) | - | - | - | - |
| >4 | 223 (50) | - | - | - |
Available complete questionnaires/answers.
Abbreviations: PSQI Pittsburgh Sleep Quality Index; ESS Epworth Sleepiness Scale; BQ Berlin Questionnaire; ND No data.
Table 2 shows sleep patterns of the surveyed population. We collected 300 analyzable PQSI scores, where we found a poor sleep quality prevalence of 65.7%.
Table 2.
General characteristics by questionnaire score.
| Variable [ n (%)] | PSQI (n=300)* | ESS (n=471)* | BQ (n=544)* | ||||||
|---|---|---|---|---|---|---|---|---|---|
| <5 | >5 | p-value | <10 | >10 | p-value | Low risk | High risk | p-value | |
| Age | |||||||||
| <16 | 3 (75) | 1 (25) | 0.02 | 2 (40) | 3 (60) | 0.69 | 5 (100) | 0 (0) | 0.38 |
| 16-18 | 38 (47) | 42 (53) | 46 (38) | 75 (62) | 125 (91) | 13 (9) | |||
| 19-21 | 48 (30) | 113 (70) | 86 (35) | 158 (65) | 251 (92) | 23 (8) | |||
| 22-24 | 13 (27) | 35 (73) | 24 (28) | 61 (72) | 87 (85) | 16 (15) | |||
| >25 | - | 2 (100) | 3 (37) | 5 (63) | 7 (88) | 1 (12) | |||
| ND | 1 (20) | 4 (80) | 4 (50) | 4 (50) | 15 (94) | 1 (6) | |||
| Gender | |||||||||
| Male | 42 (38) | 68 (62) | 0.25 | 73 (42) | 100 (58) | 0.036** | 167 (85) | 30 (15)# | 0.028** |
| Female | 52 (31) | 118 (69) | 75 (29) | 184(71) | 275 (93) | 21 (7)# | |||
| ND | 9 (45) | 11 (55) | 17 (44) | 22 (56) | 48 (94) | 3 (6) | |||
| Currently working | |||||||||
| Yes | 2 (20) | 8 (80) | 0.62 | 4 (40) | 6 (60) | 0.94 | 14 (93) | 1 (7) | 0.80 |
| No | 100 (35) | 187 (65) | 159 (35) | 296 (65) | 461 (90) | 52 (10) | |||
| ND | 1 (33) | 2 (67) | 2 (33) | 4 (67) | 15 (94) | 1 (6) | |||
| Study stage | |||||||||
| Preclinical | 68 (37) | 117 (63) | 0.26 | 107 (38) | 174 (62) | 0.09 | 297 (91) | 28 (9) | 0.21 |
| Clinical | 35 (30) | 80 (70) | 58 (30) | 132 (70) | 193 (88) | 26 (12) | |||
Available complete questionnaires.
Significant after Bonferroni's correction.
After a significant p-value by Chi square test was obtained, Fisher's exact test was performed to determine which subgroups differed significantly.
**Values were calculated on available questionnaires for both variables.
Abbreviations: PSQI Pittsburgh Sleep Quality Index; ESS Epworth Sleepiness Scale; BQ Berlin Questionnaire; ND No data
A higher frequency of poor sleep quality was observed in 19-21 and 22-24 groups, compared with 16-18 group. No difference was observed regarding gender. Poor sleep quality was above 50% in every semester and no difference was observed between them (p=0.4685) (Figure 1). Prevalence of daytime sleepiness, measured by ESS (n=471), was 65% using a 10 points cut-off. Women showed a significant higher frequency of daytime sleepiness (p=0.036). No difference was observed regarding age group. The proportion of students with high risk of OSA, according to the BQ (n=544), was 9.9%, which was significantly more frequent in men. No difference was observed regarding age group. Neither work status nor study stage showed significant differences in any scale.
Figure 1.
Distribution of PSQI by semester. A score of more than 5 in Pittsburgh Sleep Quality Index (PSQI) define bad sleepers.
A significantly higher frequency of daytime sleepiness was observed in students with poor sleep quality (p=0.0002). However, no difference was obtained concerning high risk of OSA (p=0.2578).
Regarding academic performance, 50% of the population reported an overall GPA above 4.0 (Table 1). However, as shown in Figure 2 and Table 3, poor overall academic performance was associated with poor sleep patterns. GPA >4 were more frequent in students with PSQI <5 (67% vs. 43%, p=0.001; OR=2.6 [1.5-4.5]), ESS <10 (62% vs 45%, p=0.002; OR=2 [1.3-3.1]), and low risk BQ (52% vs. 26%, p=0.002; OR=3.1 [1.6-5.9]).
Figure 2.
Depressive symptoms and GPA>4 distribution by PSQI. Solid line denotes depressive symptoms. Dotted line denotes GPA >4. Abbreviations: GPA Grade Point Average, PSQI Pittsburgh Sleep Quality Index.
Table 3.
Depressive symptoms and GPA score by questionnaire score.
| Variable | PSQI (n=296)* | Epworth (n=464)* | Berlin (n=529) | ||||||
|---|---|---|---|---|---|---|---|---|---|
| n (%) | <5 | >5 | p-value | <10 | >10 | p-value | Low risk | High risk | p-value |
| Fulfilled DSM-IV Depression Criteria | |||||||||
| Yes | 4 (4) | 117 (40) | <0.0001** | 26 (16) | 98 (32) | 0.001** | 114 (24) | 25 (47) | 0.002** |
| No | 97 (96) | 78 (60) | 135 (84) | 205 (68) | 362 (76) | 28 (53) | |||
| GPA | |||||||||
| <4 | 29 (33) | 94 (57) | 0.001** | 52 (38) | 145 (55) | 0.002** | 193 (47) | 34 (74) | 0.002** |
| >4 | 59 (67) | 72 (43) | 86 (62) | 119 (45) | 211 (52) | 12 (26) | |||
| Total Available (%)+ | 254 (86) | 402 (87) | 450 (85) | ||||||
Available complete questionnaires.
Significant after Bonferroni's correction.
As some individuals did not answer all the questions, only complete questionnaires with available GPA were included.
Regarding DSM-IV Depression Criteria all data were available.
Abbreviations: GPA Grade point average; PSQI Pittsburgh Sleep Quality Index; ESS Epworth Sleepiness Scale; BQ Berlin Questionnaire; ND No data.
As shown in Table 1, we found a high frequency of students fulfilling DSM-IV criteria for major depression (26%), with a particularly dramatic share in the poor sleep quality population (PSQI >5) compared with good sleepers (40% vs. 4%, p<0.0001) (Figure 2, Table 3). Further, this finding was reproduced in students with daytime sleepiness (ESS>10 32% vs. 16%, p=0.001), and high risk of OSA (47% vs. 24%, p=0.002).
As disclosed in Figure 3, we conducted an analysis of the prevalence of self-reported psychoactive substance use in the last month. Overall prevalence of alcohol use in at least one occasion was 79%; thirty-two percent of students reported one or more episodes of binge drinking. Energy drink usage was reported by 32% of the surveyed, whereas tobacco use was disclosed by 19% of the population (Table 4).
Figure 3.
Substance exposure at least once in the last month. Stimulants include methylphenidate - Ritalin, methamphetamine - “Meth”. Tranquilizers include diazepam, lorazepam, alprazolam, clonazepam, etc.
Table 4.
General characteristics by substance*.
| Variable[n (%)] | Tobacco (n=530) | Alcohol (n=509) | Binge drink(n=524) | Energy drinks(n=516) | Marihuana (n=526) | Cocaine(n=524) | Stimulants+ (n=528) | Tranquilizers+ (n=532) |
|---|---|---|---|---|---|---|---|---|
| General popu-lation | 103 (19) | 402 (79) | 197 (38) | 166 (32) | 52 (10) | 4 (0.8) | 12 (2) | 11 (2) |
| Age | ||||||||
| <16 | - | 4 (1) | 2 (40) | 3 (2) | 1 (2) | - | - | - |
| 16-18 | 19 (18) | 90 (22) | 50 (37) | 35 (21) | 11 (21) | 2 (50) | 1 (8) | 3 (27) |
| 19-21 | 52 (50) | 208 (52) | 100 (37) | 78 (47) | 32 (62) | 2 (50) | 6 (50) | 3 (27) |
| 22-24 | 27 (26) | 84 (21) | 40 (42) | 46 (28) | 7 (14) | - | 5 (42) | 5 (45) |
| 25 + | 3 (3) | 6 (2) | 2 (25) | 2 (1) | - | - | - | - |
| ND | 2 (2) | 10 (3) | 3 (25) | 2 (1) | 1 (2) | - | - | - |
| Gender | ||||||||
| Male | 51 (50) | 162 (40) | 90 (47) | 71 (43) | 29 (56) | 2 (50) | 6 (50) | 5 (45) |
| Female | 45 (44) | 203 (51) | 92 (32) | 77 (46) | 17 (33) | 2 (50) | 5 (42) | 5 (45) |
| ND | 7 (7) | 37 (9) | 15 (34) | 18 (11) | 6 (12) | - | 1 (8) | 1 (9) |
| Study stage | ||||||||
| Preclinical | 56 (54) | 232 (58) | 118 (38) | 88 (53) | 34 (65) | 3 (75) | 6 (50) | 6 (54) |
| Clinical | 47 (46) | 170 (42) | 79 (37) | 78 (47) | 18 (35) | 1 (25) | 6 (50) | 5 (45) |
Every exposure is measured as at least one episode in the last month. Only available responses were included, which explains the difference in sample sizes for each substance.
Stimulants includes Methylphenidate - Ritalin, Methamphetamine - "Meth"; Tranquilizers includes Diazepam, Lorazepam, Alprazolam, Clonazepam, etc.
Abbreviations: ND No data.
Students with PSQI >5 reported a more frequent consumption of energy drinks (37% vs. 17%, p=0.002). Those with high risk BQ exposed a high usage of energy drinks (47% vs. 30%) and marihuana (21% vs. 9%), although these findings were not significant after correction. Overall prevalence of both alcohol or tobacco use, were not different between groups (Table 5).
Table 5.
Substance use at least once in the last month by questionnaire score*.
| Variable | p-value | p-value | |||||||
|---|---|---|---|---|---|---|---|---|---|
| <5 | >5 | <10 | >10 | Low | High | ||||
| Tobacco | 14 (14) | 42 (21) | 0.1187 | 26 (16) | 60 (20) | 0.3807 | 88 (18) | 15 (28) | 0.1052 |
| Alcohol | 77 (79) | 145 (77) | 0.7639 | 123 (78) | 226 (78) | 0.9055 | 357 (78) | 45 (86) | 0.2079 |
| Binge drink | 30 (29) | 79 (40) | 0.0757 | 50 (31) | 121 (40) | 0.0682 | 176 (37) | 21 (40) | 0.7662 |
| Energy drinks | 18 (17) | 72 (37) | 0.002** | 42 (26) | 101 (34) | 0.0730 | 142 (30) | 24 (47) | 0.0260 |
| Marihuana | 6 (6) | 21 (11) | 0.2046 | 16 (10) | 28 (9) | 0.8691 | 41 (9) | 11 (21) | 0.0112 |
| Cocaine | - | 2 (1) | 0.5461 | - | 2 (1) | 0.5432 | 4 (1) | - | 0.9999 |
| Stimulants+ | - | 6 (3) | 0.964 | 3 (2) | 8 (3) | 0.7542 | 10 (2) | 2 (4) | 0.3344 |
| Tranquilizers+ | - | 4 (2) | 0.3022 | 3 (2) | 6 (2) | 0.9999 | 11 (2) | - | 0.6136 |
Values were calculated on available questionnaires for both variables.
Significant after Bonferroni's correction
Stimulants includes Methylphenidate - Ritalin, Methamphetamine - "Meth"; Tranquilizers includes Diazepam, Lorazepam, Alprazolam, Clonazepam, etc.
Abbreviations: PSQI Pittsburgh Sleep Quality Index; ESS Epworth Sleepiness Scale; BQ Berlin Questionnaire; ND No data.
DISCUSSION
Poor sleep quality is present in up to two-thirds of medical students and is more frequent in those with daytime sleepiness. Furthermore, it appears to be associated with reduced academic performance, depressive symptoms and substance use.
The proportion of women in our study is in line with the gender distribution in Bogota (56%) according to data (2001-2014) from the Colombian Ministry of Education39. A significant proportion of our population presents poor sleep quality. This finding has been reported worldwide8. In our region, Cardoso et al. reported a prevalence close to 14%, in contrast to studies in Peru, which found a prevalence ranging from 60%40 to 90%41. A recent study in Brazil reported altered PSQI scores in up to 70% of medical students42. In Colombia, a prevalence as high as 88% has been reported24. Further, our results support the evidence of poor sleep quality in medical students in public and private schools in Colombia. Escobar-Córdoba et al.25 and Machado-Duque et al.43 reported a prevalence of up to 80% of poor sleep quality in public universities in Colombia. Similar results support that medical students' populations in Latin America are akin to each other regarding their demanding curricula. Differences in socioeconomic characteristics between public and private universities seem to be unrelated to poor sleep quality. A relationship between chronic illnesses (i.e., asthma, diabetes, hypertension) and poor sleep quality has been described in medical students44, which may be a subject for future research.
Two-thirds of our medical student population present daytime sleepiness that warrants medical intervention (i.e., ESS >10). Previous reports have shown similar results worldwide8, although our findings seem slightly higher. A study in Saudi Arabia showed a prevalence of 36%20, similar to the prevalence in Argentina45 and Peru40. In contrast, higher levels have been reported in Brazil (39% to 50%)46,47 and Colombia (up to 60%)24-26,43,48. Daytime sleepiness tends to get worse with each semester, which may induce students to decreased attention and performance, and may be associated with diminished academic results46,49.
One in ten students may present OSA, which reduces quality of sleep and may pose a potential health threat. Data regarding OSA risk in medical students and physicians is scarce; BQ has not been previously applied to these populations. In contrast to our results, Lucero et al.45 showed, using SLEEP-50 score, a prevalence of 0.32% in Argentinian medical students. Although a large portion of general population might be at high risk of OSA50, a Nigerian study found that knowledge on OSA in medical students warrants improvement, and suggested to formally incorporate the evaluation of sleep disorders in undergraduate curricula51. This phenomenon has been reported in primary care physicians in Latin America52. Our findings support the need for further research on OSA risk in medical students and health-care workers, which may impair professional and academic performance, and pose a potential risk for health50.
Poor sleep quality is associated with daytime sleepiness and OSA. Cardoso47 previously described a significant relation between daytime sleepiness and poor sleep quality in medical students. In addition, a positive trend was found in Nigeria44.
Poor sleep quality and daytime sleepiness prevalence is similar in both preclinical and clinical years in our population, which agrees with previous reports40,48. Our findings support the importance of identification of the population at-risk of sleep disturbances in early-years of medicine school and the implementation of targeted interventions.
Our results show that 1 in 4 students presents depressive symptoms, which are associated with poor sleep quality, daytime sleepiness and high risk of OSA. The relationship between mood and anxiety disorders and sleep disturbances is clear8. A study at Johns Hopkins School of Medicine found that insomnia posed a greater risk for clinical depression, which may last up to 30 years53 Moreover, Hidalgo & Caumo54 reported a higher risk for developing minor psychiatric disorders in Brazilian medical students with sleepiness, insomnia or less than 7 hours per night of sleep, which was further reproduced using a different questionnaire55. Loayza et al.55 suggested insomnia screening “might be a good tool for the identification of medical students who could benefit from psychiatric assessment and preventive measures”. Some researchers have even related subjective poor sleep quality with an increased risk of suicide56. On the other hand, daytime sleepiness was associated with depression and anxiety in an Estonian cohort of medical students57 and was found to be related to psychological distress in Malaysia58 However, it is known that sleep disturbances may be a comorbidity, a symptom or even a cause of psychiatric disorders59. The role of sleep quality in medical students should be addressed with curricula modifications processes to prevent potential mental health issues in future health-care workers.
Some researchers described a significant relation between daytime sleepiness and academic performance20,45 as we did using a GPA cutoff point of 4.0. However, evidence is conflicting24,25,43,60. Our results suggest that students with good sleep quality, low daytime sleepiness or low risk of OSA might present a better academic performance. This was described previously by Veldi et al.22, who ascertained poor sleep quality correlated with lower academic achievements, although non-validated questionnaires were used.
Due to chronic sleep deprivation, medical students may be prone to psychotropic substance use. Substance use is frequent among medical students and is significantly associated to living alone, self-reported study difficulty and being a clinical student, whereas self-reported good mental health might be a protective factor61 Young people may present a higher frequency of alcohol or stimulant agents consumption in order to regulate sleep/wake cycle8. This behavior favors the use of stimulant to halt daytime sleepiness, and subsequently promotes the use of sedative agents to revert its effects, phenomenon known as “stimulant-sedation loop”62 For instance, a study in Peru found that 20% of medical students had used sleep medications during the last month60. Further, frequent use of alcohol was significantly associated to poor sleep quality in Nigeria44. Our results highlight a frequent use of alcohol, energy drinks and tobacco. Noteworthy, marihuana use was frequent in students with high risk of OSA. Although energy drinks are not naturally a category of substance abuse as other drugs, they were the most frequently associated with poor sleep patterns; no associations were found with alcohol or tobacco consumption.
Despite these alarming findings, our study has some limitations. First, we describe a cross-sectional design that may depict sub-acute disturbances, despite the time frame considered in most of the applied questionnaires (i.e., the last month). Follow-up studies are warranted to determine the impact of bad sleep habits on medical students. Second, although our study was planned as a census, to survey every student in each semester was a challenge. This phenomenon was reflected in the variability of sample within each semester, which may somehow bias results. Nonetheless, the fact that no differences were detected between semesters supports the validity of the findings. Third, some students did not answer all the questions, which yielded a reduction in the number of complete questionnaires available for analysis. Although this probably may bias the results, we show the usefulness of a computer-based method using wireless response clickers for large samples self-reported assessment. This may facilitate data collection with an adequate reliability. Fourth, we acknowledge the self-application of DSM-IV may limit the external validity of the results. Alternative questionnaires would be more useful and comparable, such as Beck's Depression Inventory-II, which was adapted to DSM-IV and validated in Spanish in college students63,64. Nonetheless, as acknowledged by the current DSM-5, depressive disorders are often accompanied by significant levels of anxiety. Thus, instruments that consider this feature may be more adequate for future research, such as the Patient Health Questionnaire Anxiety and Depression Scale65. In any case, we consider our data highlights the high frequency of depression among this population. Fifth, we did not capture relevant information such as that obtained through sleep logs and actigraphy, and neither applied a chronotype questionnaire. For instance, the latter could shed light into whether students with depressive mood were more frequently evening-type, a feature that should be explored. Further studies to screen and determine risk factors for mood disorders, that may allow targeted interventions, are warranted.
Our results support the fact that sleep disturbances are common among medical students. A higher prevalence, when compared with non-medical students and the general population, has been suggested8. It is essential to consider the effects of poor sleep in medical trainees on their physical and mental health, in order to improve their academic performance and quality of life8. Furthermore, the quality of health care delivery will improve and society overall could benefit.
CONCLUSION
Poor sleep quality and daytime sleepiness are common in our medical student population, and are associated with a diminished academic performance, depressive symptoms and substance abuse. Students with high risk of OSA are more common than expected. Noteworthy, a significant portion of students present mood disorders, which warrant medical intervention. As previously suggested8, more research and initiatives towards the improvement of sleep quality and knowledge, the identification of students at risk, and the development of targeted programs are urgently needed. Strategies to guarantee healthcare professionals safety and performance in developing countries are pivotal to support the quality of healthcare delivery.
Acknowledgements
We thank the students who participated in this study and the School of Medicine at the Pontificia Universidad Javeriana for their trust, collaboration and support.
REFERENCES
- 1.Consensus Conference Panel. Watson NF, Badr MS, Belenky G, Bliwise DL, Buxton OM, Buysse D, et al. Recommended Amount of Sleep for a Healthy Adult: A Joint Consensus Statement of the American Academy of Sleep Medicine and Sleep Research Society. J Clin Sleep Med. 2015;11(6):591–592. doi: 10.5664/jcsm.4758. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Ohayon MM, Smirne S. Prevalence and consequences of insomnia disorders in the general population of Italy. Sleep Med. 2002;3(2):115–120. doi: 10.1016/s1389-9457(01)00158-7. [DOI] [PubMed] [Google Scholar]
- 3.Lee W, Nagubadi S, Kryger MH, Mokhlesi B. Epidemiology of Obstructive Sleep Apnea: a Population-based Perspective. Expert Rev Respir Med. 2008;2(3):349–364. doi: 10.1586/17476348.2.3.349. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Ohayon MM. Epidemiology of insomnia: what we know and what we still need to learn. Sleep Med Rev. 2002;6(2):97–111. doi: 10.1053/smrv.2002.0186. [DOI] [PubMed] [Google Scholar]
- 5.Olson EJ, Drage LA, Auger RR. Sleep deprivation, physician performance, and patient safety. Chest. 2009;136(5):1389–1396. doi: 10.1378/chest.08-1952. [DOI] [PubMed] [Google Scholar]
- 6.Diekelmann S, Born J. The memory function of sleep. Nat Rev Neurosci. 2010;11(2):114–126. doi: 10.1038/nrn2762. [DOI] [PubMed] [Google Scholar]
- 7.Walker MP. The role of sleep in cognition and emotion. Ann N Y Acad Sci. 2009;1156:168–197. doi: 10.1111/j.1749-6632.2009.04416.x. [DOI] [PubMed] [Google Scholar]
- 8.Azad MC, Fraser K, Rumana N, Abdullah AF, Shahana N, Hanly PJ, et al. Sleep disturbances among medical students: a global perspective. J Clin Sleep Med. 2015;11(1):69–74. doi: 10.5664/jcsm.4370. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Pérez-Olmos I, Talero-Gutiérrez C, González-Reyes R, Moreno CB. Ritmos circadianos de sueño y rendimiento académico en estudiantes de medicina. Rev Cienc Salud. 2006;4(2):147–157. [Google Scholar]
- 10.Sitticharoon C, Srisuma S, Kanavitoon S, Summachiwakij S. Exploratory study of factors related to educational scores of first preclinical year medical students. Adv Physiol Educ. 2014;38(1):25–33. doi: 10.1152/advan.00162.2012. [DOI] [PubMed] [Google Scholar]
- 11.Chang E, Eddins-Folensbee F, Coverdale J. Survey of the prevalence of burnout, stress, depression, and the use of supports by medical students at one school. Acad Psychiatry. 2012;36(3):177–182. doi: 10.1176/appi.ap.11040079. [DOI] [PubMed] [Google Scholar]
- 12.Dyrbye LN, Massie Jr FS, Eacker A, Harper W, Power D, Durning SJ, et al. Relationship between burnout and professional conduct and attitudes among US medical students. JAMA. 2010;304(11):1173–1180. doi: 10.1001/jama.2010.1318. [DOI] [PubMed] [Google Scholar]
- 13.Söderström M, Jeding K, Ekstedt M, Perski A, Akerstedt T. Insufficient sleep predicts clinical burnout. J Occup Health Psychol. 2012;17(2):175–183. doi: 10.1037/a0027518. [DOI] [PubMed] [Google Scholar]
- 14.Schlarb AA, Kulessa D, Gulewitsch MD. Sleep characteristics, sleep problems, and associations of self-efficacy among German university students. Nat Sci Sleep. 2012;4:1–7. doi: 10.2147/NSS.S27971. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Hayley AC, Downey LA, Stough C, Sivertsen B, Knapstad M, Øverland S. Social and emotional loneliness and self-reported difficulty initiating and maintaining sleep (DIMS) in a sample of Norwegian university students. Scand J Psychol. 2017;58(1):91–99. doi: 10.1111/sjop.12343. [DOI] [PubMed] [Google Scholar]
- 16.Schredl M. Effects of state and trait factors on nightmare frequency. Eur Arch Psychiatry Clin Neurosci. 2003;253(5):241–247. doi: 10.1007/s00406-003-0438-1. [DOI] [PubMed] [Google Scholar]
- 17.Engle-Friedman M, Riela S, Golan R, Ventuneac AM, Davis CM, Jefferson AD, et al. The effect of sleep loss on next day effort. J Sleep Res. 2003;12(2):113–124. doi: 10.1046/j.1365-2869.2003.00351.x. [DOI] [PubMed] [Google Scholar]
- 18.Gaultney JF. The prevalence of sleep disorders in college students: impact on academic performance. J Am Coll Health. 2010;59(2):91–97. doi: 10.1080/07448481.2010.483708. [DOI] [PubMed] [Google Scholar]
- 19.Friedrich A, Schlarb AA. Let's talk about sleep: a systematic review of psychological interventions to improve sleep in college students. J Sleep Res. 2018;27(1):4–22. doi: 10.1111/jsr.12568. [DOI] [PubMed] [Google Scholar]
- 20.Abdulghani HM, Alrowais NA, Bin-Saad NS, Al-Subaie NM, Haji AMA, Alhaqwi AI. Sleep disorder among medical students: relationship to their academic performance. Med Teach. 2012;34(Suppl 1):S37–S41. doi: 10.3109/0142159X.2012.656749. [DOI] [PubMed] [Google Scholar]
- 21.Ahrberg K, Dresler M, Niedermaier S, Steiger A, Genzel L. The interaction between sleep quality and academic performance. J Psychiatr Res. 2012;46(12):1618–1622. doi: 10.1016/j.jpsychires.2012.09.008. [DOI] [PubMed] [Google Scholar]
- 22.Veldi M, Aluoja A, Vasar V. Sleep quality and more common sleep-related problems in medical students. Sleep Med. 2005;6(3):269–275. doi: 10.1016/j.sleep.2004.12.003. [DOI] [PubMed] [Google Scholar]
- 23.Nojomi M, Ghalhe Bandi MF, Kaffashi S. Sleep pattern in medical students and residents. Arch Iran Med. 2009;12(6):542–549. [PubMed] [Google Scholar]
- 24.Monterrosa Castro A, Ulloque Caamaño L, Carriazo Julio S. Calidad del dormir, insomnio y rendimiento académico en estudiantes de medicina. Duazary. 2014;11(2):85–97. [Google Scholar]
- 25.Escobar-Córdoba F, Benavides-Gélvez RE, Montenegro-Duarte HG, Eslava-Schmalbach JH. Somnolencia diurna excesiva en estudiantes de noveno semestre de la Universidad Nacional de Colombia. Rev Fac Med. 2011;59(3):191–200. [Google Scholar]
- 26.Escobar-Córdoba F, Cortés-Rueda MF, Canal-Ortiz JS, Colmenares-Becerra LA, Becerra-Ramírez HA, Caro-Rodríguez CC. Somnolencia diurna excesiva en estudiantes de tercer semestre de medicina de la Universidad Nacional de Colombia. Rev Fac Med. 2008;56(3):235–244. [Google Scholar]
- 27.Taylor DJ, Clay KC, Bramoweth AD, Sethi K, Roane BM. Circadian phase preference in college students: relationships with psychological functioning and academics. Chronobiol Int. 2011;28(6):541–547. doi: 10.3109/07420528.2011.580870. [DOI] [PubMed] [Google Scholar]
- 28.Adan A, Archer SN, Hidalgo MP, Di Milia L, Natale V, Randler C. Circadian typology: a comprehensive review. Chronobiol Int. 2012;29(9):1153–1175. doi: 10.3109/07420528.2012.719971. [DOI] [PubMed] [Google Scholar]
- 29.Bakotic M, Radosevic-Vidacek B, Koscec Bjelajac A. Morningness-eveningness and daytime functioning in university students: the mediating role of sleep characteristics. J Sleep Res. 2017;26(2):210–218. doi: 10.1111/jsr.12467. [DOI] [PubMed] [Google Scholar]
- 30.Migacz E, Wichniak A, Kukwa W. Are questionnaires reliable in diagnosing sleep-disordered breathing in university students. J Laryngol Otol. 2017;131(11):965–971. doi: 10.1017/S0022215117001839. [DOI] [PubMed] [Google Scholar]
- 31.Escobar-Córdoba F, Eslava-Schmalbach J. Colombian validation of the Pittsburgh Sleep Quality Index. Rev Neurol. 2005;40(3):150–155. [PubMed] [Google Scholar]
- 32.Chica-Urzola HL, Escobar-Córdoba F, Eslava-Schmalbach J. Validating the Epworth sleepiness scale. Rev Salud Publica (Bogota) 2007;9(4):558–567. doi: 10.1590/s0124-00642007000400008. [DOI] [PubMed] [Google Scholar]
- 33.Ruiz-Morales A, Hidalgo-Martinez P, Páez S, Perilla AM, González CA. Correlation of the Epworth somnolence scale with the diagnosis and severity of the obstructive sleep apnea-hypopnea syndrome (OSAHS) Rev Colomb Neumol. 2013;25(1):10–15. [Google Scholar]
- 34.Polanía-Dussan IG, Escobar-Córdoba F, Eslava-Schmalbach J, Netzer NC. Validación colombiana del cuestionario Berlín. Rev Fac Med. 2013;61(3):231–238. [Google Scholar]
- 35.Netzer NC, Hoegel JJ, Loube D, Netzer CM, Hay B, Alvarez-Sala R, et al. Prevalence of symptoms and risk of sleep apnea in primary care. Chest. 2003;124(4):1406–1414. doi: 10.1378/chest.124.4.1406. [DOI] [PubMed] [Google Scholar]
- 36.Zimmerman M, McGlinchey JB, Young D, Chelminski I. Diagnosing major depressive disorder: II: is there justification for compound symptom criteria? J Nerv Ment Dis. 2006;194(4):235–240. doi: 10.1097/01.nmd.0000207423.36765.89. [DOI] [PubMed] [Google Scholar]
- 37.Wakefield JC. DSM-5: proposed changes to depressive disorders. Curr Med Res Opin. 2012;28(3):335–343. doi: 10.1185/03007995.2011.653436. [DOI] [PubMed] [Google Scholar]
- 38.Gobierno Nacional de la República de Colombia Estudio nacional de consumo de sustancias psicoactivas en poblacion escolar - Informe final. 2011. [2018 Jul 20]. Internet. Available from: https://www.unodc.org/documents/colombia/Documentostecnicos/Estudio_Consumo_Escolares.pdf.
- 39.Ministerio de Educación de Colombia Graduados - Observatorio laboral para la educación. [2018 Jul 20]. Internet. Available from: http://bi.mineducacion.gov.co:8380/eportal/web/men-observatorio-laboral/ubicacion-geografica.
- 40.Rosales E, Egoavil M, Cruz CL, Castro JR. Somnolencia y calidad del sueño en estudiantes de medicina de una universidad peruana. An Fac Med Lima. 2007;68(2):150–158. [Google Scholar]
- 41.Del Pielago Meoño AF, Failoc Rojas VE, Plasencia Dueñas EA, Díaz Vélez C. Calidad de sueño y estilo de aprendizaje en estudiantes de Medicina Humana de la Universidad Nacional Pedro Ruiz Gallo. Acta Méd Peruana. 2013;30(4):63–68. [Google Scholar]
- 42.Corrêa CC, Oliveira FK, Pizzamiglio DS, Ortolan EVP, Weber SAT. Sleep quality in medical students: a comparison across the various phases of the medical course. J Bras Pneumol. 2017;43(4):285–289. doi: 10.1590/S1806-37562016000000178. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Machado-Duque ME, Echeverri Chabur JE, Machado-Alba JE. Excessive Daytime Sleepiness, Poor Quality Sleep, and Low Academic Performance in Medical Students. Rev Colomb Psiquiatr. 2015;44(3):137–142. doi: 10.1016/j.rcp.2015.04.002. [DOI] [PubMed] [Google Scholar]
- 44.James BO, Omoaregba JO, Igberase OO. Prevalence and correlates of poor sleep quality among medical students at a Nigerian university. Ann Niger Med. 2011;5(1):1–5. [Google Scholar]
- 45.Lucero C, Buonanotte CF, Perrote FM, Concari IA, Quevedo P, Passaglia G, et al. Trastornos del sueño-vigilia en alumnos de 5. ° año de Medicina de la Universidad Nacional de Córdoba y su impacto sobre el rendimiento académico. Neurol Argentina. 2014;6(4):184–192. [Google Scholar]
- 46.Rodrigues RN, Viegas CA, Abreu E Silva AA, Tavares P. Daytime sleepiness and academic performance in medical students. Arq Neuropsiquiatr. 2002;60(1):6–11. doi: 10.1590/s0004-282x2002000100002. [DOI] [PubMed] [Google Scholar]
- 47.Cardoso HC, Bueno FCC, Mata JC, Alves APR, Jochims I, Vaz Filho IHR, et al. Avaliação da qualidade do sono em estudantes de Medicina. Rev Bras Educ Med. 2009;33(3):349–355. [Google Scholar]
- 48.Ulloque-Caamaño L, Monterrosa-Castro A, Carriazo-Julio S. Somnolencia diurna y rendimiento académico de estudiantes de medicina de una universidad publica Colombiana. Rev Cienc Biom. 2013;4(1):31–41. [Google Scholar]
- 49.Pérez-Olmos I, Ibáñez-Pinilla M. Night shifts, sleep deprivation, and attention performance in medical students. Int J Med Educ. 2014;5:56–62. doi: 10.5116/ijme.531a.f2c9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Ruiz-Morales A, Hidalgo Martínez P, Sanchez Mejía SI, Perea Bello AH, Segura Salguero JC, Chavarriaga Soto J, et al. Síndrome de apnea-hipopnea obstructiva del sueño (SAHOS) y su relación con complicaciones perioperatorias: revisión de la literatura. Univ Med (Colombia) 2014;55(4):407–423. [Google Scholar]
- 51.Ozoh OB, Iwuala SO, Desalu OO, Ojo OO, Okubadejo NU. An Assessment of the Knowledge and Attitudes of Graduating Medical Students in Lagos, Nigeria, Regarding Obstructive Sleep Apnea. Ann Am Thorac Soc. 2015;12(9):1358–1363. doi: 10.1513/AnnalsATS.201412-561OC. [DOI] [PubMed] [Google Scholar]
- 52.Cherrez Ojeda I, Jeffe DB, Guerrero T, Mantilla R, Santoro I, Gabino G, et al. Attitudes and knowledge about obstructive sleep apnea among Latin American primary care physicians. Sleep Med. 2013;14(10):973–977. doi: 10.1016/j.sleep.2013.06.005. [DOI] [PubMed] [Google Scholar]
- 53.Chang PP, Ford DE, Mead LA, Cooper-Patrick L, Klag MJ. Insomnia in young men and subsequent depression: The Johns Hopkins Precursors Study. Am J Epidemiol. 1997;146(2):105–114. doi: 10.1093/oxfordjournals.aje.a009241. [DOI] [PubMed] [Google Scholar]
- 54.Hidalgo MP, Caumo W. Sleep disturbances associated with minor psychiatric disorders in medical students. Neurol Sci. 2002;23(1):35–39. doi: 10.1007/s100720200021. [DOI] [PubMed] [Google Scholar]
- 55.Loayza HMP, Ponte TS, Carvalho CG, Pedrotti MR, Nunes PV, Souza CM, et al. Association between mental health screening by self-report questionnaire and insomnia in medical students. Arq Neuropsiquiatr. 2001;59(2-A):180–185. doi: 10.1590/s0004-282x2001000200005. [DOI] [PubMed] [Google Scholar]
- 56.Bernert RA, Turvey CL, Conwell Y, Joiner Jr TE. Association of poor subjective sleep quality with risk for death by suicide during a 10-year period: a longitudinal, population-based study of late life. JAMA Psychiatry. 2014;71(10):1129–1137. doi: 10.1001/jamapsychiatry.2014.1126. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Eller T, Aluoja A, Vasar V, Veldi M. Symptoms of anxiety and depression in Estonian medical students with sleep problems. Depress Anxiety. 2006;23(4):250–256. doi: 10.1002/da.20166. [DOI] [PubMed] [Google Scholar]
- 58.Zailinawati AH, Teng CL, Chung YC, Teow TL, Lee PN, Jagmohni KS. Daytime sleepiness and sleep quality among Malaysian medical students. Med J Malaysia. 2009;64(2):108–110. [PubMed] [Google Scholar]
- 59.Ford DE, Kamerow DB. Epidemiologic study of sleep disturbances and psychiatric disorders: An opportunity for prevention? JAMA. 1989;262(11):1479–1484. doi: 10.1001/jama.262.11.1479. [DOI] [PubMed] [Google Scholar]
- 60.Sáez GJ, Santos RG, Salazar CK, Carhuancho-Aguilar J. Calidad del sueño relacionada con el rendimiento académico de estudiantes de medicina humana. Horiz Med (Barcelona) 2013;13(3):25–32. [Google Scholar]
- 61.Makanjuola AB, Daramola TO, Obembe AO. Psychoactive substance use among medical students in a Nigerian university. World Psychiatry. 2007;6(2):112–114. [PMC free article] [PubMed] [Google Scholar]
- 62.Lund HG, Reider BD, Whiting AB, Prichard JR. Sleep patterns and predictors of disturbed sleep in a large population of college students. J Adolesc Health. 2010;46(2):124–132. doi: 10.1016/j.jadohealth.2009.06.016. [DOI] [PubMed] [Google Scholar]
- 63.Estrada Aranda BD, Delgado Álvarez C, Landero Hernández R, González Ramírez MT. Propiedades psicométricas del modelo bifactorial del BDI-II (versión española) en muestras mexicanas de población general y estudiantes universitarios. Univ Psychol. 2015;14(1):125–136. [Google Scholar]
- 64.Beck AT, Steer RA, Ball R, Ranieri W. Comparison of Beck Depression Inventories -IA and -II in psychiatric outpatients. J Pers Assess. 1996;67(3):588–597. doi: 10.1207/s15327752jpa6703_13. [DOI] [PubMed] [Google Scholar]
- 65.Kroenke K, Wu J, Yu Z, Bair MJ, Kean J, Stump T, et al. Patient Health Questionnaire Anxiety and Depression Scale: Initial Validation in Three Clinical Trials. Psychosom Med. 78(6):716–727. doi: 10.1097/PSY.0000000000000322. [DOI] [PMC free article] [PubMed] [Google Scholar]