Abstract
Purpose: To validate and to evaluate the performance in depression screening of the Patient’s Health Questionnaire – 9 (PHQ-9) for Romanian patients with Type 2 Diabetes Mellitus (T2DM).
Patients and methods: In a consecutive-case, population-based, non-interventional study design 107 hospitalized patients with T2DM were enrolled. The PHQ-9 was evaluated in regard to its acceptability, ceiling and floor effect, reproducibility, and test–retest performance. Its validity was tested by comparing it to the Beck depression scale, which is a validated tool for this patient population.
Results: The PHQ-9 questionnaire had an excellent acceptability, having no non-responders for any of its questions and a median completion time of 3 mins and 12 s, a good reliability with a Cronbach’s α=0.897 [0.865–0.924] 95%CI, a good test–retest performance (Spearman’s rho =0.972; p<0.001 between two administrations of the test) without any ceiling or floor effect observed for the total score. At the same time, the score demonstrated a good validity when compared to a more complex and previously validated instrument like the Beck depression scale (Spearman’s rho =0.777; p<0.001). The limits of agreement (Bland-Altmann analysis) between the two measurements are 2.6 to −2.5 points.
Conclusion: The PHQ-9 is a valid, useful instrument for depression screening in Romanian patients with T2DM.
Keywords: questionnaire validation, depression screening, chronic diseases, screening instrument, depression in diabetes
Introduction
Diabetes Mellitus is one of the major public health issues we are confronted with on a global scale. The disease’s burden is mediated through its chronic and acute complications. Those, in turn, may lead, both in a direct and indirect manner, to impaired quality of life or disability.1 Among the indirect pathways in which diabetes has an impact on the patient’s health and quality of life and respectively on public health, an important role is attributed to the incidence of depression. Depression is cited to occur more frequently, being also more severe in patients with chronic diseases, including diabetes.2 Since effective interventions are available for treating depression, it becomes clear that depression screening is the first step of paramount importance for an adequate intervention.3 For depression screening several tools are available – however, it is known that different populations have significantly similar behaviors, raising the need to validate the assessment tools used. In the Romanian population of patients with Type 2 Diabetes Mellitus (T2DM), the Beck depression scale is a validated and widely used tool for depression’s screening and severity assessment.4 Patients with T2DM, similar to patients with other chronic, lifelong diseases, have a series of particularities regarding their mental and psychic dimensions. Among those particularities, depression or anxiety prevalence and severity, coping mechanisms and self-esteem differ in diabetic patients when compared to the general population.5,6 Taking these facts into account, the need for adaptation and validation of the instruments used for assessing these particular patients from a psychological point of view occurs.
The Beck depression scale is a useful instrument used for the assessment of depression, having good sensitivity and specificity. At the same time, the Beck scale is a validated tool for use in Romanian patients with T2DM.4 However, despite its good performance, the Beck scale has a relatively long completion time, and it has been reported that some questions may be difficult to comprehend by the patients, making the scale an appropriate instrument for use in hospitalized patients, in patients at high-risk or those with established depression, but difficult to use as a screening tool.7 On the other hand, the Patient’s Health Questionnaire – 9 (PHQ-9) is a much simpler, quicker, and easier to use depression screening instrument, developed and designed to be used as a screening method which brings the perspective of an improved, cost-efficient screening.8 The main barrier in the use of the PHQ-9 instrument in this subpopulation is caused by the fact that until now there is no validation for the translated and culturally adapted version of the instrument on the Romanian population of patients with T2DM. The importance of validating an effective screening instrument for depression which allows cost-efficient diagnosis, with minimal requirements are emphasized by the major impact depression has on the patient’s quality of life. This initiative is further supported by the existence of effective intervention methods with good outcomes if readily implemented, the increased prevalence of both T2DM in the general population and the surging incidence of depression in patients with T2DM.
The aim of this study was to validate and to evaluate the performance of the translated and culturally adapted version of the PHQ-9 scale on the diagnosis of depression in the Romanian population of patients with T2DM.
Material and methods
Study design and patients
A total of 107 patients with T2DM were enrolled in a consecutive-case, population-based, cross-sectional study design. All patients were hospitalized in the Diabetes Clinic of the “Pius Brinzeu” University Emergency Hospital, Timisoara, Romania. The study design, protocol, and informed consent form were reviewed and approved the Ethics Committee of the Emergency Hospital Timisoara; all patients involved in the study provided written informed consent prior to any study-related procedure or activity.
The enrolled patients had a median age of 63 years, a median duration of diabetes of 8 years, 48.6% of them (n=52) being males. The patient’s characteristics at enrollment are very similar to the characteristics of the general Romanian patients with diabetes making thus the studied sample highly representative for the Romanian population of patients with T2DM.9
The detailed characteristics of the studied sample are presented in Table 1.
Table 1.
Patient’s general characteristics at enrollment
| Parameter | Value |
|---|---|
| Age (years)a | 63 [56–69] |
| Male genderb | 52 (48.6%) |
| Diabetes duration (years)a | 8 [4–13] |
| HbA1c (%)c | 9.0±1.8 |
| BMI (kg/m2)c | 32.6±6.2 |
| Diabetic neuropathy (%)b | 72 (67.3%) |
| Diabetic retinopathy (%)b | 32 (29.9%) |
| Hypertension (%)b | 93 (86.9%) |
| History of stroke (%)b | 9 (8.4%) |
| Peripheral artery disease (%)b | 9 (8.4%) |
| Peripheral venous insufficiency (%)b | 25 (23.4%) |
| Non-alcoholic steatohepatitis (%)b | 61 (57.0%) |
| Total cholesterol (mg/dL)c | 202±90 |
| LDLc (mg/dL)c | 116±50 |
| HDLc (mg/dL)c | 42±14 |
| Triglycerides (mg/dL)a | 158 [108–269] |
| Uric acid (mg/dL)c | 5.5±1.9 |
| Urinary albumin/creatinine ratio (mg/g)a | 29.0 [15.1–110.5] |
| Usage of anti-diabetic treatment classes in the study group – patients using the mentioned class of drugs | |
| Metforminb | 72 (67.3%) |
| Sulphonylureasb | 36 (33.6%) |
| Basal insulinb | 33 (30.8%) |
| Bolus insulinb | 28 (26.2%) |
| Dipeptidyl peptidase – 4 inhibitorsb | 22 (20.6%) |
| Glucagon-like peptide 1 receptor agonistsb | 13 (12.1%) |
| Sodium-glucose co-transporter 2 inhibitorsb | 18 (16.8%) |
Note: aPresented as median [interquartile range]. bPresented as number (%). cPresented as mean ± standard deviation.
Depression assessments
PHQ-9
The PHQ-9 is a shortened version of the Patient Health Questionnaire, containing only 9 items, one for each DSM-IV criteria which are encompassing in fact the criteria for diagnosing depression. With 9 items, the PHQ-9 instrument is approximately half as lengthy as the majority available instruments, providing good sensitivity, being easily completed, over a short timespan.10 The PHQ-9 is derived from the main Patient Health Questionnaire, which can be entirely self-administered by the patient and which assesses eight diagnosis corresponding to specific DSM-IV classes: major depressive disorder, panic disorder, anxiety disorder, bulimia nervosa, other depressive disorder, probable alcohol dependence, somatoform, and binge eating disorder. The PHQ-9 questionnaire, derived from the main PHQ contains only items for depressive disorder, aiming to provide a shorter and easier method for depression screening.8
The use of PHQ-9 instrument allows quantification of depression severity. Each of the 9 items is scored by the patient on a scale from 0 (not at all) to 3 (nearly every day) summing a possible total score from 0 to 27 points. An increased PHQ-9 score is associated with a more severe degree of depression.
The following stages of depression may be diagnosed based on the PHQ-9 score:
0–4: absent or minimal
5–9: mild depression
10–14: moderate depression
15–19: moderately severe depression
20–27: severe depression
Beck depression scale
Beck depression inventory is a tool developed to diagnose depression in both clinical settings and on healthy individuals. The scale contains 21 items, aiming to evaluate both the positive diagnosis and severity of the depression-related symptomatology. The positive criteria for depression are in accordance with the Diagnostic and Statistical Manual of Mental Disorders, the fourth edition. Each item contained is a list of four statements, arranged in increasing order in respect to the severity of an individual symptom of depression. The Beck depression scale is validated both for healthy subjects and specific sub-populations, including patients with T2DM and may be used extending downward to 13 years of age.7
According to the Beck scale grading system, a higher score is associated with a more severe depression-related symptomatology.
PHQ-9 validation process
Translation
The original version of the PHQ-9 was developed by Robert L. Spitzer et al, during a scientific grant funded by Pfizer Inc, and there is no need for permission to use, reproduce, translate, or distribute the instrument. The Romanian version of the instrument was translated by the developers of the instrument and was downloaded from: https://www.phqscreeners.com (accessed at 25th of January 2019).8 This version is considered to be the official translated version of the instrument; however, it is not validated for the Romanian population of patients with T2DM.
Construct validity
For the assessment of the construct and convergent validity, the Romanian version of the PHQ-9 was handed out to all patients in parallel with the Romanian version of the Beck depression screening instrument, which is a validated questionnaire for the Romanian population of patients with T2DM. Approval to use the Beck instrument was obtained by the developers of the instrument. As per licensing instructions, there is no need to ask for permission to use the PHQ-9 questionnaire.
The convergent validity of the PHQ-9 questionnaire was evaluated using correlation and regression analysis – a higher correlation coefficient being associated with an increased agreement between the analyzed scales.
Reliability and internal consistency
The internal consistency of the screening instrument was assessed using the Cronbach’s alpha method, a measure which analyses the cross-related associations between the component items of the questionnaire. In other words, the Cronbach’s alpha method provides the tools to evaluate at which extent different items of the questionnaire are pointing to a similar outcome measured. In practice, the Cronbach’s alpha method brings information related to the probability that all items are measuring a similar construct.
Reproducibility
The reproductibility of the instrument was evaluated using a test–retest assessment, having the retest performed after 3 days following the initial administration. The rationale for this short period was that after the interventions during the hospitalization or even after discharge, patient’s perception regarding depression might be influenced. Thus, we considered that the optimal test–retest session should have the presented timeframe. To be sure that no other confounders might influence the outcome, as an exclusion criterion, patients which underwent significant intervention in this timeframe were removed from the study. The test–retest assessment was performed using two methods: the correlation and regression analysis between the two instances of questionnaire administration respectively Bland–Altman analysis, considering the average results from the two measurements as reference for comparison. During this analysis, an increased correlation coefficient between the two time-moment was considered to be associated with an increased reproducibility of the measure.
Statistical analysis
Data were collected and analyzed using SPSS v.17 statistical software package (SPSS Inc, Chicago, IL, USA) and are presented as mean ± standard deviation for numerical variables with Gaussian distribution, median, and [interquartile range] for numerical variables with non-parametric distribution respectively number of individuals and (percentage from the total) in case of categorical variables. The distribution of the variables was evaluated using the Shapiro–Wilk test (the variables being considered part of a Gaussian distribution if a p-value from the Shapiro–Wilk test was higher or equal to 0.05).
To evaluate the strength of the association between numerical variables, correlation and regression analysis was performed. For variables with Gaussian distribution, the Pearson’s correlation coefficient was used, respectively Spearman’s one correlation coefficient for variables with non-parametric distribution the. The statistical significance of the correlation between variables was assessed using the t-score distribution test.
The test–retest differences were analyzed using the Bland–Altman plot method.
In this study, the threshold for statistical significance was considered a p-value lower than 0.05.
Results
PHQ-9 score distribution
In both initial and follow-up administrations of the PHQ-9 scale, 107 responses were recorded. The median scores were 5 points for both instances. The distribution of the scores was unimodal, non-Gaussian, Kolmogorov–Smirnov’s (Lillefors corrected) =0.186; p<0.001. The deviances from the Gaussian distribution are presented in Figure 1.
Figure 1.
Detrended normal Q–Q plot of PHQ-9 scores.
According to our results, the occurrence of depression symptomatology, as evaluated by the PHQ-9 instrument, was the following: 55 (51.4%) no signs of depression, 21 (19.6%) mild, 19 (17.8%) moderate, 7 (6.5%) moderate-severe, respectively, 5 (4.7%) severe depression. The distribution of the scores from the initial and follow-up is presented in Figure 2.
Figure 2.
PHQ-9 distribution score at the initial and follow-up evaluation.
Acceptability, ceiling, and floor effect
At both initial and follow-up evaluation all the patients included in the study (n=107) responded to all of the questionnaire’s items. The median time needed for completion was 3 mins and 12 s (minimum 1:54 mins; maximum 5:12 mins). There was no question for which more than 21.5% of the responses concentrated at the top end of the scale, while the question with the most of the responses at the lowest point of the scale concentrated 52.3% of those minimum responses. For the total score, there was no aggregation at the top of the scale, no participant recording a maximum score. The minimum possible score was also in the desired range, only 4.7% of the participants having a total score of zero points on the PHQ-9 scale. Thus, we can accept that the floor and ceiling effect was ruled out.
Reliability
The internal consistency of the PHQ-9 questionnaire for the Romanian population of patients with T2DM was evaluated using the Cronbach’s alpha method. The PHQ-9 proved to have a good internal consistency as revealed by the Cronbach’s α=0.897 [0.865–0.924] 95%CI at the first administration respectively α=0.714 [0.624–0.789] 95% CI at the repeated measurement. In the same time, for both instances, the nine components inside the questionnaire had a high inter-item correlation with a decreased inter-item covariation (Table 2)
Table 2.
Inter-item correlations and covariations
| First administration (n=107) | Follow-up (n=107) | |
|---|---|---|
| Inter-item covariances | 0.366 | 0.376 |
| Inter-item correlations | 0.499 | 0.441 |
The inter-item correlations for each possible pair of questions from both first and follow-up administration of the PHQ-9 ranged from 0.390 up to 0.730 which denotes also a good internal consistency of the questions in the questionnaire.
Validity
The PHQ-9 scores demonstrated a strong positive and statistically significant correlation with the previously validated Beck scale, in both first (Spearman’s rho =0.777; p<0.001; Figure 3) and second administration (Spearman’s rho =0.775, p<0.001; Figure 4), indicating thus a good convergent validity.
Figure 3.
Correlation between Beck and PHQ-9 scales (first administration).
Figure 4.
Correlation between Beck and PHQ-9 scales (second administration).
Test–retest
A good test–retest performance was observed, as described by the strong, positive, and statistically significant correlation between the scores obtained at the first and second administration of the questionnaire (Spearman’s rho =0.972; p<0.001; Figure 5). The differences between the two administrations of the PHQ-9 scale were not statistically significant (the median scores being 5 at both administrations, p=0.982; Mann–Whitney U test). The mean difference between the two instances was 0.61±1.147 (minimum difference =0; maximum difference =7) points, the differences being statistically non-significant (p=0.76; unpaired t-student test). No heteroscedasticity was observed between the two administrations (Levene’s p=0.82)
Figure 5.
Correlation between the two administrations of the PHQ-9 questionnaires.
The Bland-Altmann plot (Figure 6) showed no systematic bias between the two measurements (initial and follow-up administration of the PHQ-9 questionnaire), the limits of agreement (as described by the 1.96 standard deviation interval) ranging from 2.6 to −2.5 point on the PHQ-9 scale, describing thus a good reproductivity for the measurement instrument.
Figure 6.
Bland-Altmann plot for the two instances of PHQ-9 administration.
Discussion
Alongside other more well-known complications or co-morbidities11 which are present in patients with T2DM, such as obesity, heart disease, sleep apnea or non-alcoholic fatty liver disease, depression is a frequent condition occurring in patients with T2DM.12,13 The presence of depression decreases both the overall prognosis of T2DM (mediated mainly through impaired adherence to diabetes-related self-management methods) and the patient’s quality of life, quality of life which is cited in many other chronic as well as acute diseases as revealed by several scoring instruments.14 Currently, there are many effective interventions available that are aiming to treat depression, with spectacular results challenged by neglectable side-effects.15 These interventions are having an increased impact when initiated in the early stages of depression,16 generating the premises of bringing improvements to the patient’s quality of life, thus inadvertently improving the prognosis of T2DM.17
Having a well-documented increase in prevalence among patients with T2DM18 and knowing that T2DM has a high prevalence and incidence worldwide, it becomes clear that the depression associated with T2DM is a veritable public health issue.19 These facts emphasize the need for a proactive screening of depression in all patients with T2DM. Given the large number of patients at risk as well as the complex interventions which are frequently required to manage the main disease, it is evident that the rate of adoption success for such a tool is influenced, aside from its performance, by its ease of use and time needed for completion.20 Consecrated tools, despite their good diagnosis performances are quite difficult to self-administer by the patient, frequently need substantial intervention from the health care professional and are time-consuming.21 All these disadvantages are nullified when using a simple tool such as the PHQ-9.
Our study results reveal that PHQ-9 may become a useful tool in the diagnosis of depression in the Romanian population of patients with T2DM, demonstrating good internal consistency, good reliability and validity (when compared to a consecrated tool) as well as a good test–retest performance, all of this in the absence of a ceiling or floor effect. The fact that there were no questions which the patients found it impossible to respond to and the fact that the median time of completion was approximately 3 mins, generate the premises that PHQ-9 is a valuable screening tool in this particular population, feasible to be used on a large scale and which should be performed on all patients on a regular basis corresponding to their clinical status and previous results.
The good validation results obtained in this study are sustained by other validations of the same PHQ-9 score, validations which shown promising results in other sub-populations, on different cultures and on patients with different conditions as well as healthy individuals.22–25
Particularities of this study. Strengths and weaknesses
This study is the first one which aimed to validate the translated and culturally adapted version of the PHQ-9 depression screening instrument for the Romanian population of patients with T2DM. The good performance observed for all the analyzed components, alongside the adequate sample size of the studied sample allows us to conclude that the PHQ-9 may be considered a validated tool for depression screening in these patients. The studied sample had very similar baseline characteristics to the ones found in the general population of patients with T2DM from Romania (ie, age, diabetes duration, prevalence of complications, co-morbidities, or similar profile of risk biomarkers like lipid profile, body mass index, or uric acid) which allows us to conclude that the sample in this study is a representative one for the population of Romanian patients with T2DM. The high prevalence of diabetes complications as well as the relatively poor glycemic control observed in our study cohort is explained by the fact that the studied patients were hospitalized due to metabolic imbalances which were not possible to be treated just in outpatient scenario. In these patients, we may expect higher rates of complications. The high rate of depression may also be linked to the presence of metabolic imbalances which required hospitalization and also to the high rate of chronic complications. A relative weakness of the study was that it evaluated only patients in an inpatient setting, all the enrolled participants were hospitalized for metabolic-related imbalance. These enrollment criteria led to a higher value of the HbA1c (9.0% ±1.8) compared to the average value we should expect when screening the general population of patients with T2DM (both in outpatient and inpatient setting). However, there are no sustainable reasons to think that this might decrease the value of the validation process – a validated tool in any of those scenarios should work in the other one without any kind of differences. In the same time, another relative weakness of the study is that depression diagnosis was performed using two questionnaires and not by a DSM-based interview.
Conclusion
The PHQ-9 self-administered questionnaire is a valid tool for depression screening in patients with T2DM. The PHQ-9 screening instrument has an excellent acceptability, a good reliability, and test–retest performance in parallel with a good agreement when compared to other more complex, previously validated instruments. PHQ-9 scale proved to be a valuable, useful, and reliable instrument for depression screening in this sub-population.
Disclosure
The authors report no conflicts of interest in this work.
References
- 1.International Diabetes Federation. IDF Diabetes Atlas 8th ed. Brussels: International Diabetes Federation; 2017. [Google Scholar]
- 2.Holt RIG, De Groot M, Golden SH. Diabetes and depression. Curr Diab Rep. 2014;14. doi: 10.1007/s11892-014-0491-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Kato E, Beswick-Escanlar V. Screening for depression in adults. Am Fam Physician. 2016. doi: 10.1097/01.ogx.0000482188.34232.33 [DOI] [PubMed] [Google Scholar]
- 4.Chereches RM, Litan CM, Zlati AM, Bloom JR. Does co-morbid depression impact diabetes related costs? Evidence from a cross-sectional survey in a low-income country. J Ment Health Policy Econ. 2012;15(3):127–138. [PubMed] [Google Scholar]
- 5.Albai A, Sima A, Papava I, Roman D, Andor B, Gafencu M. Association between coping mechanisms and adherence to diabetes-related self-care activities: a cross-sectional study. Patient Prefer Adherence. 2017;11:1001–1007. doi: 10.2147/PPA.S140146 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Rivera-Hernandez M. Depression, self-esteem, diabetes care and self-care behaviors among middle-aged and older Mexicans. Diabetes Res Clin Pract. 2014;105:70–78. doi: 10.1016/j.diabres.2014.04.017 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Beck AT, Steer RA, Brown GK. Manual for the Beck Depression Inventory-II. San Antonio: TX Psychol Corp; 1996. doi: 10.1002/(SICI)1097-0142(19991215)86:123.3.CO;2-I [DOI] [Google Scholar]
- 8.Kroenke K, Spitzer RL, Williams JBW. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16:606–613. doi: 10.1046/j.1525-1497.2001.016009606.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Mota M, Popa SG, Mota E, et al. Prevalence of diabetes mellitus and prediabetes in the adult Romanian population: PREDATORR study. J Diabetes. 2016;8(3):336–344. doi: 10.1111/1753-0407.12297 [DOI] [PubMed] [Google Scholar]
- 10.Reddy P, Philpot B, Ford D, Dunbar JA. Identification of depression in diabetes: the efficacy of PHQ-9 and HADS-D. Br J Gen Pract. 2010;60(575):e239–e245. doi: 10.3399/bjgp10X502128 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Roșu -M-M, Moța M. The prevalence of arterial hypertension in Romanian adult population: results from the PREDATORR Study. Rom J Diabetes Nutr Metab Dis. 2018;25(3):303–312. doi: 10.2478/rjdnmd-2018-0036 [DOI] [Google Scholar]
- 12.Serafinceanu C, Elian V, Catrinoiu D, et al. Clinical and therapeutic characteristics of patients with Type 2 Diabetes Mellitus in Romania – MENTOR Study. Rom J Diabetes Nutr Metab Dis. 2018;25(4):409–418. doi: 10.2478/rjdnmd-2018-0049 [DOI] [Google Scholar]
- 13.Sima A, Timar R, Vlad A, et al. Nonalcoholic fatty liver disease: a frequent condition in type 2 diabetic patients. Wien Klin Wochenschr. 2014;126:11–12. doi: 10.1007/s00508-014-0530-8 [DOI] [PubMed] [Google Scholar]
- 14.Andor B, Patrascu JM, Florescu S, et al. Comparison of different knee implants used on patients with osteoarthritis control study. Mater Plast. 2016;53(1):119–125. [Google Scholar]
- 15.Sartorius N. Depression and diabetes. Dialogues Clin Neurosci. 2018. doi: 10.1016/j.mppsy.2009.03.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Briones-Aranda A. Why early psychological attention for type 2 diabetics could contribute to metabolic control. Rom J Diabetes Nutr Metab Dis. 2018;25(3):329–334. doi: 10.2478/rjdnmd-2018-0039 [DOI] [Google Scholar]
- 17.Mezuk B, Eaton WW, Albrecht S, Golden SH. Depression and type 2 diabetes over the lifespan: a meta-analysis. Diabetes Care. 2008;31:2383–2390. doi: 10.2337/dc08-0985 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Bădescu SV, Tătaru C, Kobylinska L, et al. The association between diabetes mellitus and depression. J Med Life. 2016;9(2):120–125. [PMC free article] [PubMed] [Google Scholar]
- 19.Anderson RJ, Freedland KE, Clouse RE, Lustman PJ. The prevalence of comorbid depression in adults with diabetes: a meta-analysis. Diabetes Care. 2001;24:1069–1078. doi: 10.2337/diacare.24.6.1069 [DOI] [PubMed] [Google Scholar]
- 20.Katon WJ, Lin EHB, Von Korff M, et al. Collaborative care for patients with depression and chronic illnesses. N Engl J Med. 2010. doi: 10.1056/NEJMoa1003955 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Berg A, Lönnqvist J, Palomäki H, Kaste M. Assessment of depression after stroke a comparison of different screening instruments. Stroke. 2009;40:523–529. doi: 10.1161/STROKEAHA.108.527705 [DOI] [PubMed] [Google Scholar]
- 22.Monahan PO, Shacham E, Reece M, et al. Validity/reliability of PHQ-9 and PHQ-2 depression scales among adults living with HIV/AIDS in Western Kenya. J Gen Intern Med. 2009. doi: 10.1007/s11606-008-0846-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Seo JG, Park SP. Validation of the Patient Health Questionnaire-9 (PHQ-9) and PHQ-2 in patients with migraine. J Headache Pain. 2015;16. doi: 10.1186/s10194-015-0552-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Beard C, Hsu KJ, Rifkin LS, Busch AB, Björgvinsson T. Validation of the PHQ-9 in a psychiatric sample. J Affect Disord. 2016;193:267–273. doi: 10.1016/j.jad.2015.12.075 [DOI] [PubMed] [Google Scholar]
- 25.Van Steenbergen-Weijenburg KM, De Vroege L, Ploeger RR, et al. Validation of the PHQ-9 as a screening instrument for depression in diabetes patients in specialized outpatient clinics. BMC Health Serv Res. 2010;10. doi: 10.1186/1472-6963-10-235 [DOI] [PMC free article] [PubMed] [Google Scholar]