Accuracy of nine-item Patient Health Questionnaire against psychiatric diagnosis for depression among people with HIV

Abstract

Objective:

The aim of this study was to assess the performance of the nine-item Patient Health Questionnaire (PHQ-9) against psychiatrist diagnosis in people with HIV (PWH).

Design:

Cross-sectional analysis of data collected between January 2018 and July 2022 across five sites in Cameroon, Cote d’Ivoire, Kenya, Senegal, and the Republic of Congo. Participants were ≥18 years and receiving HIV care at the participating site. PHQ-9 was administered by study staff followed by a psychiatrist's evaluation within 3 days.

Results:

Overall, 778 participants with complete data were included: 297 (38.2%) in Cameroon, 132 (17.0%) in Congo, 148 (19.0%) in Cote d’Ivoire, 98 (12.6%) in Kenya, and 103 (13.2%) in Senegal. The area under the curve for PHQ-9 score was generally high ranging from 0.935 [95% confidence interval (CI): 0.893, 0.977] in Cote d’Ivoire to 0.768 (95% CI: 0.589, 0.947) in Congo. However, for the common cut-off score ≥10, sensitivity was low: 50% or lower in Cameroon, Congo and Senegal, 66.7% in Kenya and 70.6% in Cote d’Ivoire. But negative predictive values (NPV) were high: 98.9% (95% CI: 96.9%, 99.8%) in Cameroon, 96.1 (95% CI: 91.1, 98.7) in Cote d’Ivoire, 96.3% (95% CI: 89.7%, 99.2%) in Kenya, 95.7% (95% CI: 90.2%, 98.6%) in Congo, and 89.0% (95% CI: 81.2%, 94.4%) in Senegal.

Interpretation:

Across all countries, PHQ-9 score ≥10 performed very poorly (low sensitivity) as a tool to identify psychiatrist diagnosed depression. However, the observed high NPV suggests it can be used to rule out depression.

Introduction

Depression is common among people with HIV (PWH) and affects their quality of life, adherence to antiretroviral therapy, and viral suppression [1–4]. A timely diagnosis of major depressive disorder (MDD) and treatment is thus essential for retention in care, better quality of life, and clinical outcomes among PWH [5–8]. Affordable pharmacological and nonpharmacological interventions exist but access is limited because of challenges identifying depression, particularly in people with preexisting conditions like HIV [9–11]. Diagnosis requires the exercise of clinical judgment, assessment of symptom severity and functional impairment, and the exclusion of alternative medical or psychiatric explanations for symptoms [12].

In low-resource settings where such expertise is scarce, structured instruments are often used by nonspecialists as an alternative diagnostic technique to generate ratings of patient symptoms. The rating scores are then used to define cases of a depressive syndrome or measure its severity [13]. Among the screening instruments, the nine-item Patient Health Questionnaire (PHQ-9) is widely preferred [14]. The PHQ-9 was originally developed to screen for and identify severity of depression in American adults in clinical settings via patient self-administration. This instrument is based on the diagnostic criteria for DSM-IV MDD and its total score ranges from 0 to 27. When treated as unidimensional and sum-scored, the PHQ-9, using a score threshold of ≥10, has been found to have high sensitivity (88%) and specificity (89%) for MDD compared to a psychiatrist's diagnosis used as a reference, in United States [14]. Assessments of the PHQ-9 performance in non-Western contexts remain limited. The tool has been minimally evaluated for comprehension and overall performance, particularly in low-resource settings with high HIV burden [15–16]. Studies from sub-Saharan Africa that examined the PHQ-9's validity among different population groups include those in Cameroon, Ethiopia, Ghana, Kenya, Malawi, Nigeria, South Africa, and Uganda [12]. Only three of these studies assessed the PHQ-9's performance among PWH. These studies provided mixed evidence on the PHQ-9's performance in identifying depression with a sensitivity and specificity of 27% and 94% in Cameroon (Gold Standard: Composite International Diagnostic Interview), and 78.7% and 83.4% in South Africa [Gold Standard: Mini International Neuropsychiatric Instrument (MINI)] [17–18]; and 91.6% and 81.2% in Uganda (Gold Standard: MINI) [19].

Reasons for the large variability in PHQ-9's performance among PWH in sub-Saharan Africa have not been investigated. The role of culture has emerged as a salient concern because as a general trend, members of many cultures express mental distress by way of somatic complaints rather than the more psychological affective and cognitive manifestations that are considered the hallmarks of depression in a Western context [20–22]. A similar concern emerges when assessing depression in populations with co-morbid conditions like HIV, because the symptoms assessed may occur as a result of depression, HIV, side effects of antiretroviral therapy (ART), or opportunistic infections. Given these findings, if the PHQ-9 (or the shorter PHQ-2) is to be used to diagnose and initiate PWH on pharmacological or nonpharmacological interventions, as it has been done in reported trials of integration of depression management in HIV care [23,24], assessment of its performance against a psychiatrist's diagnosis in this population is urgently needed.

This study was designed to assess the performance of PHQ-9 as a tool to identify MDD against a diagnosis by a psychiatrist.

Methods

Design, setting, and population

This cross-sectional study was conducted across five countries participating in the International Epidemiology Databases to Evaluate AIDS (IeDEA): two within the Central Africa IeDEA (Brazzaville Ambulatory Treatment Center, the Republic of Congo and Yaoundé Jamot Hospital, Cameroon), two within the West Africa IeDEA (Medical center for blood donor follow-up (CNTS-CI) in Abidjan Cote d’Ivoire and Department of Infectious and Tropical Diseases, CHNU Fann Dakar, Senegal), and one in East Africa IeDEA (AMPATH clinic at Moi Teaching and Referral Hospital in Eldoret, Kenya). Participating hospitals are in urban areas, serve large active cohorts of PWH, and were selected because of onsite psychiatric services. Within each site in Cameroon, Congo, and Kenya, a sample of PWH, 18 years and older, enrolled in care at the site at least 3 months prior (to avoid the high anxiety associated with initial diagnosis that, while it may become depression for some, is more anxiety and fear-driven and may confound measurement of depression [25]) and for less than 24 months HIV care enrollment were eligible for the study. The sample inclusion criteria were applied in Cote d’Ivoire and Senegal, except for the requirement to be enrolled in HIV care for <24 months. Participants were excluded if they were not fluent in any of the languages in which the questionnaire was translated (English, French, Lingala, Kituba, Luo, and Swahili), unable to consent, or declined to participate in the study. At each participating site, healthcare providers shared brief information about the study to potential participants during routine care visit at the participating clinic. Individuals who expressed their interest were then referred to the study staff who assessed their eligibility and conducted informed consent. Study staff then administered the PHQ-9 questionnaire and following administration took the participant to the psychiatrist for evaluation within 3 days. All enrolled participants provided written consent and were compensated for their time and transport to the clinic ($3 in local currency). Any mental disorder diagnosed during psychiatrist evaluation was managed according to standard local protocol including in interpersonal group therapy or pharmacologic management for depression. The study was approved by the Ohio State University Institutional Review Board (Protocol #2018H0195), Albert Einstein College of Medicine Institutional Review Board (#2018-9355), Regional Ethics Committee for Research on Human Health in Cameroon (#1846/CRERSHC/2019), Moi University College of Health Sciences and Moi Teaching and Referral Hospital's Institutional Research and Ethics Committee, Kenya (#3329/IREC/2019/57), the Indiana University Institutional Review Board (#1810880690), the Comite d’Ethique de la Recherche en Sciences de la Sante in Congo (# 017/CEI/FCRM/2018), the National Ethics Committee for Health Research in Senegal (#00036/MAS/CNERS/SP), and the National Ethics Committee for Health and Life Sciences in Cote d’Ivoire (#013–21/MSHP/CNESVS-km).

Assessments

The English, French, and Swahili versions of the PHQ-9 were retrieved from the Pfizer Patient Health Questionnaires website [26]. In Congo, the country team translated the PHQ-9 from French to Lingala and Kituba and then, back-translated to French, and field tested the translated versions as in previous study [8].

Trained research assistants administered the PHQ-9 and a questionnaire that collected socio-demographic information. A PHQ-9 score ≥10, indicating moderate to severe depression, was used to identify probable depressed individuals [27]. After completing the questionnaire, participants were referred to a psychiatrist with experience working with PWH, who independently evaluated them for MDD within 3 days (expect in Kanya all assessments were completed on the same day). Psychiatrists were blinded to the PHQ-9 scores. At all sites, because MINI is not routinely used, for standardization purposes, psychiatrists were provided with MINI and encouraged to use it to guide their evaluation. But no data on MINI was collected as we did not receive permission to use it for this research. In addition, in both Central and East Africa, psychiatrists were provided with a separate diagnosis form to complete at the end of the participant visit and returned to the study team. The diagnosis form asked psychiatrists to evaluate each participant for a diagnosis of Major Depressive Episode/Disorder and Generalized Anxiety Disorder among other. Psychiatrists indicated “Yes” or “No” for each diagnosis. If a participant met similar criteria for a diagnosis but did not meet the DSM-5 criteria in full, they could indicate “Not Specified”. If a positive diagnosis was selected, the psychiatrist specified the time frame in which the symptoms occurred: e.g. corresponding time period: current (over the “past two weeks”), recent (within the past year), or past (over one year ago).

Analyses

Proportions, means, and medians were used to summarize participants’ socio-demographic characteristics. The extent to which the PHQ-9 is able to identify the likely presence or absence of depression was assessed using sensitivity, specificity, negative predictive value (NPV) and positive predictive value (PPV), positive and negative likelihood ratio (LR), and areas under the curve (AUC). The psychiatrist's diagnosis was used as a gold standard. Sensitivity and specificity with corresponding exact binomial 95% confidence intervals (CIs) were calculated as the proportion of participants with depression and the proportion without depression respectively, who were correctly classified as such based on their PHQ-9 score ≥10, respectively. The following alternative PHQ-9 score cut-offs of >4, >7, >11, >14, >17, and >19 were also considered [20]. We also performed the same analyses for the PHQ-2 (i.e., the first two items of the PHQ-9) using a cut-off score of ≥3 for defining depression. Psychometric assessments including Cronbach alpha (for internal consistency of the PHQ-9) and exploratory factor analyses was also performed to assess the PHQ-9's dimensional structure among PWH. Analyses were stratified by the country/clinic and language of interview. All statistical analyses were completed using SAS 9.4 (SAS Institute, Cary, NC).

Results

Participants’ characteristics

Overall 1064 PWH were screened for participation; 422 in Cameroon (January 2018–December 2019); 348 in Congo (January 2019–April 2019), 294 in Kenya (February 2020–June 2021), 148 in Cote d’Ivoire (March 2021–May 2021), and 103 in Senegal (March 2021–July 2022). Of those, 779 had a completed psychiatric evaluation form and were included in the analyses including 297 (38.2%) in Cameroon, 132 (17.0%) in Congo, 148 (19.0%) in Cote d’Ivoire, 98 (12.6%) in Kenya, and 103 (13.2%) in Senegal. The main reason for missing the psychiatric diagnosis was the unavailability of the psychiatrist as only one potential participant screened, declined to take part in the study. The median age was 41 years (range 19–69) in Cameroon, 41 (range 22–74) in Congo, 48 (range 40–56) in Cote d’Ivoire, 36 (range 21–69) in Kenya, and 46 (range 35–55) in Senegal (Table 1). About one-third in Kenya and Congo, and one-half of participants in Cameroon, Cote d’Ivoire and Senegal were male. Respectively, 31.7% (n = 95) in Cameroon, 19.7% (n = 26) in Congo, 41.2% (n = 61) in Cote d’Ivoire, 38.8% (n = 38) in Kenya, and 61%% (n = 62) in Senegal had completed no more than primary education. PHQ-9 was administered to most participants (78.1%, n = 612) in French, including 97.0% (n = 291) of participants in Cameroon, 53.0% (n = 70) in Congo and all participants from Cote D’Ivoire and Senegal. Sixty participants (61.2%) in Kenya and 55 (41.7%) in Congo were interviewed in Swahili and Lingala, respectively.

Table 1.

Socio-demographic and clinical characteristics of 784 PWH who completed a PHQ-9 questionnaire and a psychiatrist assessment in Cameroon, Republic of Congo, Cote d’Ivoire, Kenya, and Senegal.

	Cameroon		Congo		Kenya		Cote d’Ivoire		Senegal		Total
	N	%	N	%	N	%	N	%	N	%	N	%
Age: median (range)	297	41 (19, 69)	132	41 (22, 74)	98	36 (21, 69)	148	48 (21, 84)	103	46 (23, 80)	778	42 (19, 84)
Gender
Male	151	50.8	45	3.1	33	33.7	74	50.0	48	46.6	351	44.9
Female	149	49.2	83	62.9	64	65.3	74	50.0	55	53.4	425	54.4
Missing		0.0	4	3.0	1	1.0					5	0.6
Educational level
Primary	95	31.7	26	19.7	38	38.8	61	41.2	64	62.1	284	36.4
Secondary	145	48.2	67	50.8	38	38.8	54	36.5	30	29.1	334	42.8
Tertiary	32	10.8	32	24.2	17	17.3	33	22.3	9	8.7	123	15.7
Missing	28	9.4	7	5.3	5	5.1					40	5.1
Interview language
English	9	3.0	0	0.0	38	38.8					47	6.0
French	291	97.0	70	53.0	0	0.0	148	100.0	103	100.0	612	78.1
Kituba	0	0.0	6	4.5	0	0.0					6	0.8
Lingala	0	0.0	55	41.7	0	0.0					55	7.0
Swahili	0	0.0	0	0.0	60	61.2					60	7.7

Nine-item Patient Health Questionnaire and two-item Patient Health Questionnaire scores and psychiatric diagnoses

The median PHQ-9 scores were 2 (range 0–21) in Cameroon, 4 (range 0–19) in both the Congo and Kenya, 3 (range 0–22) in Cote d’Ivoire and 2 (range 0–14) in Senegal. The median PHQ-2 scores were 0 in Cameroon (range 0–6) and Senegal (range 0–6), 1 (range 0–6) in Congo and Cote d’Ivoire, and 1.5 (range 0–6) in Kenya, respectively. Overall, 6.0% (n = 18) participants in Cameroon, 11.5% (n = 17) in Cote d’Ivoire, 13.6% (n = 14) in Senegal, 17.4% (n = 23) in Congo, and 34.0% (n = 33) in Kenya were found by a psychiatrist to have MDD (Table 2).

Table 2.

PHQ-9 scores and psychiatric diagnosis.

	Cameroon		Republic of Congo		Kenya		Cote d’Ivoire		Senegal		Total
	N	%	N	%	N	%	N	%	N	%	N	%
PHQ-9 score
Median score (range)	2 (0–21)	4 (0–19)	4 (0–19)	3 (0–22)	2 (0–14)	3 (0–22)
0–4	235	78.3	70	53	54	55.1	87	58.8	82	79.6	528	67.8
5-10	48	16	45	34.1	28	28.6	41	27.7	20	19.4	182	23.1
10-14	10	3.3	12	9.1	13	13.3	13	8.8	1	1.0	49	6.3
15-19	6	2	5	3.8	3	3.1	5	3.4	0	0.0	19	2.4
20-27	1	0.3	0	0	0	0	2	1.4	0	0.0	3	0.4
PHQ-2 score
Median score (range)	0 (0–6)	1 (0–6)	1.5 (0–6)	1 (0–6)	0 (0–5)	1 (0–6)
0–2	263	87.7	99	75	70	71.4	104	70.3	93	90.3	629	80.5
3–6	37	12.3	33	25	28	28.6	44	29.7	10	9.7	152	19.5
Psychiatric diagnosis
Major depressive disorder
Yes	18	6.1	23	17.4	33	33.7	17	11.5	12	11.9	103	13.2
No	278	93.9	109	82.6	65	66.3	131	88.5	89	88.1	672	86.3
Not Specified	3	1	0	0	0	0					3
Missing	1										1
Time frame
Current (over past 2 weeks)	6	35.3	10	45.5	9	27.3	17	100	14	100	56	52.8
Recent (within past year)	7	41.2	0	0	0	0	0	0	0	0	7	6.6
Past (over one year ago)	4	23.5	12	54.5	24	72.7					40	37.7
Missing	1		1		1						3	2.8

Characteristics of nine-item Patient Health Questionnaire and two-item Patient Health Questionnaire against psychiatric diagnosis

Table 3 presents data on sensitivity, specificity, PPV and NPV of the PHQ-9 (scores > 4, 7, 9, 11, 14, 17, and 19), PHQ-2 (score ≥ 3) and the CDQ major and other depression syndrome against psychiatrist's diagnosis. Overall, across countries, the sensitivity of the PHQ-9 for current MDD was low, ranging from 8.0% (95% CI: 0.2, 38) in Senegal, 50.0% (95% CI: 11.8%, 88.2%) in Cameroon 50.0% (95% CI: 18.7%, 81.3%) in Congo, 66.7% (95% CI: 29.9%, 92.5%) in Kenya, to 70.6% (95% CI: 44.0%, 99.0%) in Cote d’Ivoire for PHQ-9 score ≥10. This sensitivity was similar to that of PHQ-2 score ≥3: 25.0% (95% CI: 5.5%, 57.2%) in Senegal, 30.0% (95% CI: 6.7%, 65.3%) in Congo, 50.0% (95% CI: 11.8%, 88.2%) in Cameroon, 55.6% (95% CI: 21.2%, 86.3%) in Kenya, and 64.7% (95% CI: 38.3%, 85.8%) in Cote d’Ivoire. When stratified by the language of interview (Table S1, Supplemental Digital Content), despite the wider confidence intervals, there was more variations with sensitivity as low as 44.7% (95% CI: 28.6%, 61.7%) for interviews in French, 62.5% (95% CI: 24.5%, 91.5%) for Swahili, and 75.0% (95% CI: 19.4, 99.4) for Lingala.

Table 3.

Sensitivity and specificity of the PHQ-9, PHQ-2 and the CDQ major and other depression symdrome against a psychiatric diagnosis (current depression).

	Overall	Cameroon	Congo	Kenya	Cote d’Ivoire	Senegal
Sensitivity
PHQ-9 ≥5	83.3 (71.7, 92.4)	83.3 (35.9, 99.6)	80.0 (44.4, 97.5)	88.89 (51.8, 99.7)	100 (80.5, 100)	64.3 (35.1, 87.2)
PHQ-9 ≥8	62.5 (48.6, 75.1)	50.0 (11.8, 88.2)	70.0 (34.8, 93.3)	66.7 (29.9, 92.5)	88.2 (63.3, 98.5)	28.6 (8.4, 58.1)
PHQ-9 ≥10	48.2 (34.7, 62.0)	50.0 (11.8, 88.2)	50.0 (18.7, 81.3)	66.7 (29.9, 92.5)	70.6 (44.0, 99.0)	7.1 (0.2, 33.9)
PHQ-9 ≥12	33.9 (21.8, 47.8)	33.3 (4.3, 77.7)	50.0 (18.7, 81.3)	55.6 (21.2, 86.3)	35.3 (14.2, 61.7)	7.1 (0.2, 33.9)
PHQ-9 ≥15	17.9 (8.9, 30.4)	33.3 (4.3, 77.7)	30.0 (6.7, 65.3)	22.2 (2.8, 60.0)	17.6 (3.8, 43.4)	–
PHQ-9 ≥18	10.7 (4.0, 21.9)	33.3 (4.3, 77.7)	10.0 (0.3, 44.5)	11.1 (0.3, 48.3)	11.8 (1.5, 36.4)	–
PHQ-9 ≥20	8.9 (3.0, 19.6)	33.3 (4.3, 77.7)	0	11.1 (0.3, 48.3)	11.8 (1.5, 36.4)	–
PHQ-2 ≥3	46.3 (32.6, 60.4)	50.0 (11.8, 88.2)	30.0 (6.7, 65.3)	55.6 (21.2, 86.3)	64.7 (38.3, 85.8)	25.0 (5.5, 57.2)
CDQ Major depression syndromea	16.1 (7.2, 28.3)	16.7 (0.4, 64.1)	30.0 (6.7, 65.3)	22.2 (2.8, 60.0)	11.8 (1.5, 36.4)	8.3 (0.2, 38.5)
CDQ Other depression syndromeb	46.4 (33.0, 60.3)	50.0 (11.8, 88.2)	50.0 (18.7, 81.3)	66.7 (29.9, 92.5)	64.7 (38.3, 85.8)	8.3 (0.2, 38.5)
Specificity
PHQ-9 ≥5	71.6 (68.2, 74.9)	79.7 (74.6, 84.2)	55.7 (46.5, 64.7)	59.6 (48.6, 69.8)	66.4 (57.6, 74.4)	86.5 (77.6, 92.8)
PHQ-9 ≥8	86.3 (83.6, 88.7)	91.8 (88.0, 94.6)	74.6 (65.9, 82.0)	75.3 (65.0, 83.8)	84.7 (77.4, 90.4)	97.8 (92.1, 99.7)
PHQ-9 ≥10	93.9 (91.9, 95.5)	95.2 (92.1, 97.4)	90.2 (83.5, 94.8)	88.8 (80.3, 94.5)	93.9 (88.3, 97.3)	100 (95.9, 100)
PHQ-9 ≥12	96.5 (94.9, 97.8)	96.6 (93.8, 98.7)	94.3 (88.5, 97.7)	95.5 (88.9, 98.8)	97.0 (92.3, 99.2)	100 (95.9, 100)
PHQ-9 ≥15	98.3 (97.1, 99.1)	98.3 (96.0, 99.4)	98.4 (94.2, 99.8)	98.9 (93.9, 100)	97.0 (92.3, 99.2)	100 (95.9, 100)
PHQ-9 ≥18	99.6 (98.8, 99.9)	99.3 (97.5, 99.9)	99.2 (95.5, 100)	100 (95.9, 100)	100 (97.2, 100)	100 (95.9, 100)
PHQ-9 ≥20	99.7 (99.0, 100)	99.7 (98.1, 100)	99.2 (95.5, 100)	100 (95.9, 100)	100 (97.2, 100)	100 (95.9, 100)
PHQ-2 ≥3	84.1 (81.2, 86.7)	88.3 (84.1, 91.8)	75.4 (66.8, 82.8)	74.2 (63.8, 82.9)	80.9 (73.1, 87.3)	96.6 (90.4, 99.3)
CDQ Major depression syndromea	98.3 (97.1, 99.1)	97.6 (95.1, 99.0)	98.4 (94.2, 99.8)	100 (95.9, 100)	97.7 (93.5, 99,5)	100 (95.9, 100)
CDQ Other depression syndromeb	90.9 (88.5, 92.9)	93.5 (90.0, 96.0)	83.6 (75.8, 89.7)	85.4 (76.3, 92.0)	89.3 (82.7, 94.0)	100 (95.9, 100)

^aIf an answer to Q1 or Q2 ≥2 and five or more answers to any Q3–Q9 ≥2.

^bIf answer to Q1 or Q2 ≥2 and 2+ of answers to any Q3–Q9 ≥2.

The PPVs was also relatively low ranging from 17.7% (3.8%, 43.3%) in Cameroon, to 29.4% (95% CI: 10.3%, 56.0%) in Congo, 37.5% (95% CI: 15.2%, 64.6%) in Kenya, and 60.0% (95% CI: 36.1%, 80.9%) in Cote d’Ivoire, for a score ≥10. On the other hand, the NPVs were generally quite high (Table 4). The probability of not having a current MDD when one was screened negative based on the PHQ-9 score ≥10 was 98.9% (95% CI: 96.9%, 99.8%) in Cameroon, 95.7% (95% CI: 90.2%, 98.6%) in Congo, 96.3% (95% CI: 89.7%, 99.2%) in Kenya, and 96.1% (95% CI: 91.1%, 98.7%) in Cote d’Ivoire. PHQ-2 with cut-off at ≥3 also had similar high NPV (Table 4). Using the CDQ definition of MDD or other depression produces lower or similar performance against the goal standard.

Table 4.

Positive predictive value and Negative Predictive value of the PHQ-9, PHQ-2 and the CDQ major and other depression symdrome against a psychiatric diagnosis (current depression).

	Overall	Cameroon	Congo	Kenya	Cote d’Ivoire	Senegal
Positive predictive values
PHQ-9 ≥5	18.0 (13.4, 23.3)	7.8 (2.6, 17.3)	12.9 (5.7, 23.9)	18.2 (8.2, 32.7)	27.9 (17.2, 40.9)	36.8 (16.3, 61.6)
PHQ-9 ≥8	26.1 (18.9, 34.4)	11.1 (2.4, 29.2)	18.4 (7.7, 34.3)	21.4 (8.3, 41.0)	42.9 (26.3, 60.7)	66.7 (22.3, 95.7)
PHQ-9 ≥10	38.0 (26.8, 50.3)	17.7 (3.8, 43.3)	29.4 (10.3, 56.0)	37.5 (15.2, 64.6)	60.0 (36.1, 80.9)	100 (2.5, 100)
PHQ-9 ≥12	43.2 (28.4, 59.0)	16.7 (2.1, 48.4)	41.7 (15.2, 72.3)	55.6 (21.2, 86.3)	60.0 (26.2, 87.8)	100 (2.5, 100)
PHQ-9 ≥15	45.5 (24.4, 67.8)	28.6 (3.7, 71.0)	60.0 (14.7, 94.7)	66.7 (9.4, 99.2)	42.9 (9.9, 81.6)
PHQ-9 ≥18	66.7 (29.9, 92.5)	50.0 (6.8, 93.2)	50.0 (1.3, 98.7)	100 (2.5, 100)	100 (15.8, 100)
PHQ-9 ≥20	71.4 (29.0, 96.3)	66.7 (9.4, 99.2)	100 (2.5, 100)	100 (2.5, 100)	100 (15.8, 100)
PHQ-2 ≥3	17.9 (11.9, 25.2)	8.1 (1.7, 21.9)	9.1 (1.9, 24.3)	17.9 (6.1, 36.9)	30.6 (16.4, 48.1)	50.0 (11.8, 88.2)
CDQ Major depression syndromea	42.9 (21.8, 66.0)	12.5 (0.3, 52.7)	60.0 (14.7, 94.7)	100 (15.8, 100)	40.0 (5.3, 85.3)	100 (2.5, 100)
CDQ Other depression syndromeb	28.3 (19.4, 38.6)	13.6 (2.9, 34.9)	20.0 (6.8, 40.7)	31.6 (12.6, 56.6)	44.0 (24.4, 65.1)	100 (2.5, 100)
Negative predictive values
PHQ-9 ≥5	98.3 (96.8, 99.2)	99.6 (97.6, 100)	97.1 (90.1, 99.7)	98.2 (90.1, 99.9)	100 (95.9, 100)	93.9 (86.3, 98.0)
PHQ-9 ≥8	97.0 (95.4, 98.2)	98.9 (96.8, 99.8)	96.8 (91.0, 99.3)	95.7 (88.0, 99.1)	98.2 (93.8, 99.8)	91.6 (84.1, 96.3)
PHQ-9 ≥10	96.2 (94.5, 97.5)	98.9 (96.9, 99.8)	95.7 (90.2, 98.6)	96.3 (89.7, 99.2)	96.1 (91.1, 98.7)	89.0 (81.2, 94.4)
PHQ-9 ≥12	95.2 (93.4, 96.7)	98.6 (96.5, 99.6)	95.8 (90.5, 98.6)	95.5 (88.9, 98.8)	92.0 (86.2, 96.0)	89.0 (81.2, 94.4)
PHQ-9 ≥15	94.2 (92.2, 95.7)	98.6 (96.5, 99.6)	94.5 (89.0, 97.8)	92.6 (85.4, 97.0)	90.1 (83.9, 94.5)	88.1 (80.2, 93.7)
PHQ-9 ≥18	93.7 (91.8, 95.4)	98.6 (96.5, 99.6)	93.1 (87.3, 96.8)	91.8 (84.4, 96.4)	89.7 (83.6, 94.1)	88.1 (80.2, 93.7)
PHQ-9 ≥20	93.6 (91.7, 95.3)	98.6 (96.6, 99.6)	92.4 (86.4, 96.3)	91.8 (84.4, 96.4)	89.7 (83.6, 94.1)	88.1 (80.2, 93.7)
PHQ-2 ≥3	95.4 (93.5, 96.9)	98.9 (96.7, 99.8)	92.9 (86.0, 97.1)	94.3 (86.0, 98.4)	94.6 (88.7, 98.0)	90.5 (82.8, 95.6)
CDQ Major depression syndromea	94.0 (92.1, 95.6)	98.3 (96.0, 99.4)	94.5 (89.0, 97.8)	92.7 (85.6, 97.0)	89.5 (83.3, 94.0)	89.0 (81.2, 94.4)
CDQ Other depression syndromeb	95.9 (94.1, 97.3)	98.9 (96.9, 99.8)	95.3 (89.4, 98.5)	96.2 (89.3, 99.2)	95.1 (89.7, 98.2)	89.0 (81.2, 94.4)

^aIf an answer to Q1 or Q2 ≥2 and five or more answers to any Q3–Q9 ≥2.

^bIf answer to Q1 or Q2 ≥2 and 2+ of answers to any Q3–Q9 ≥2.

The positive likelihood ratios (LR+) range from 5.08 (95% CI: 0.91, 9.25) in Congo and 5.93 (95% CI: 1.52, 10.35) in Kenya to 10.40 (95% CI: 0.53, 20.26) in Cameroon and 11.56 (95% CI: 3.03, 20.09) in Cote d’Ivoire for a PHQ-9 score ≥10. The LR+ for PHQ-2 range from 1.22 (95% CI: 0.00, 2.44) in Congo and 2.15 (95% CI: 0.68, 3.62) in Kenya, 3.39 (95% CI: 1.70, 5.08) in Cote d’Ivoire, to 4.28 (95% CI: 0.60, 7.96) in Cameroon and 7.42 (95% CI: 0.00, 18.41) for a score ≥3 (Table 5).

Table 5.

Positive and negative likelihood ratios for PHQ-9 and PHQ-2 against a psychiatric diagnosis (current depression).

	Cameroon		Congo		Kenya		Cote d’Ivoire		Senegal		Overall
	LR+	LR−	LR+	LR−	LR+	LR−	LR+	LR−	LR+	LR−	LR+	LR−
PHQ-9 ≥5	4.11 (2.37, 5.85)	0.21 (0.00, 0.58)	1.81 (1.14, 2.47)	0.36 (0.00, 0.81)	2.20 (1.45, 2.95)	0.19 (0.00, 0.53)	2.97	0	4.33 (1.25, 7.40)	0.48 (0.16, 0.81)	2.93 (2.45, 3.42)	0.23 (0.09, 0.37)
PHQ-9 ≥8	6.06 (0.68, 11.44)	0.54 (0.11, 0.98)	2.75 (1.36, 4.15)	0.40 (0.02, 0.79)	2.70 (1.11, 4.28)	0.44 (0.03, 0.86)	5.78 (3.24, 8.32)	0.14 (0.00, 0.32)	14.83 (0.00, 38.37)	0.68 (0.41, 0.91)	4.73 (3.46, 6.00)	0.41 (0.26, 0.56)
PHQ-9 ≥10	10.40 (0.53, 20.26)	0.53 (0.10, 0.95)	5.08 (0.91, 9.25)	0.55 (0.21, 0.90)	5.93 (1.52, 10.35)	0.38 (0.03, 0.72)	11.56 (3.03, 20.09)	0.31 (0.08, 0.54)			8.20 (4.99, 11.42)	0.53 (0.39, 0.67)
PHQ-9 ≥12	9.70 (0.00, 22.17)	0.69 (0.30, 1.08)	8.71 (0.44, 16.98)	0.53 (0.20, 0.86)	12.36 (0.00, 26.23)	0.47 (0.12, 0.81)	11.56 (0.00, 24.97)	0.67 (0.43, 0.90)			10.16 (4.79, 15.53)	0.67 (0.54, 0.80)
PHQ-9 ≥15	19.40 (0.00, 47.08)	0.68 (0.29, 1.06)	18.30 (0.00, 48.84)	0.71 (0.42, 1.00)	19.78 (0.00, 65.28)	0.79 (0.51, 1.06)	5.78 (0.00, 13.92)	0.85 (0.66, 1.04)			11.14 (2.31, 19.97)	0.83 (0.72, 0.93)
PHQ-9 ≥18	48.5 (0.00, 135.10)	0.67 (0.29, 1.05)	12.20 (0.00, 45.09)	0.91 (0.71, 1.10)	∞	0.91 (0.72, 1.10)		0.86 (0.64, 1.09)			26.74 (0.00, 63.06)	0.89 (0.81, 0.98)
PHQ-9 ≥20	97.0 (0.00, 316.24)	0.67 (0.29, 1.05)	0	1.01 (0.99, 1.02)	∞	–		0.86 (0.64, 1.09)			33.43 (0.00, 87.45)	0.91 (0.83, 0.99)
PHQ-2 ≥3	4.28 (0.60, 7.96)	0.57 (0.11, 1.02)	1.22 (0.00, 2.44)	0.93 (0.54, 1.32)	2.15 (0.68, 3.62)	0.60 (0.16, 1.04)	3.39 (1.70, 5.08)	0.44 (0.15, 0.72)	7.42 (0.00, 18.41)	0.78 (0.52, 1.03)	2.91 (1.94, 3.87)	0.64 (0.48, 0.80)

LR−, negative likelihood ratio; LR+, positive likelihood ration; MDD, major depression disorder as diagnosis by a psychiatrist.

When the analysis was stratified by the language of interview (Table S1), the sensitivity of the PHQ-9 score ≥10 for the diagnosis of any MDD was lowest for French 44.7% (95% CI: 28.6%, 61.7%), follow by and Swahili 62.5% (95% CI: 24.5%, 91.5%), and Lingala 75.0% (95% CI: 19.4%, 99.4%).

The AUC did not vary substantially by country or language (Table S2, Supplemental Digital Content). The AUC for the continuous PHQ-9 score varied from 0.935 (95% CI: 00.893, 0.977) in Cote d’Ivoire, 0.888 (95% CI: 0.784, 0.992) in Cameroon, 0.866 (95% CI: 0.775, 0.958) in Senegal, 0.802 (95% CI: 0.603, 1.00) in Kenya, to 0.768 (95% CI: 0.589, 0.947) in Congo or from 0.875 (95% CI: 0.829, 0.921) for French, 0.855 (95% CI: 0.570, 1.000) for Lingala, to 0.759 (95% CI: 0.541, 0.978) for Swahili.

Psychometric analyses of nine-item Patient Health Questionnaire for HIV patients

Overall, the internal consistency of PHQ-9 items was acceptable with Cronbach's alpha varying from 0.80 in Cameroon and Kenya, 0.78 in Senegal, 0.72 in Cote d’Ivoire, to 0.69 in Congo. Except for Lingala in Congo, the language of the interview did not appear to influence the internal consistency of the PHQ-9 with Cronbach's alpha varying from 0.81 for English, to 0.76 for French, 0.79 for Swahili, and 0.66 for Lingala (Table S2, Supplemental Digital Content).

In exploratory principal component factor analysis, in the overall combined sample or in stratified sample by country, only one factor had Eigenvalue ≥1. However, in the analysis stratified by language, two factors meet the same criteria for interviews conducted in Swahili. Eigenvalue for factor 2 in the English sample was 0.82, 0.63 in the Lingala group, and only 0.34 among those interviewed in French (Table S3, Supplemental Digital Content).

Among those interviewed in French, all items have a Factor 1 loading between 0.46 and 0.56. Across all four languages, Items 2 and 6 loaded consistently high and item 5 consistently low on Factor 1. The loading of the remaining six items depended on the language of the interview. For those interviewed in Lingala, Item 1 (“Interest”), Items 3 (“Sleep”), 4 (“Tired”), and 7 (“Concentration”) had factor loading < 0.40 while items 9, 6, and 2, respectively, had the first (0.69), second (0.64), and third (0.63) highest loading on Factor 1 (Table 4). Among those interviewed in Swahili item 8 (“Slowdown”) was the only item with factor 1 loading < 0.40. Item 7 (“Concentration”), 2 (“depressed”), and 4 (“Tired”) have factor 1 loading of 0.83, 0.73, and 0.61, respectively. For those interviewed in English only two items have factor loading < 0.50: Items 7 (“Concentration”); 0.25 and 5; 0.39 (Table S3, Supplemental Digital Content).

Discussion

The PHQ-9 has been proposed as a screening tool for MDD and is being used as such to support integration of depression management into HIV care in sub-Saharan Africa [23,24]. In this multicountry study, we found the sensitivity and PPV of PHQ-9 were generally low when compared to a diagnosis assigned by a psychiatrist, irrespective of the cut-off considered. We also found that, regardless of the country or the language of evaluation, the NPV was very high, confirming the usefulness of PHQ-9 as a good tool to screen for the absence of depression symptoms, particularly in setting of relatively low prevalence.

Our findings of low sensitivity and PPV of the PHQ-9 across cultural contexts of Central, East-Africa, and West Africa are consistent with the existing literature on the use of PHQ-9 from the region and beyond [28]. The low PPV even in context of high depression, is a call for caution for those temped to use the PHQ-9 as a diagnostic tool and more critically, to suggest that patients who screen positive be initiated on pharmacological treatment without additional evaluations to confirm diagnosis of MDD [20].

The performance of the PHQ-9 varied across the five countries in our study with lower sensitivity observed in Cameroon and the highest in Cote d’Ivoire. This is consistent with what has been reported in previous studies as discussed in the introduction. However, the magnitude of the variation is substantially smaller in our single protocol study, suggesting that at least part the previously observed variations may stem from protocol differences, rather than from underlining cultural differences. Originally, we had planned in this study to use MINI to standardize depression diagnosis across setting, unfortunately MINI is proprietary and the author and copyright order did not give us the permission. We also noticed differences based on the language used in the administration of the interviews, but the limited sample size for English interviews, did not allow us to explore this difference deeper. However, such variation coupled with the differential loading of item by language interview, suggest a difficulty in understanding of at least some question in some language. But whether this is due to a translation or cognition problem needs to be explored.

There is growing evidence supporting that the PHQ-9 as a two-factor instrument [29,30]. Yet in our exploratory factor analysis, the second eigenvalue was > 0.5 only in Kenya. When analysis was performed by language, the eigenvalue of the second factor was >1 for Swahili (used only in Kenya together with English), and >0.5 for Lingala (used only in Congo). This suggests that at least, there are some difficulties as related to comprehension of questions depending on how fluent in the language of interview one is and may explain the low performance of PHQ-9.

The strengths of this study include the use of a common protocol across all five participating sites and the availability of a diagnosis by a psychiatrist. However, the study also had important limitations: (i) the PHQ-9 was administered by interviewers due to some participants’ low literacy; the questionnaire was originally designed to be self-administered. (ii) The inability to enroll 300 participants as planned at all sites due to COVID-19 pandemic. A larger sample in Congo and in Kenya would have allowed us to deepen the analysis by languages, particularly the comparison of Kituba and Lingala in Congo or Swahili and English in Kenya. In addition, because of the smaller than planned sample size, our estimates have wide confidence intervals, weakening the strength of our conclusion. (iii) The study was conducted in urban settings with over two-thirds of participants having completed at least elementary school. (iv) Performance of the PHQ-9 in a rural, less educated sample of participants is likely to differ. (v) In addition to COVID interruption, the limited availability of psychiatrist at each site means some patients who consent to be part of the study were not evaluated (except in Cote d’Ivoire and Senegal), further limiting the available sample size. (vi) We had planned to perform in-depth interviews in each country to better assess the source of differences in PHQ-9 across countries, however, we were able to complete this only in Cameroon due to COVID interruption.

In conclusion, PHQ-9 performed very poorly (low sensitivity and PPV) as a screening tool for depression. However, the observed high NPV suggests that PHQ-9 can be used to identify PWH who should be referred for further evaluation because we cannot rule out depression.

Acknowledgements

This study was funded via an administrative supplement to the Central Africa International epidemiology Database to Evaluate AIDS (IeDEA) grant from National Institute of health (NIAID U01AI096299-08S1). The Redcap Database was supported by a grant from the National Center for Advancing Translational Sciences (UL1TR001070). Moi Teaching and Referral Hospital is a participating site of the East Africa IeDEA also supported by the National Institute of Allergy and Infectious Diseases of the National Institutes of Health under Award Number U01AI069911. Medical center for blood donor follow-up (CNTS-CI) in Cote d’Ivoire, and the Department of Infectious and Tropical Diseases (CHNU Fann) in Senegal are participating sites of the West Africa IeDEA which are also supported by the National Institute of Allergy and Infectious Diseases of the National Institutes of Health under Award Number U01AI069919

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

We are extremely grateful to study participants for their time and willingness to participate in research. We are also grateful to the staff of the Jamot Hospital in Yaoundé, Centre Ambulatoire de Traitement of Bazzaville, Moi Teaching and Referral Hospital in Eldoret, Medical center for blood donor follow-up (CNTS-CI) in Abidjan and Department of Infectious and Tropical Diseases, CHNU Fann in Dakar and the leadership of IeDEA in each participating country for facilitating the study. Many thanks to Dr Paul De Boeck, at the Ohio State University and to Dr Pim Brouwers at NIMH for their feedbacks on the design and analysis of this study.

Funding: NIAID U01AI096299-08S1, administrative supplement; NIAID U01AI069919, U01AI069911 National Center for Advancing Translational Sciences, UL1TR001070. Einstein-Rockefeller-CUNY Center for AIDS Research (P30-AI124414).

Conflicts of interest

There are no conflicts of interest.

Supplementary Material

aids-38-1765-s001.doc ^{(111.5KB, doc)}