One size does not fit all: Psychometric properties of the Shona Symptom Questionnaire (SSQ) and symptomology among adolescents and young adults in Zimbabwe

Abstract

Poor mental health among is a major contributor to the burden of disease among adolescents. For this paper we use cross sectional survey data among a sample of 2,768 adolescent (aged 15-19) and 2,027 young adults (aged 20-24) living in Manicaland Province, Zimbabwe to calibrate the Shona Symptom Questionnaire (SSQ) against the Self Report Questionnaire (SRQ-20) and examined the performance indices of the SSQ based on various cut points for classification. The SSQ depression screening tool performed best with a cut point of five or more positive responses out of 14 questions, resulting in the following validation coefficients for adolescents and young adults respectively: AUC (0.83, 088); kappa statistic (0.64, 0.66); sensitivity (0.89, 0.95); specificity (0.94, 0.92); PPV (0.45, 0.55); NVP (0.99, 1.00). The modified SSQ cut point of five or more substantially increase the depression estimates for both age groups to align more closely with the SRQ-20 estimates. The prevalence of depression increased from 3.5% to 13.2% among adolescents and from 5.1% to 16.2% among young adults based on these revisions to the SSQ. Using a multivariate logistic regression model we isolated particular characteristics to test their association with the odds of being misclassified as non-depressed based on the conventional SSQ cut point. Findings suggest that adolescents who were orphaned (OR 1.48) or ever had sex (2.13) were at a significantly greater odds of being a false negative than their counterparts. Secondary education was significantly associated with false negative misclassification among young adults (OR 2.11). When retested using the modified cut point of five or greater, associations with misclassification disappeared. This study highlight that not all depression scales are appropriate for use among adolescents given their unique developmental stage. While using culturally-appropriate scales such as the SSQ is important, we strongly recommend modification to the cut point in order to improve adolescent depression detection. Alternatively we recommend continued use of the standardized cross-cultural SRQ given it continued success at classify cases of depression across age groups.

Introduction

Though largely overlooked, depression is the leading cause of disability worldwide with an estimated 350 million people affected (1,2). Depression is a central measure of one’s psychosocial well-being and is associated with a litany of poor health outcomes across the life span such as suicide, somatic symptoms, and behavioral problems (3). In 2010 depression accounted for over eight percent of global years of life lived with a disability (YLDs) (3).

Most mental disorders have their onset during the adolescent period (4). Depression rates have been estimated at 4-8% among adolescents with substantial increases over time (5). Some estimates suggest that early–onset depression has a 60–70% risk of continuing into adulthood (6). Adolescents suffering from depression are at greater risk for self-harm, conduct disorders, delinquency, and high-risk behaviors such as substance use and early sexual debut (7–9).

This is particularly relevant in Zimbabwe, where the burden of disease due to depression is likely to be severe given the catastrophic economic and political situation in the country, coupled with the weakened healthcare infrastructure, chronic and widespread poverty, and reverberating effects of the HIV epidemic. Adolescents living in HIV-affected communities are often faced with multiple experiences of loss and hardship such as the death of parent, family dissolution or migration, and caring for the sick (10–13). In addition to the contextual and social effects of HIV, the threat of becoming infected can be a significant source of anxiety, depression, and feelings of hopelessness among adolescents and young adults who are becoming infected with the disease at faster rates than any other age group (1,14). Long-term stress accumulation coupled with their unique developmental stage puts adolescents at elevated risk for affective disorders such as depression.

Depression in developing countries has received little international attention despite evidence that suggests the epidemiological patterns of mental illness are comparable worldwide (15–17). A majority of the world’s population lives in resource-poor countries and yet mental health research has been concentrated elsewhere. The dearth of mental health specialists in resource-poor countries like Zimbabwe make obtaining treatment for depression a major obstacle for most people. This can be particularly challenging for adolescents who may be less confident in their ability to navigate the health system. While there is growing evidence that community lay health providers can begin to fill this treatment gap, more needs to be done to strengthen mental health services throughout much of sub-Saharan Africa (SSA) (18–21).

Having a simple, validated screening tool is the first step to strengthening community-level mental health services in SSA (19–22). Studies in rural areas have shown that a good instrument does a far better job in identifying and correctly classifying cases of depression than general practitioners who often lack training in mental health (23). A good depression screening instrument utilized by lay health advisors can help identify depressed individuals, monitor symptom changes over time, and support overburdened, ineffective health systems by serving as the first point of contact. Continued efforts are needed in Zimbabwe to raise awareness about the symptoms of depression, those who at risk for early onset, and sources of care in the community. By understanding the appropriateness of depression screening tools among different age groups and settings, mental health advocates will have a stronger platform to encourage both the integration of mental services into primary healthcare as well as the training of providers in the symptomology, diagnosis, and treatment of affective disorders.

With the development of multiple depression screening tools in recent decades, it is becoming increasingly important to select a measure that is reliable, valid, and appropriate for the targeted age group and their context. Scales that may be appropriate in settings of the developed world may be less suitable in resource-poor countries. Diagnostic labels of depression and other mental health disorders anchored in Western settings may not always be easily conveyed in other cultures (21). Studies of depression in non-western settings should carefully consider the impact of emic understandings of health and illness on the validity of the scales in order to ensure cross-cultural equivalence. Similarly, symptom measures based on individual questions within a depression screening scale developed for adults may be less useful among adolescents who have different behaviors and internalized symptoms due to their developmental stage.

The two depression screening tools of interest for this paper include the Self-Reported Questionnaire (SRQ-20) and the Shona Symptom Questionnaire (SSQ). The SRQ-20 has been translated into several languages and validated in many different cultural settings and age groups including but not limited to regions of SSA (24–37). The SSQ is a more recently developed tool intended for Shona-speaking countries including Zimbabwe, Botswana, and Mozambique. Much remains unknown about the validity of the SSQ, particularly among adolescent populations.

To date only one study has compared the SSQ to the SRQ-20. This was done among a sample of adults living in Wales with the goal of measuring if emic and etic depression screening instruments function differently among the same population (30). The authors found that both scales functioned well among their study sample with less than a 10% misclassification rate. However, the authors lowered the SSQ cut point for classification of depression in order to achieve optimization for the sample. Based on the lack of data validating the SSQ, this paper will be the first to explore how the SSQ functions among a sample of Zimbabwean adolescents and young adults using the SRQ-20 as the gold standard criterion. This analysis will provide evidence to help mental health researchers determine the strengths and drawbacks of these two scales for use both in SSA countries as well as among adolescents and young adults. Closer examination of these depression screening instruments provides a platform for researchers to scrutinize the nuances of various scales as they pertain to their study setting and population.

The following sections provide a more complete overview of both scales and their use in various populations.

Self-Reported Questionnaire (SRQ-20)

The Self-Reported Questionnaire (SRQ) was developed in the 1970’s by an expert panel within the World Health Organization (WHO) to detect depression in developing countries (38,39). It was partially derived from the Present State Examination (PSE) – a 140-question tool used by British clinicians for diagnosis of mental disorders (30). The original format of the SRQ included 25 yes-no questions asking whether individuals experienced specific mental distress symptoms over the previous week. Two questions were reverse coded (“Were you able to play a useful part in life?” and “Did you feel able to cope with most of the problems in your life?”).

The 25-item SRQ questionnaire is rarely used in its entirety and typically limited to a shorter subset of 20 questions shown to have high content validity with neurosis, depression, anxiety, and psychosomatic complaints (38,39). Although initially intended for primary health care settings, the SRQ-20 has also been used as a screening tool within community settings. Studies have used various cut points based on the discriminate ability of the SRQ-20 to capture cases and non-cases of depression within their sample based on a clinical criterion tool (28,31,36,39). The most common cut point used to determine depression, and one recommended by the WHO, is a positive response to eight or more of the twenty questions (38). It is recommended that this screening scale be used in tandem with more formalized clinical diagnosis by physicians.

Criterion validity of the SRQ-20 has been established in various studies across continents and assessed against gold-standard diagnostic tools including the Clinical Interview Schedule - Revised (CIS-R), the Diagnostic Interviewer Schedule (DIS), and the Diagnostic and Statistical Manual of Mental Disorders 4^th edition (DSM-IV) (28,35,36). This suggests that the SRQ-20, while not specifically designed for one country, is culturally adaptable, easy to use and administer, and is consistently valid in measuring psychiatric symptoms across various populations (24–37). Based on our confidence in the SRQ-20 and its validation across several studies we will use it as the gold standard criterion for assessing the functionality of the SSQ among adolescents and young adults.

Shona Symptom Questionnaire (SSQ)

The Shona Symptom Questionnaire (SSQ) is a more recent depression screening tool intended for Zimbabwe and neighboring regions of SSA (40). Validated among a sample of adults by nurses and traditional medical practitioners, the SSQ was developed as a shorter, culturally-relevant tool for this region. The tool consists of 14 yes-no questions, nine of which overlap with the SRQ-20. The tool adopted indigenous idioms used by patients for the five culturally-specific measures not captured by the SRQ-20. (See Table 5.)

Table 5. Comparison of depression classification using the SRQ-20 and SSQ scales.

WHO SRQ-20
Shona SSQ	Total sample of 15-24 year olds
		Depressed     n(%)	Not Depressed        n(%)	Total
	Depressed	199 (4.2)	10 (0.2)	209 (4.4)
	Not Depressed	213 (4.4)	4373 (91.2)	4586 (95.6)
	Total	412 (8.6)	4383 (91.4)	4795
	Subsample of 15-19 year olds
		Depressed     n(%)	Not Depressed       n(%)	Total  n (%)
	Depressed	98 (3.5)	8 (0.3)	106 (3.8)
	Not Depressed	126 (4.6)	2536 (91.7)	2662 (96.2)
	Total	224 (8.1)	2544 (91.9)	2768
	Subsample of 20-24 year olds
		Depressed N (%)	Not Depressed N (%)	Total n (%)
	Depressed	101 (5.0)	2 (0.1)	103 (5.1)
	Not Depressed	87 (4.3)	1837 (90.6)	1924 (94.9)
	Total	188 (9.3)	1924 (90.7)	2027

^*Note: For both scales, depression in classified as through the standard cut point of eight or more positive response.

The SSQ was originally validated against two criteria – a clinical diagnosis based on the CIS and judgment by a clinical care provider. Based on the classification of cases, the SSQ was analyzed using a receiver-operating curve (ROC) for an optimal cut point of 8 or more positive responses out of fourteen questions. This cut point yielded a sensitivity of 67%, specificity of 83%, and misclassification rate of 22% among a sample of 302 patients aged 16-65 (mean age 30.9) (40).

To date only two studies have used the SSQ among adolescents exclusively (41,42). Langhaug et al. used the SSQ to assess the prevalence of affective disorders among 1495 adolescents 15-23 years old in rural Zimbabwe validated against the CIS-R with a score of 12 or more. The questionnaire was altered to include “always”, “sometimes” and “never” as response options rather than yes/no. The authors classified individuals as depressed for those who gave affirmative response of always or sometimes to eight out of 14 questions. This alteration to the questionnaire inflated the summative individual scores and depression prevalence (52%) among the sample. The authors’ use of categorical versus dichotomous response options limits our ability to make comparisons with our own study sample. In a separate study, Mayhu et al. reported a depression prevalence of 63% among a sample of 229 HIV positive youth 6-18 years old living in Harare, Zimbabwe based on the SSQ cut point of eight or more. No gold standard was used to validate these findings. This prevalence should be interpreted with caution given the unique challenges and psycho-social needs among this group, which are not generalizable to HIV negative adolescents.

Focus of this Study

As mental health research in SSA grows, it is imperative that researchers and mental health professionals select high-functioning, validated scales based on the specific population of interest. While there is value in having culturally-relevant scales such as the SSQ that measure mental distress in SSA settings, it is prudent for researchers and programmers to think critically about how well these novel tools function for various ages. While more concise questionnaires such as the SSQ may reduce response fatigue among participants, they may fail to capture the necessary measures important in distinguishing those who are depressed from those who are not. Using data from Zimbabwe, this paper contributes to the literature by 1.) assessing the functionality of the SSQ in comparison with the SRQ-20 among adolescents and young adults in Zimbabwe, and 2.) highlighting individual characteristics associated with depression misclassification based on the SSQ scale with the SRQ-20 serving as the gold standard.

Methods

Setting population and data collection

The data source for this study was the Manicaland HIV Prevention Project, which began in 1998 as a population-based survey of adults (15-54 years) investigating the dynamics of HIV transmission across socio-economic groups (43). This study used cross sectional data collected from October 2009 to July 2011 consisting of a survey of 15-64 year olds (n=14,464). The sample for the present study was limited to adolescents ages 15-19 and young adults ages 20-24 with complete survey data. There was a 90.6% response rate for all items in both depression questionnaires among adolescents and young adults, resulting in a final sample of 4,795 participants.

The survey was translated into the local language (Shona) prior to administering interviews and included questions regarding psychosocial health, socio-demographic characteristics, orphan status, sexual behaviors, and migration. Both depression scales were designed to be self-administered, however due to the broader survey goals of the Manicaland HIV Prevention Project, interviewers administered both tools verbally as part of the entire individual survey.

Scales were not asked separately but rather overlapping items were only asked once to minimize redundancy and interview time. In the case of the nine overlapping questions, questionnaire administrators utilized SRQ-20 phrasing. The phrasing differences were minor, albeit noteworthy. For example, the SRQ-20 asked, “Do you cry more than usual?”, while the SSQ asked, “Were there moments when you felt life was so tough that you cried or wanted to cry?”. The final composite questionnaire contained 25 unique questions that maintained the chronological order of the SRQ-20 intermixed with items from the SSQ that were inserted non-chronologically throughout the assessment.

Statistical Analysis

Objective 1: We assessed the correlation between the two scales, stratified by age groups (15-19 years old; 20-24 years old) in order to determine how strongly the scales as a whole related to one another. Next, we calibrated the SSQ against the SRQ-20 and examined the performance indices of the SSQ based on various cut points for classification using the SRQ-20 with a cut point of eight or more as the gold standard criterion measure. Performance indices of interest included sensitivity, specificity, positive predictive value, negative predictive value, percent of cases overall screened correctly, kappa statistic, SSQ prevalence based on a particular cut point value, and area under the curve (AUC) obtained through a receiver operator characteristics (ROC) curve. Performance indices were assessed for both adolescents and young adult and were stratified by gender between both groups.

Objective 2: Using a multivariate logistic regression analyses we isolated particular characteristics to test their association with the odds of being misclassified as non-depressed by the SSQ using the recommended cut point of eight or greater and a modified cut point of five or greater despite being classified as depressed based on the SRQ-20 gold standard criterion.

Although previous studies worldwide vary greatly in their methodology, sufficient convergence has emerged to implicate several sociodemographic variables captured in this data set as potential risk factors for misclassification of depression (17,28,31,36,41,42,44–48). Studies have shown that gender, education and wealth levels, orphan status and age can have varying effect on depression misclassification (28,49–52). Social roles and cultural norms have been shown to affect the reporting of depression symptoms among different groups (men vs. women, orphans vs. non-orphans, married vs. non-married) (51,53,54). Furthermore, scales may be more or less meaningful for different groups depending on their position in society and past experiences. For example an individual who has lived through multiple hardship may perceive depression as a normal experience and consequently underreports the severity of their symptomology. Premarital sex, taboo in Zimbabwean culture, may result in underreporting of depression symptoms in an attempt to mask feelings of guilt for acting outside of one’s cultural expectations. Lastly, recent migration is likely associated with short-term grief due to changing circumstances, and could be misclassified as depression - a longer-term derivative of grief.

Denial of depression symptoms may be another manifestation among any of these characterized groups (i.e., the poor, married adolescents, orphans, adolescents who moved as a result of external events). Denial is a common response among individuals who feel they lack control over their circumstances. While denial is often a stage of both depression and grief, it is hard to capture as a symptomology and may therefore exacerbate misclassification rates (55).

Based on supporting literature and these linkages the key independent variables of interest were orphan status (non-orphan, maternal orphan, paternal orphan, and double orphan), gender (male/female), age (15-24), marital status (yes/no), ever had sex (yes/no), recent migration (yes/no), school enrollment (yes/no), level of education (primary school or less/secondary school or beyond), and wealth (poorest, poor, middle, wealthy, wealthiest). Table 1 provides the derivation of all key variables.

Table 1. Variable derivation.

Variables	Survey Questions	Coding
Orphan status	type of orphan determined by the following two questions: •    Is your natural biological father still alive? •    Is your natural biological mother still alive? “Don’t know” responses were counted as dead	non-orphan; paternal orphan; maternal orphan; double orphan
Gender	Sex	male, female
Age	Age at most recent birthday	15;16;17;18;19;20;21;22;23;24
Marital status	Reporting being married, in a long-term, or cohabitating relationship at the time of survey	no; yes
Ever had sex	Has begun sexual activity at time of survey	no; yes
Recent migration	moved to a homestead since the prior survey (< two years ago)	no, yes
School enrollment	enrolled in school full-time	no, yes
Education level	highest level of schooling completed	<=primary, >=secondary
Wealth quintile	households were divided into wealth quintiles based on summed score of household asset ownership based on the following: source of drinking water (piped into residence, private tap in yard or plot, communal tap, private well or borehole, other well or borehole, protected spring, other); access to electricity (yes or no); type of toilet facility (flush, blair, pit latrine, other, none); type of house (pole and dagga structure, brick house with thatched roof, brick house with tiled/sheeting roof, cabin/other); type of floor in the main dwelling (natural floor (earth/sand/dung); rudimentary (planks/palm/bamboo), finished (wood/cement/carpet)); ownership of a radio, a television, a motorbike or a car.	poorest, poor, middle, wealthy, wealthiest

Ethical Approval

IRB approval for this secondary data analysis was obtained from the University of North Carolina at Chapel Hill.

Results

Sample Characteristics

Table 2 present the summary statistics for the SRQ-20 and the SSQ depression scales, stratified by age groups (15-19 year olds and 20-24 year olds). The SRQ-20 had a mean of 2.59 and a median of 2 positive responses out of 20 questions for the full sample (mean: adolescents 2.48; young adults 2.74; median: adolescents 1; young adults 2). The SSQ had a mean of 1.90 and a median of 1 positive responses out of 20 questions for the full sample (mean: adolescents 1.76; young adults 2.07; median: adolescents 1; young adults 1). The Crohnbach’s alpha coefficient was 0.79 or higher for both the SRQ-20 and SSQ indicating good internal consistency.

Table 2. Summary statistics for the SRQ-20 and SSQ depression-scales among adolescents (15-19 year olds) and young adults (20-24 year olds).

	Adolescents ages 15-19 years (n=2,768)					Young adults ages 20-24 years (n=2,027)					Total Sample ages 15-24 years (n=4,795)
DEPRESSION SCALES
	M	Med	SD	α	Ran	M	Med	SD	α	Ran	M	Med	SD	α	Ran
SRQ-20 scale	2.48	1	3.05	0.82	0-18	2.74	2	3.21	0.83	0-19	2.59	2	3.12	0.82	0-19
SSQ scale	1.76	1	2.36	0.79	0-13	2.07	1	2.58	0.81	0-14	1.90	1	2.46	0.80	0-14

Table 3 presents the summary statistics stratified by age groups. Slightly over half of the sample was orphaned (15-19 year olds: 28% paternal 5% maternal and 18% double; 20-24: 28% paternal 6% maternal and 20% double). There were slightly more females (54.9%) than males (45.1%) in the sample. The mean age was 18.8 with the largest one-year age group comprised of 18 year-olds (14%) and the smallest one-year age group comprised of 24 year-olds (8%). Only 12% of the younger cohort were married, compared to over half of the older age group (55%). A similar pattern was seen when examining those who reported ever having sex (15-19 year olds: 18%; 20-24 year olds: 77%). Fifteen percent of the sample reported recent migration, with minor differences between age groups (15-19 year old: 13%; 20-24 year old: 18%). Nearly two-thirds of the younger age group were enrolled in school (60%), with less than 5% of the older group enrolled at the time of the survey. Just under half of the sample (43%) completed secondary school or higher with a larger portion of young adults having completed compared to adolescents (61% vs. 30%). The largest portion of the sample was in the wealthiest quintile (29%) while between 15-20% of the sample fell into the remaining quintiles.

Table 3. Summary statistics of key variables among adolescents (15-19 year olds) and young adults (20-24 year olds).

	Adolescents ages 15-19 years (n=2,768)		Young adults ages 20-24 years (n=2,027)		Total Sample ages 15-24 years (n=4,795)
Variable	N	%	N	%	N	%
INDIVIDUAL CHARACTERISTICS
Orphan status
non-orphan	1346	48.7	895	44.3	2241	46.8
paternal	778	28.2	543	26.9	1321	27.6
maternal	139	5.0	150	7.4	289	6.0
double	500	18.1	433	21.4	933	19.5
Female	1426	51.5	1204	59.4	2630	54.9
Age
15	591	21.4			591	12.3
16	575	20.8			575	12.0
17	461	16.7			461	9.6
18	675	24.4			675	14.1
19	466	16.8			466	9.7
20			426	21.0	426	8.9
21			401	19.78	401	8.4
22			410	20.2	410	8.6
23			412	20.3	412	8.6
24			378	18.7	378	7.9
Married	338	12.2	1114	55.1	1452	30.3
Ever had sex	490	17.7	1566	77.3	2056	42.9
Recent migration	358	13.0	368	18.3	726	15.2
School enrollment	1653	59.7	96	4.7	1749	36.5
Secondary school completion	2764	30.3	1241	61.4	4786	43.4
Wealth quintile
poorest	480	17.4	399	19.8	879	18.7
poor	569	20.6	365	18.1	934	19.8
middle	587	21.3	333	16.5	920	19.5
wealthy	380	13.8	306	15.2	686	14.6
wealthiest	741	26.9	610	30.3	1352	28.7

Factors Associated with Risk of Depression

Table 4 provides bivariate associations for each predictor variable on the two depression outcomes (SRQ-20 and SSQ), stratified by age group. Eight percent (n=224) of adolescents reported enough symptoms to be classified as depressed using the SRQ-20, while only 3.8% (n=106) reported depression using the SSQ. Slightly over 9% (n=188) of young adults reported depression using the SRQ-20, while only 5.1% (n=103) reported depression using the SSQ. For both depression scales, those who were male, married, ever had sex, and recently migrated had a statistically higher percent of depression than their counterparts. Orphan types were statistically different for both age groups using the SRQ-20 but not for the SSQ. There were statistically different associations with depression based on school enrollment for 20-24 year olds across both scales, and for 15-19 year olds using only the SRQ-20 measure of depression. Young adults with a primary education only had statistically higher levels of depression based on the SRQ-20 only. There were statistically significant differences in depression based on wealth quintiles for adolescents only using the SSQ but not the SRQ-20.

Table 4. Comparison of proportions of adolescents (15-19) and young adults (20-24) with depression by socio-demographic characteristic based on the SRQ-20 and SSQ scales¹: Bivariate analysis.

	Adolescents ages 15-19				Young adults ages 20-24
Variables	SRQ-20 n=224 (8.1%)	p	SSQ n=188 (9.3%)	p	SRQ-20 n= 106 (3.8%)	p	SSQ n=103 (5.1%)	p
Orphan status		*				*
non-orphan	6.76		3.27		8.04		3.75
paternal	10.03		4.76		8.10		4.62
maternal	10.70		5.04		12.67		5.19
double	8.20		3.60		12.01		5.14
Gender		**		***		***		***
male	9.68		2.83		12.21		5.78
female	6.41		4.77		4.98		2.63
Age
15	7.78		4.06				--
16	8.17		4.87				--
17	6.72		1.95				--
18	8.74		4.89				--
19	8.80		2.58				--
20					7.51		3.99
21					10.97		5.49
22					11.22		5.85
23					7.52		4.13
24					9.26		6.08
Married		***		***		***		***
no	7.13		3.22		7.11		3.39
yes	14.79		7.99		11.98		6.54
Ever had sex		***		***		***		***
no	6.72		3.12		6.50		3.14
yes	14.49		7.14		11.38		5.98
Recent migration		***		**		**		*
no	7.29		3.42		8.07		4.08
yes	13.41		6.70		11.57		5.92
School enrollment		***						*
no	10.22		4.48		9.69		4.86
yes	6.65		3.39		6.63		3.43
Education level						*
primary or less	8.35		4.10		11.78		5.89
secondary or more	7.54		3.23		7.57		4.59
Wealth quintile				**
poorest	10.83		6.04		11.03		5.51
poor	9.31		4.75		8.49		4.11
middle	6.98		2.56		7.81		4.80
wealthy	6.58		2.11		9.15		5.23
wealthiest	7.14		3.64		9.67		5.57

^*p < 0.05;

^**p < 0.01

¹using the common cut point of 8 or more positive answers for each scale, independently

Depression Prevalence

Among the entire sample of adolescents and young adults the two scales showed a Spearman correlation of 0.66. Correlation was lower among the younger age group (0.62) and slightly higher among the older age group (0.71). Using the conventional cut point of eight or above for both scales, Table 5 compares prevalence rates across scales. Among the full sample, the SRQ-20 reported a prevalence of 8.6% (n=412) while the SSQ reported a prevalence of 4.4% (n=209). T-test reveals non-significant differences in percentages between age groups for the SRQ-20 [mean (SD): 15-19 year olds 8.1% (27.3); 20-24 year olds 9.3% (29.0); P= 0.15] however, significant differences between groups were observed using the SSQ [mean (SD): 15-19 year olds 3.8% (19.2); 20-24 yr. olds 5.1% (22.0); P= 0.04].

SSQ performance using SRQ as gold standard

Table 6 shows the performance indices of the SSQ at various cut points for both age groups using the SRQ-20 with a cut point of eight or higher positive responses as the gold standard criterion. The kappa statistic was highest with a cut point of six (15-19 year olds: 0.71; 20-24 year olds: 0.76). There was an inverse relationship between both AUC scores and cut points, whereby the AUC was highest at the five cut point (15-19 year olds: 0.83; 20-24 year olds: 0.88) with only slight decreases at the six cut point among 20-24 year olds (AUC 0.83). Sensitivity was 0.89 and above for both age groups at cut points of five or lower, while specificity remained above 0.90 for all cut points of five and higher. The positive predictive value (PPV) was the most variable performance index, falling at or below 0.55 with a cut point of five or lower. The negative predictive value (NPV) performed well (>0.95) at all cut points for both age groups. The percent of cases screened correctly ranged from 85.5-95.2%. SSQ prevalence rates ranged from 3.5% to 18.4% among 15-19 year olds and 5.1% to 23.6% among 20-24 year olds. Performance indices for the SSQ were maximized with a cut point between four and six with slightly higher performance indices seen among 20-24 year olds compared to their younger counterparts. In additional to age stratification, performance indices were assessed by gender based on data that suggests that females are more severely affected and studies that have recommended different cut point for the men and women (49,53). The performance indices among gender-stratified subsamples did not reveal any significant differences and as such does not warrant alternative recommendations for the two genders.

Table 6. Performance indices for the SSQ, stratified by age groups.

Criterion Validity for SSQ compared to SRQ-20 with eight or more “yes” responses as the cut point
Subsample of 15-19 year olds (n=2768)
“Yes” responses for SSQ	Sensitivity	Specificity	PPV	NVP	k	% Cases (TP + FP)	AUC
8	.44	.99	.92	.95	.57	3.5	.44
7	.58	.99	.82	.96	.66	5.7	.57
6	.77	.97	.70	.98	.71	8.9	.75
5	.89	.94	.45	.99	.64	13.2	.83
4	.96	.89	.42	1.00	.53	18.4	.84
Subsample of 20-24 year olds (n=2027)
“Yes” responses for SSQ	Sensitivity	Specificity	PPV	NVP	k	% Cases (TP + FP)	AUC
8	.54	.99	.98	.95	.67	5.1	.53
7	.69	.99	.87	.97	.75	7.4	.68
6	.86	.97	.73	.99	.76	11.0	.84
5	.95	.92	.55	1.00	.66	16.2	.88
4	.99	.84	.39	1.00	.49	23.6	.83

Symptom Profile

Table 7 provides the symptom profile with the three most prevalent items for each scale in bold for the entire sample, stratified by age (15-19 year olds and 20-24 year olds) and depression status (yes/no) based on the standard cut point of eight or higher for both scales.

Table 7. Symptom profile, stratified by age groups.

	Adolescents 15-19 (n=2768)		Depressed Adolescents 15-19 (n=224) based on the SRQ 20		Depressed Adolescents 15-19 (n=106) based on the SSQ		Youth 20-24 (n=2027)		Depressed Youth 20-24 (n=188) based on the SRQ 20		Depressed Youth 20-24 (n=103) based on the SSQ		Total youth 15-24 (n=4795)		Depressed Youth 15-24 (n=412) based on the SRQ 20		Depressed Youth 15-24 (n=209) based on the SSQ
	(n)	%	(n)	%	(n)	%	(n)	%	(n)	%	(n)	%	n	%	(n)	%	(n)	%
WHO SRQ 20 Items
Were you having headaches?	720	26	182	81.3			561	27.7	137	72.9			1281	26.7	319	77.4
Was your appetite poor?	438	15.8	157	70.1			350	17.3	109	58.0			788	16.4	266	64.6
Did your hands shake?	167	6	77	34.4			102	5.0	49	26.1			269	5.6	126	30.6
Did you feel tense nervous or worried?	248	9	128	57.1			198	9.8	103	54.8			446	9.3	231	56.1
Did you have trouble thinking clearly?	337	12.2	157	70.1			272	13.4	124	66.0			609	12.7	281	68.2
Did you have trouble enjoying your daily activities?	338	12.2	146	65.2			281	13.9	127	67.6			619	12.9	273	66.3
Were you able to play a useful part in life?	815	29.4	109	48.7			637	31.4	90	47.9			1452	30.3	199	48.3
Did you lose interest in things?	304	11	114	50.9			255	12.6	117	62.2			559	11.7	231	56.1
Did you feel a worthless person?	222	8	107	47.8			231	11.4	116	62.0			452	9.5	223	54.3
Did you have uncomfortable feelings in your stomach?	380	13.7	121	54.0			307	15.2	120	63.8			687	14.3	241	58.5
Did you feel able to cope with most of the problems in your life?	624	22.6	93	41.7			451	22.3	73	38.8			1075	22.4	166	40.4
Overlapping Items
Were you having problems sleeping?	260	9.4	111	49.6	76	71.7	235	11.6	89	47.3	57	55.3	495	10.3	200	48.5	133	63.6
Were you easily frightened?	227	8.2	101	45.1	64	60.4	143	7.1	67	35.6	49	47.6	370	7.7	168	40.8	113	54.1
Were you having digestion (tummy) problems?	360	13.0	133	59.4	72	67.9	293	14.5	110	58.5	72	69.9	653	13.6	243	59.0	144	68.9
Did you feel more unhappy than usual?	356	12.9	145	64.7	84	79.3	277	13.7	121	64.4	80	77.7	633	13.2	266	64.6	164	78.5
Did you cry more than usual?	192	6.9	94	42.0	60	56.6	134	6.6	71	37.8	54	52.4	326	6.8	165	40.1	114	54.6
Did you find it difficult to make decisions?	225	8.1	103	46.0	66	62.3	209	10.3	112	59.6	79	76.7	434	9.1	215	52.2	145	69.4
Was your daily work suffering?	329	11.9	117	52.2	68	64.2	318	15.7	124	66.0	76	73.8	647	13.5	241	58.5	144	68.9
Has the thought of ending your life been on your mind?	66	2.4	37	16.5	27	25.5	66	3.3	44	23.4	31	30.1	132	2.8	81	19.7	58	27.8
Were you feeling tired all the time?	267	9.7	121	54.0	69	65.1	233	11.5	110	58.5	71	68.9	500	10.4	231	56.1	140	70.0
SSQ Items
Did you have nightmares or bad dreams?	585	21.1			79	74.5	443	21.9			84	81.6	1028	21.4			163	78.0
Did you sometimes think deeply or think about many things?	662	22.5			93	87.7	670	33.1			96	93.2	1292	27			189	90.4
Did you sometimes see or hear things which others could not see or hear?	109	3.9			42	39.6	71	3.5			33	32.0	180	3.8			75	35.9
Did you find yourself sometimes failing to concentrate?	518	18.7			91	95.9	444	21.9			94	91.3	962	20.1			185	88.5
Did you lose your temper or get annoyed over trivial matters?	769	27.8			84	79.3	666	32.9			95	92.2	1434	29.9			179	85.7

Factors Affecting Misclassification

False negatives were identified as depressed using the SRQ-20 and not depressed using the SSQ. Table 8,provides a multivariate logistic regression for key characteristics associated with being classified as a false negative. Model 1 uses the recommended cut point of eight or greater for the classification as depressed by the SSQ. (We were unable to analyze associations with false positives because of the rareness of the event.) Of the 4,795 subjects, 4.7% (n=223) were misclassified. Of those who were misclassified, a majority (95.6%) were false negatives (n=213), while only 4.6% of the misclassified sample were false positives (n=10). In the present study, the statistically significant associations between socio-demographic variables and false negative misclassification for adolescents included orphan status (OR 1.48, p<0.05; paternal: OR 1.48, p<0.05), being married (OR 0.47, p<0.05) and ever having sex (OR 2.13, p<0.05). Secondary education or higher was associated with misclassification among young adults (OR 1.35, p<0.01).

Table 8. Parameter estimates (odds ratio) of being classified as a false negatives by the SSQ using a cut point of eight or greater, stratified by age group.

	MODEL 1						MODEL 2
	Ages 15-19 n=2715			Ages 20-24 n=1957			Ages 15-19 n=2715
Variables	OR	(95% CI)	p	OR	(95% CI)	p	OR	(95% CI)	p
Orphan status
non-orphan	Ref			Ref			Ref
orphan	1.48	(1.01, 2.18)	*	1.35	(0.83, 2.19)
paternal	1.48	(1.08, 2.46)	*	1.17	(0.64, 2.11)		0.83	(0.33, 2.12)
maternal	1.56	(0.72, 3.37)		1.89	(0.89, 4.04)		1.21	(0.27, 5.47)
double	1.46	(0.87, 2.45)		1.39	(0.77, 2.53)		0.54	(0.15, 1.83)
Gender
male	Ref			Ref			Ref
female	1.11	(0.74, 1.66)		1.44	(0.83, 2.48)		0.82	(0.34, 1.93)
Age
15	Ref						Ref
16	0.87	(0.47, 1.61)					1.43	(0.40, 5.11)
17	1.05	(0.57, 1.94)					0.85	(0.19, 3.84)
18	0.93	(0.49, 1.78)					1.04	(0.27, 4.10)
19	1.11	(0.55, 2.26)					1.42	(0.30, 6.68)
20				Ref
21				1.49	(0.74, 3.03)
22				1.41	(0.71, 2.80)
23				0.79	(0.36, 1.71)
24				0.70	(0.31, 1.59)
Married
no	Ref			Ref			Ref
yes	0.47	(0.22, 1.00)	*	1.02	(0.56, 1.83)		1.07	(0.20, 5.62)
Ever had sex
no	Ref			Ref			Ref
yes	2.13	(1.09, 4.13)	*	1.47	(0.66, 3.28)		1.32	(0.28, 6.17)
Migration in the past year
no	Ref			Ref			Ref
yes	1.29	(0.73, 2.27)		1.04	(0.58, 1.86)		1.48	(0.40, 5.50)
School Enrollment
no	1.27	(0.76, 2.13)		1.35	(0.30, 6.07)		1.94	(0.56, 6.68)
yes	Ref			Ref			Ref
Education level
primary or less	Ref			Ref			Ref
secondary or more	1.12	(0.70, 1.80)		2.11	(1.29, 3.47)	**	1.97	(0.72, 5.50)
Wealth quintile
poorest	1.06	(0.57,1.94)		1.24	(0.58, 2.65)		0.53	(0.12, 2.36)
poor	1.13	(0.66, 1.95)		1.29	(0.58, 2.85)		0.67	(0.18, 2.53)
middle	Ref			Ref			Ref
wealthy	1.08	(0.56, 2.08)		1.14	(0.50, 2.61)		1.51	(0.48, 4.73)
wealthiest	0.82	(0.48, 1.42)		1.27	(0.62, 2.61)		0.82	(0.28, 2.43)

^*p < 0.05;

^**p < 0.01

Based on performance indices and our recommendation of a revised cut point of five or greater for the SSQ, we conducted an additional multivariate logistic regression for key characteristics associated with being classified as a false negative using this modified cut point for the classification as depressed by the SSQ. Results are provided in Model 2. Misclassification of the full sample declined to less than 1%. None of the socio-demographic variables presented statistically significant associations with being systematically misclassified. This indicates an improvement in the SSQ’s functionality and correct identification of the depression.

Discussion

Adolescence is a period of rapid change and development. It is a period full of new experiences, including sexual initiation, school completion, and growing obligations. Ensuring that adolescents integrate well into society is essential for them to be able to meet the variety of demands that they will face. Being able to correctly identify those who are suffering from depression will allow services to reach those in need and enable a safe transition into adulthood.

This paper adds to the existing literature by comparing the performance of two depression screening instruments and provides suggestions on modifications to the SSQ cut point in order to improve its ability to capture depression among adolescents. This paper highlights adolescents who are most at-risk of being overlooked for depression by the SSQ at it currently stands. This information can be used among providers of social services, teachers, clinicians, and parents to understand the characteristics of adolescents who may fall through the cracks. Overall, this paper aims to convince researchers and mental health service providers to think critically about their use of depression screening instruments and how they function within their study’s relative context.

SSQ Performance as a Depression Screening Tool Among Youth

In evaluating scales, the most appropriate cut point score is a compromise between high sensitivity and acceptable specificity. A high rate of misclassified false negatives was a shortcoming among the tool’s conventional cut point and sample (sensitivity: 0.67), which was exacerbated among our sample of adolescents and young adults (sensitivity: 0.44-0.54). Given the tool’s low ability to capture potential cases, we choose to err on the side of caution by giving preference to high sensitivity over high specificity. This may result in a higher chance of false positive classification, but will ensure more potential cases are being identified within the population. Given the two-step process of screening and diagnosis, clinicians are likely to detect false positives.

We were careful to avoid the kappa paradox when recommending a modified cut point given the rareness of the event. Research has shown that rare events can produce low kappa values even when there are higher levels of agreement among other performance indices (56). For this reason, we recommend a cut point of five, where sensitivity, specificity and AUC were collectively highest despite a slightly lower but substantial kappa value. This alternate cut point substantially increases the functionality of the SSQ in capturing depression cases among youth, with greater levels of improvements among 20-24 year olds compared to 15-19 year olds.

Patterns of Misclassification

Systematic patterns of misclassification using the recommended cut point of eight or greater exist among 15-19 year olds with particular characteristics. This pattern of misclassification among adolescents disappeared using a modified cut point of five or higher. For young adults, classification was consistent between the current SSQ scale and the SRQ-20 with the exception of education levels.

Compared against the SRQ-20, the current SSQ scale falls short in capturing 15-19 year old orphans who may be depressed, despite evidence that orphans experience elevated rates of depression, anxiety, and negative views about the future compared to non-orphans (45,57–63). This is of particular concern in Zimbabwe, a country that has one of the largest numbers of AIDS orphans per capita in the world (64). Studies reveal that adolescents with major life changes, particularly parental death, feel they lack control over their lives and have feelings of helplessness and hopelessness (62,65–67). As such it is important that tools intended to capture affective disorders among this subgroup are functioning well. By adequately screening and attending to adolescent orphans’ emotional needs, they will be better able to cope with the continued challenges they will confront in adulthood including but not limited to risks associated with HIV infection.

Equally important is the association between misclassification and early sexual debut, especially in communities with high HIV prevalence rates. Studies have shown a strong association between early sexual debut and depression (8,45,47,68), yet the SSQ in its current form underperforms in its ability to correctly screen depression among sexually active adolescents. Adolescent depression has also been shown to be a strong predictor for additional sexual risk-taking behaviors such as intergenerational partnerships, transactional sex, and poor condom use – all of which increase one’s susceptibility for contracting HIV (7,8,45,47). Consequently it is essential that interventionists screen adolescents who become sexually active at early ages. This will ensure that they are able to access necessary psychosocial support - one step toward addressing risk predictors of HIV infection.

Our findings suggest higher levels of education among 20-24 year olds are associated with a higher likelihood of being false negatives. Other studies in Africa among same aged post-secondary students produced similar findings, suggesting that test-taking and performance-based stress can increase young adults’ odds for depression (44,69). Students in the later stages of their education are likely to be undertaking competitive examinations that determine admission to highly competitive tertiary institutions. In order to avoid missing students who may be suffering from depression, teachers must be educated on common depressive signs and symptoms. By recognizing relevant mood and behavior changes over time, teachers can serve as effective conduits for depression screening and support efforts.

Limitations

There are several noteworthy limitations to this study. A national estimate of depression for Zimbabwean adolescents remains unknown, which limits our ability to compare our sample to the broader population. Studies among adolescents in similar regions suggest that our depression estimates based on the SRQ-20 are appropriate for this age group (35). While there is a movement towards the development and use of culturally sensitive tools, this study finds that the standardized SRQ-20 does a better job in capturing depression than the unrevised SSQ. However, scales were not administered separately or in their entirety but rather overlapping items were only asked once, utilizing exclusively SRQ-20 phrasing rather than the culturally-emic language in the SSQ. These deviations from the intended format of the scales have the potential to compromise the integrity of each individual scale and bias depression rates to favor the SRQ-20. However the five questions unique to the SSQ did remain as stated using the original Shona-specific idioms.

Also noteworthy are the differences in wording between the two scales. Several of the SSQ questions had greater ambiguity than the SRQ-20 and were doubled barreled. Despite having being previous piloted during the scale’s development, the SSQ lacks clarity which may result in misunderstanding among younger participants. Furthermore, interviewers administered both tools verbally despite their design for self-administration. This has the potential to create social desirability bias, especially among younger as well as more educated participants (70). Lastly, the multivariate regression used to assess the odds of misclassification failed to include several important variables such as interview identification and other scale development measure that bias estimates and overlook additional causes for misclassification.

Conclusion

With a conservative estimate of 9% depression among this sample, there is a clear need for psychosocial interventions to prevent, screen, diagnose, and treat mental health problems among adolescents and young adults in Zimbabwe. Having an instrument that can correctly screen for depression is the first step to ensuring that young people move healthily into adulthood.

While the SSQ may appropriately meet the need for an emic depression diagnostic tool in Zimbabwe, our findings suggest a strong recommendation towards either 1.) revising the SSQ cut point to five or greater to ensure it is both appropriate for and effective in capturing adolescent cases or 2.) to continue utilizing the standardized SRQ-20.

Traumatic events can contribute to the emotional difficulties and increase depression symptomology. Individuals with characteristics including orphan status, being married, and early sexual debut must be carefully observed to ensure they are not falling through the cracks in an environment with underdeveloped mental health services. By better understanding the depressive symptoms affecting this population, mental health services and community-centered social programs can be tailored to address the immediate psychosocial concerns of this growing population and aid in the development of mental health interventions. In conclusion, researchers who are currently using the SSQ in regions of SSA should carefully consider the age of their study population and make adjustments according to our findings for more accurate identification of depressed adolescents.