Use of short messaging services to assess depressive symptoms among refugees in South Africa: Implications for social services providing mental health care in resource-poor settings

Abstract

Few studies in developing nations have assessed the use of short messaging services (SMS) to identify psychological challenges in refugee populations. This study aimed to assess the feasibility of SMS-based methods to screen for depression risk among refugees in South Africa attending mental health services, and to compare its reliability and acceptability with face-to-face consultation. Of the 153 refugees enrolled at baseline, 135 were available for follow-up assessments in our cohort study. Depression symptomatology was assessed using the 16-item Quick Inventory of Depressive Symptomatology (QIDS) instrument. Nearly everyone possessed a mobile phone and utilized SMS. Furthermore, low incomplete item response in QIDS and high perceived ease of interacting via SMS with service providers supported the feasibility of this method. There was a fair level of reliability between face-to-face and SMS-based screening methods, but no significant difference in preference rating between the two methods. Despite potential implementation barriers (network delay/phone theft), depression screening using SMS may be viable for refugee mental health services in low-resource settings.

Introduction

Psychological distress, particularly depression, is a serious health challenge for many refugees who have endured difficult circumstances and fled their home nation. Meta-analyses^1,2 estimate the prevalence of depression among adult refugees to range between 30-44%. Depression is treatable, but unmet mental health treatment needs are pervasive across developing countries. For example, a study by the WHO World Mental Health Surveys Initiative involving 17 countries found that the majority of people with depression were untreated, particularly in developing countries.³ Moreover, refugees often face considerable obstacles in accessing appropriate mental health services, including language barriers, transport challenges, and lack of trust and culturally appropriate care.⁴

Similar to other developing countries, the mental health treatment gap in South Africa⁵ is a significant impediment to psychological well-being for many of its inhabitants. This gap is estimated as high as 75% (25% of individuals with mental disorder sought treatment within 12 months) ⁶, a serious challenge that has implications for a nation hosting approximately 300,000 officially recognized refugees/asylum seekers.⁷ While the South African health authorities established a target of screening and treating 30% more individuals for mental disorders by 2030⁸, such a declaration is viewed with caution, given the financial constraints and conditions of the health system.⁹ In developing countries, improving the uptake of screening and treatment without adding to the financial burden on service providers and the refugee population requires innovative approaches to how mental health care services are provided.

This study explored the viability of short messaging services (SMS) in screening for depression within a refugee population in the city of Durban, South Africa. Our decision to test SMS-based depression assessment was in part guided by a behavioral change theory, specifically the Andersen's Behavioral Model of Health Services Use framework^10-12, which posits that health services use is a function of predisposing and enabling factors, as well as client needs. The framework has been applied previously in studies of help-seeking behavior among refugees for mental health challenges.^13,14 While refugees may desire treatment for psychological challenges, they may not necessarily have the resources and opportunities to carry out intended behaviors. For disenfranchised and vulnerable populations (e.g. unemployed and low income), contact and support via SMS has been found to be favorable.^15-17

A systematic review of SMS applications for health in developing countries suggests these interventions are appropriate and promising (i.e. prevention, management, surveillance and treatment support) tools¹⁸. In the United States, a non-automated SMS-based intervention was implemented by social work staff in a psychiatric rehabilitation agency setting¹⁹, raising their possible use for social service organizations that are often the first responders with expertise in assisting refugees in resource-limited settings. Widely used to improve a broad range of health behavior^20,21, SMSs have been utilized in South Africa to provide emotional support to individuals for stress²². While the SMS-based interventions for mental health have been used in developed nations, such initiatives have seldom been used for mental health concerns²³, particularly in developing nations, with the majority of applications focusing on HIV/AIDS¹⁸. The use of SMS-based screening for depression by social service providers targeting refugee populations has, to our knowledge, not been assessed systematically in sub-Saharan Africa. With the purpose of informing potential future interventions in low-resource settings, the study consisted of two components: firstly, assessing the feasibility of SMS-based method of screening for depression risk among refugees residing in South Africa within social services settings; and secondly, comparing its reliability and acceptability with face-to-face consultation.

Methods

Sampling and data collection

Data was collected (July 2013-April 2014) at a non-government organization (NGO) that provides refugee services in Durban, South Africa. The NGO provides health and social services to refugees, asylum seekers and migrants who comprised our target sample population. Potential study participants were referred to our study team by nurse clinicians unrelated to our study. The inclusion criteria were (a) 21 years or older, (b) non-South African, (c) an NGO client and (d) the ability to read/speak English, the language in which the SMSs were sent. Individuals determined by the NGO nurse clinician to have a developmental disability or mental impairment that would interfere with capacity to consent to the interview were not referred and were excluded. In addition, we excluded those who were unable to provide informed consent. Our study utilized a longitudinal cohort design that involved baseline and follow-up assessments. Potential study participants meeting the above criteria were provided with a study description, and gave their informed consent for both assessments; with 153 and 135 individuals participating in the baseline and follow-up assessment respectively (88.2% follow-up rate). Those lost to follow-up (n=18) were significantly more likely to be Zimbabwean's (n=16), and to lack tertiary-level education (n=13). The median time between the assessments was 33 days. We provided ZAR10 (US$1) per study participant for the cost of SMS use. Given the emphasis on cultural competency in refugee mental health services²⁴, interviews were conducted at the social service organization by a trained Master's degree volunteer who had refugee status in South Africa. The Columbia University Institutional Review Board and University of KwaZulu-Natal Biomedical Research Ethics Committee approved the study.

Assessment

Depression symptomatology was assessed using a 16-item 4-point Likert scale self-reported South African version of the Quick Inventory of Depressive Symptomatology (QIDS). In this psychometrically reliable/valid instrument²⁵, study participants are asked about symptoms associated with depression over the past seven days. QIDS has been used in two other mobile phone technology studies^26,27, as well as in a non-mobile South African study setting.²⁸ The total score ranges from 0-27, with higher category scores signifying greater depressive symptomatology, i.e. no depression (<5), mild (6-10), moderate (11-15), severe (16-20), and very severe (>21).

The baseline depression screening assessment took place using one of the two methods based on random assignment (SMS or face-to-face), with the study participant being screened using the alternative method at follow-up. Both forms of interaction (SMS and face-to-face) occurred at the clinic in an assessment environment of the consulting room, with the SMS interaction involving no verbal communication between the two parties. The research personnel used a USB modem-equipped laptop computer (without internet connection) and sent SMSs to the mobile phones of the participants, enquiring about possible symptoms of depression. Questions or requests for responses sent to participants were pre-stored in a word processing document, and later copied and pasted into a SMS console program (manufacturer pre-installed program installed in the USB modem) that resembles emailing software. Only one question or request was sent at a time from the SMS console/laptop. All responses to questions or requests submitted by participants were captured in the SMS console program. As QIDS entails a 4-point Likert scale, participants were asked to respond in numeric code, using 1-4. In the case of a request (e.g. “Can we proceed?”), the response was either ‘yes’ or ‘no’. Based on the directions set out by the QIDS instrument, and depending on previous responses, subsequent questions or requests were sent.

When the assessment was completed, research personnel transferred the numeric response code from the SMS console program to paper form to determine the total QIDS score. Following the interview, study related messages were deleted immediately from the SMS console and participants were advised to delete study related information from their mobile phones. The information relating to demographic, feasibility/acceptability evaluation, and feedback on the SMS-based assessment (open-ended questions) was collected using the face-to-face method.

Analysis

Analyses were conducted on baseline demographic and clinical characteristics as well as central tendencies of mobile phone and SMS consumption; barriers to (mobile) technology utilization patterns (descriptive statistics); help-seeking attitudes and behavior regarding mental health challenges (descriptive statistics); and depression outcomes and feedback on the screening process methods. The mean rating differences and proportions of assessment preferences were analyzed using the Student's dependent t-test and one-way chi-square procedures respectively. The reliability between face-to-face and SMS-based depression screening was assessed based on the test and retest method (among 135 who participated in both baseline and follow-up interview), using both the weighted Kappa coefficient and intraclass correlation coefficient twoway random effect model. For the Kappa coefficient, “<0” indicates “poor agreement”, “0-0.20” indicates “slight agreement”, “0.21-0.40” indicates “fair agreement”, “0.41-0.60” indicates “moderate agreement”, “0.61-0.80” indicates “substantial agreement”, and “0.81-1” indicates “almost perfect [excellent] agreement”.²⁹ Responses to open-ended questions were tabulated and reported using descriptive statistics.

Results

Demographic and clinical characteristics

One hundred and fifty-three study participants participated in the baseline assessment (Table 1), all of whom were black Africans born outside South Africa. Nearly all were earning less than R5 000 ($50) per month (n=148; 96.7%). Approximately half were female (n=77; 50.3%), Zimbabwean (n=79; 51.6%), married or living with a partner (n=82; 53.6%), had completed secondary education (n=85; 55.6%), and reported experiencing discrimination in South Africa (n=76; 49.7%). A number also reported experiencing homelessness (n=26; 17.0%) and psychiatric hospitalization (n=7; 4.6%) within one year prior to the baseline assessment.

Table 1.

Baseline characteristics (N=153)

	Total
	n	%
Gender:
Female	77	50.3
Male	76	49.7
African nationality:
Zimbabwe	79	51.6
Democratic Republic of Congo	66	43.1
Others	8	5.2
Income:
<ZAR1,000 a month [US$100]	59	38.5
ZAR1,001-R5,000	89	58.2
>ZAR 5,001	5	3.3
Education:
Tertiary education and training	85	55.6
Age Category:
21-29	51	33.3
30-39	59	38.6
40-59	43	28.1
Marital Status:
Single	51	33.3
Married/with partner	82	53.6
Divorced/widowed	20	13.1
Social Support:
None	41	26.8
1-2	65	42.5
3+	47	30.7
Exposure to discrimination in South Africa:	76	49.7
Homelessness ≤ 1 year:	26	17.0
Hospitalization related to mental health ≤ 1 year:	7	4.6

Technology consumption and barriers to utilization

Nearly all participants owned a mobile phone and reported using short messaging services (Table 2). In comparison, less than half owned a computer (n=66; 43.1%), and approximately one third utilized certain functions that required the use of internet data (checking email via mobile browser, instant messaging, and downloading applications to their phone). Nearly all participants were on self-funded (n=147; 96.1%) and pre-paid mobile phone plans (n=144; 94.1%). While the frequency of phone calls or SMSs to friends and family was relatively high (Table 3), communication with professional care providers was low. Nearly half utilized the “Please Call Me” function, requesting the call recipient to contact the caller. Most commonly, individuals frequented an internet café at least once a month (n=31; 20.3%) to obtain internet connectivity. While restrictions regarding use remained low (Table 2), approximately two thirds reported fear of phone theft in the near future (n=104; 68.0%).

Table 2.

Technology consumption and barriers to utilization (N=153).

	Total
	n	%
1. Currently own or borrowed a mobile phone from others	152	99.3%
Ability to use SMS	150	98.0%
Owns more than one mobile phone	24	15.7%
2. Currently owns or borrows a computer	66	43.1%
3. Mobile technology use on a typical day:
Number of SMS sent	Mdn = 2	IQR = 2
Number of SMS received	Mdn = 3	IQR = 3
Number of instant messages sent	Mdn = 6	IQR = 20
Number of instant messages received	Mdn = 7	IQR = 20
4. Currently possess data package	110	71.9%
Ability to check email by mobile phone	103	67.3%
Ability to browse internet/website by mobile phone	103	67.3%
Ability to instant message others	103	67.3%
History of downloading applications via mobile phone	97	63.4%
Possess an account with social network sites	101	66.0%
5. Currently on pre-paid plan	144	94.1%
On contract plan	8	5.2%
6. Payment for mobile phone use†:
Study participant themselves	147	96.1%
Payment support from family (including spouse/extended family)	42	27.5%
Payment support from friends (including boy/girlfriend)	16	10.5%
7. Restriction on phone utilization:
No restriction	140	91.5%
Restriction of phone use by boy/girlfriend	7	4.6%
Restriction of phone use by family	6	3.9%
8. Loss of mobile phone
Phone loss (one-year prevalence)	49	32.0%
Phone theft (one-year prevalence)	54	35.3%
Fear of future phone theft	104	68.0%
Plan to discontinue the current phone number and get new one	9	5.9%

^†Multiple contributors allowed. SMS = short messaging services (text)

Table 3.

Technology utilization frequency (N=152).

	None†		Not at all		<Once per month		At least once a month		At least once a week		Almost every day or everyday
	n	%	n	%	n	%	n	%	n	%	n	%
1. Frequency of mobile phone communication with‡:
Family members	0	0.0	3	2.0	7	4.6	21	13.8	70	46.1	51	33.3
Friends	2	1.3	5	3.3	1	0.7	5	3.3	45	29.6	94	61.4
Professional care providers	39	25.7	67	44.1	17	11.1	17	11.2	9	5.9	3	2.0
2. Frequency of SMS communication with‡:
Family members	2	1.3	24	15.8	10	6.6	16	10.5	51	33.6	49	32.0
Friends	6	3.9	20	13.2	8	5.3	9	5.9	53	34.9	56	36.6
Professional care providers	37	24.3	76	50.0	7	4.6	17	11.2	14	9.2	1	0.7
3. Frequency of internet café:	62	40.5	12	7.8	17	11.1	31	20.3	15	9.8	16	10.5
4. Frequency of “Please call me” function:	1	0.7	18	11.8	7	4.6	11	7.2	41	27.0	74	48.4

^†No family/friends/professional care providers and work.

^‡In a typical week.

SMS = short messaging services (text)

The total sample size for items 3 is based on 153.

Help seeking attitude and behavior for mental health challenges

The majority (Table 4) had discussed mental health challenges with family (n=81; 52.9%) or friends (n=87; 56.9%) within one year prior to the baseline assessment. In comparison, consultations with social workers (n=16; 10.5%) and other healthcare providers, such as general practitioners (n=13; 8.5%) and nurses (n=15; 9.8%) during the same period, were few. Family (n=109; 71.2%) and friends (n=83; 54.3%) were preferred for discussion of future mental health challenges; with approximately two-thirds (Table 5) feeling very comfortable discussing these challenges face-to-face with family (n=101; 66.0%) or with health/social service providers (n=104; 68.0%). In comparison, less than one-third felt very comfortable discussing mental health challenges using non face-to-face means of communication with health/social service providers; with SMS-based interaction being preferred over phone, instant messaging or email. Nearly all participants agreed to be contacted via SMS by the social service organization.

Table 4.

Help seeking attitude and behavior regarding mental health challenges (N=153).

	Discussed MH problem with‡:		Would consider discussing MH problem with:
	n	%	n	%
Family	81	52.94	109	71.24
Friends	87	56.86	83	54.25
Relative	35	22.88	47	30.72
Social (service) worker	16	10.46	54	35.29
Neighbors	15	9.8	15	9.8
Healthcare worker	4	2.61	48	31.37
Nurse	15	9.8	53	34.64
GP	13	8.50	85	55.56
Colleague	12	7.84	17	11.11
Traditional/spiritual leader	12	7.84	11	7.19
Police	6	3.92	8	5.23
Employer	4	2.61	7	4.58

^‡Within the last 12 months

Table 5.

Interaction preference (N=153).

	Not comfortable at all		Somewhat uncomfortable		Neither comfortable/uncomfortable		Somewhat comfortable		Very comfortable
	n	%	n	%	n	%	n	%	n	%
1. Face-to-face interaction with:
Family members	12	7.8	8	5.2	9	5.9	23	15.0	101	66.0
Friends	25	16.3	12	7.8	12	7.8	46	30.1	58	37.9
Health or social service providers	8	5.2	9	5.9	7	4.6	25	16.3	104	68.0
2. Interaction with service providers by:
Phone	32	20.9	26	17.0	27	17.6	51	33.3	17	11.1
SMS	44	28.8	21	13.7	13	8.5	32	20.9	43	28.1
Instant messages	55	35.9	19	12.4	17	11.1	24	15.7	38	24.8
Email	56	36.6	20	13.1	17	11.1	26	17.0	34	22.2

Depression screening process

Every participant responded to all questions related to depression via SMS. Based on the QIDS categories, approximately half reported moderate or higher levels of depressive symptomatology (n=79; 51.7%) during the baseline assessment (Table 6). Being female, low-income, divorced/widowed, of low educational attainment and a history of homelessness were significantly associated with higher depression ratings. Nearly a third reported moderate or higher levels of depressive symptomatology during the follow-up assessment (n=41; 30.4%). The mean depression score, based on paired t-test, indicated a significant difference between the baseline and follow-up assessments (M_baseline=10.3 vs. M_follow-up=8.0, p< 0.01). The test and retest reliability based on the intraclass correlation coefficient two-way random effect model and weighted Kappa coefficient were 0.45 and 0.25 respectively, indicating a fair level of agreement between the SMS-based and face-to-face methods. The results based on one-way chi square indicated no significant difference (P_sms=58.7% vs. P_face-to-face=41.3%, p=0.99) between their preferred methods of assessment (SMS or face-to-face).

Table 6.

Depression symptoms during baseline and follow-up assessment

	Baseline		Follow-up
Severity	n	%	n	%
None	34	22.2	41	30.4
Mild	40	26.1	53	39.3
Moderate	57	37.3	37	27.4
Severe	21	13.7	4	3.0
Very Severe	1	0.7	0	0.0

None (<5), mild (6-10), moderate (11-15), severe (16-20), and very severe (>21).

153 and 135 individuals participating in the baseline and follow-up assessment respectively

Feedback on participants’ experience related to SMS-based assessment was obtained by structured and open-ended questions. Over three-quarters rated the experience as ‘good’ or ‘excellent’, and approximately 95% would recommend it to their family and friends. Only 10% found the SMS-based assessment difficult to use. Most (n=120; 86.9%) found the SMS-based assessment somewhat or very comfortable to use. In terms of future use, 97.1% (n=134) would be interested in using the SMS-based assessment. The most commonly endorsed strength of the system was that it functioned well in general (n=41), while delays due to the phone network were cited as the biggest challenge (n=66).

Discussion

Despite a number of potential implementation challenges, our findings indicate that using SMS to screen for risk of depression may be viable for both social service agencies providing services and refugees in low-resource settings. The low rate of incomplete item responses and the relative ease of use, combined with the fact that nearly all participants owned a mobile phone and utilized SMS, support its feasibility. While participants reported a general preference for face-to-face interaction with health-providers in discussing psychological challenges, the results indicated no significant difference in preference regarding the methods of assessment. Furthermore, comfort of use was relatively high with respect to SMS-based depression screening, indicating acceptability. Lastly, there was a fair level of reliability between face-to-face and SMS-based methods in terms of their ability to screen for depression.

Several of our findings on patterns of refugee mobile phone use resemble those of a previous study at a hospital-based antiretroviral treatment center in Durban, South Africa.³⁰ That study found that most participants owned a mobile phone (81%), were on a pre-paid plan (95%), were willing to accept SMS from the clinic (96%), and had experienced lost/theft of a device (≈40%). Most participants were happy to interact with service providers via SMS.

Insignificant differences in preferred methods of assessment (SMS or face-to-face) was an unexpected result from our study that warrants further discussion. Possible explanations include (1) the role of opportunity costs, and (2) discrimination. Firstly, almost all participants earned less than ZAR 5,000 per month, with utilization of the “Please call me” function being high, indicating general economic restrictions facing this population. The analysis indicated a significant negative association between income attainment and depression symptomatology. Despite a desire to access health care providers face-to-face (68% very comfortable rating from Table 5), obtaining treatment for mental health symptoms, which may not be readily available in government facilities in South Africa, represents opportunity costs of lost economic output and wages. Long waiting time³¹ in community clinics and a lack of transportation³² were common barriers to accessing mental health care, influencing contact method preferences. Secondly, previous consultation with health professionals regarding mental health was relatively low in our study (Table 4), with people relying on and preferring family members as a source of support when discussing these challenges (Table 4). Discrimination in the public health sector is a well-documented experience for refugees and asylum seekers in South Africa, despite having the rights to access these health services.³³ Nearly half the participants experienced discrimination, highlighting the hypothesized impact of discrimination in public health facilities on service underutilization, and the preference for SMS-based mental health assessment.

Our study indicated significant differences in demographic and clinical characteristics between baseline and follow-up assessments, which have important practice implications. Individuals from Zimbabwe, and those with low educational attainment were less likely to follow-up, despite repeated attempts at contact. Given that Zimbabwe is adjacent to South Africa, it is possible that some Zimbabwean participants were in their country of origin during the implementation of the follow-up interviews. It highlights the importance of more assertive outreach efforts targeting migrants, and those of lower socioeconomic status in order to leverage appropriate technology by social service providers for individuals with symptoms of mental illness. In our study, the mean depression score was lower at follow-up compared to baseline. One possible explanation is that the interviews may have served the unintended function of an intervention where the act of engaging in the assessments provided some relief for some of study participants.

There are several study limitations, the first being generalizability with respect to cultural competency and language, as our target sample population was refugees who have some literacy in English. Qualitative research indicates that cultural differences affect how individuals engage in mobile-health interventions³⁴. Furthermore, other evidence suggests that even the tone of language used in the SMS can impact on the interaction between the provider and the recipient.³⁵ Although we did not systematically record the specific reasons for study participants not interviewed, approximately 20 potential study participants (≈ 13%) were excluded from the study due to English being a language barrier. Therefore, the viability (feasibility, acceptability and reliability) of the SMS-based assessment may not be applicable for refugees without literacy in English. We also acknowledge that language challenges may have accounted for lower reliability between the two assessments. As language diversity is a possible challenge to leveraging the full potential of mobile phone technology in South Africa³⁶, future investigations accommodating non-English individuals are warranted in order to enhance reliability and external validity. The second limitation is our use of a single-subject (A-B) study design. While we utilized the cohort method to assess reliability, the interpretability of our findings would have been enhanced using two comparison groups rather than each study participant being exposed to both types of assessment methods.

Notwithstanding these limitations, SMS-based screening appeared to be a feasible, acceptable, and reliable approach that can be operated easily and inexpensively by social service agencies without any proprietary automated system to identify mental health needs in this vulnerable and highly mobile refugee population. Technology and consumer preferences (such as lower than expected use of internet cafés in our study) are constantly evolving, and the cost of telecommunications decreasing (e.g. the cost of sending unlimited SMS, internet and smart phones). The value of this study lies in its demonstrating the possibility of screening for mental illness in low-resource settings using technology that does not require significant investment (technology/internet infrastructure and human resources) by service providers. Nor does it place a burden and responsibility on refugees to purchase expensive smart phones, download internet applications, or have a high level of technical expertise to use the service (although we do acknowledge that the process of exchanging SMSs with the end user may seem time consuming).

Bearing in mind that SMS-based assessments in our study were conducted in a highly controlled environment (funding provided for reply SMSs, and close proximity to assessor), we call for future proof-of-concept investigations that utilize a cohort design with comparison groups to assess the reliability of face-to-face and SMS-based assessment over time. Once the technology has been shown to address refugee mental health needs effectively, innovations such as translating the QIDS into other languages to accommodate refugees without literacy in English, or developing application-based screening with pictures/voice options should be explored, thereby transcending language boundaries in clinical and social services settings.