Effectiveness and Acceptability of Conversational Agents for Sexual Health Promotion: A Systematic Review and Meta-Analysis

Abstract

Digital health interventions for sexual health promotion have evolved considerably alongside innovations in technology. Despite these efforts, studies have shown that they do not consistently result in the desired sexual health outcomes. This could be attributed to low levels of user engagement, which can hinder intervention effectiveness as users do not engage with the system enough to be exposed to the intervention components. It has been suggested that conversational agents have the potential to overcome the limitations of prior systems and promote user engagement through the increased interactivity offered by bidirectional, natural language-based interactions. The present review therefore provides an overview of the effectiveness and user acceptability of conversational agents for sexual health promotion. A systematic search of seven databases provided 4,534 records and after screening, 31 articles were included in this review. A narrative synthesis of results was conducted for effectiveness and acceptability outcomes, with the former supplemented by a meta-analysis conducted on a subset of studies. Findings provide preliminary support for the effectiveness of conversational agents for promoting sexual health, particularly treatment adherence. These conversational agents were found to be easy to use and useful, and importantly resulted in high levels of satisfaction, use and intentions to reuse, while user evaluations regarding the quality of information left room for improvement. The results can inform subsequent efforts to design and evaluate these interventions, and offer insight into additional user experience constructs identified outside of current technology acceptance models which can be incorporated into future theoretical developments.

Introduction

Digital health interventions (DHIs) are interventions delivered via digital technologies such as smartphones, websites, social media or text-messaging¹. DHIs have become increasingly popular for large-scale health promotion efforts as an innovative, cost-effective, and scalable solution for addressing key public health challenges such as staff shortages and budget constraints. They can be especially suited for the sexual health domain because there is the potential to reach at-risk groups e.g. adolescents ², ethnic minorities³, sexual minorities⁴, illicit drug-users^5,6 and sex workers^7,8 are less likely to seek professional care due to limited resources, poor quality of services and stigmatization.

DHIs for sexual health promotion have evolved considerably over time alongside increased internet and mobile device adoption⁹ as well as new technologies. Early DHIs leveraging short-message service (SMS) ^10–15 and digital media e.g. websites, video and CD-ROM ^10,16–18 became widespread and were well-received by users. However, positive results were observed largely for sexual health knowledge and attitudes and less so for behaviours such as treatment adherence, human papilloma virus (HPV) vaccine uptake and condom use^10,14,17,18. Moreover, intervention effectiveness tended to decline over time. Smartphones, in spite of their pervasiveness and functionalities, failed to gain positive responses from the community with regards to mobile phone applications intended to promote STI prevention and care¹⁹. The use of emerging technologies such as serious gaming, virtual reality and social media faced resulted in similar outcomes^20–23, whereby little to no effects were observed for most sexual health behaviours with the exception of testing uptake. Ironically, while the motivation behind the dynamic adoption of technological innovations was likely increased user engagement, several authors have attributed the observed low intervention effectiveness to reduced immersion and flow i.e., feeling fully involved and focused on the activity ²⁴. This is consistent with the notion that positive user engagement precedes positive interactions with intervention components, thereby leading to increased intervention efficacy ^25,26–29

Continuing in the footsteps of adopting new technologies, there has been a recent shift towards conversational agents (CAs) for delivering DHIs across healthcare domains such as substance abuse, mental health, exercise and even stress-reduction ^30–35. For the purposes of this review, we adopt the definition of CAs as systems that can simulate conversation with users through natural language such as written text or voice thus permitting automated two-way communication between the user and system ^35,36. Examples of CAs range from the well-known open-domain virtual voice assistants such as Siri and Alexa³⁷ to customer service chatbots available through commercial websites and social media platforms such as Facebook ³⁸and even embodied CAs which employ computer-generated avatars³⁹. It has been suggested that DHIs allowing two-way interactions can increase intervention efficacy by addressing both non-intentional and intentional forms of non-adherence to target health behaviours^40,41, and promoting user engagement by encouraging users to explore their attitudes and feelings in a more productive and personally relevant manner^42,43. Furthermore, CAs particularly hold promise as a more innovative way to communicate with younger users^44,45 given their high digital literacy and familiarity with chat applications. Given the rate at which the field of natural language processing is advancing, CAs can also increase engagement by understanding the user and providing intelligent, relevant communication at all times to different target populations⁴⁵.

In essence, it appears relevant to already examine the prospects that CA-based DHIs may offer for sexual health promotion. Furthermore, prior reviews on DHIs in this domain have primarily addressed effectiveness^{16–18,46–48}, often with emphasis on randomized controlled trials^10,46,47,49 or peer-reviewed literature¹². However, this approach may result in the exclusion of studies which could provide insight into the potential of CA-based interventions by assessing user acceptability or reporting preliminary findings given the rapid advances in conversational technology. As the effectiveness and user acceptability of DHIs are inextricably linked, the aim of the present systematic review is to summarize available evidence regarding both the effectiveness and acceptability of conversational agents for sexual health promotion.

Method

Literature Search

A systematic search of the literature was performed in October 2020 using PsycINFO, Web of Science, ACM Digital Library, IEEE Xplore, Medline, Embase and CMMC, not restricted by publication year or language. An updated search was performed in February 2021. Grey literature identified in those databases, including dissertations, theses, and conference proceedings, were also included for screening given the infancy of this field. Two sets of search terms were devised by the first reviewer (DB) and a librarian (SG) and customized for each selected database. The first set addressed conversational agents and included other related terms such as ‘chatbots’, ‘relational agent’, ‘virtual assistant’, ‘dialog system’ and ‘mHealth’. The second set addressed sexual health and included other related terms such as ‘HIV’, ‘sexually transmitted diseases’, ‘HPV’ and ‘syphilis’. The full search strategy can be found in the Supplementary Material. The protocol for this review was registered at the International Prospective Register of Systematic Review (PROSPERO; registration number CRD42021222969).

Inclusion and Exclusion Criteria

Included studies had to meet three criteria: (i) described conversational agents or synonymous systems that permitted two-way interactions that were fully automated (i.e. without any human mediation)¹, (ii) addressed any sexually-transmitted condition such as HIV/AIDS, HPV and other STIs or targeted aspects of sexual health promotion such as medication adherence and reducing risky sexual behaviour and (iii) described an evaluation applied to the technology, focusing on either health outcomes or end-user evaluations (e.g. acceptability, usability, or satisfaction, but not cost-effectiveness and cost-analysis outcomes). Forms of one-way communication, human-mediated communication and systems without response contingency (e.g. quizzes, ecological momentary assessment and computer-assisted self-interviewing) were excluded, as were studies which only evaluated the idea or content that will later be implemented, as the actual technology does not undergo evaluation. Reviews, meta-analyses, protocol papers and poster abstracts were excluded, as were citations with missing abstracts. No restriction was placed on study design given the varied and dynamic nature of the field.

Screening procedures were piloted by the first and second reviewers (DB and LH). Once a sufficient kappa value⁵⁰ was achieved (>0.6, indicating substantial inter-rater agreement), the initial screening of articles was conducted independently based on the information contained in the title and abstract. Conflicts were discussed between DB and LH, and unresolved conflicts were discussed with a third reviewer (GJ). The same procedure was applied for full-text screening. After this, citation tracking was conducted to ensure that all relevant studies were identified, resulting in 31 included studies (Figure 1).

Figure 1. PRISMA Diagram.

Data Extraction and Synthesis

Data extraction was conducted by DB and reviewed by GJ for completeness. The following data from the included studies were extracted: title, author, year, study design, sample size, target health behaviour, target population, interaction frequency, intervention duration, theoretical framework, technology platform and initiator and outcomes. Outcomes were classified as either health outcomes or user evaluations and were all summarized through a vote-counting strategy and are presented as a narrative synthesis of results. Health outcomes assessed in more than one study using a randomized controlled trial design were additionally analyzed through a meta-analysis. User evaluation outcomes, quantitative and qualitative, were organized around the components of a technology acceptance model (TAM), namely the DeLone and McLean Information Systems Success Model (D&M Model)⁵¹ - system quality, information quality, user satisfaction, actual use & intent to reuse and net benefits.

Meta-analysis

Health outcomes assessed in more than one study using a randomized controlled trial design were additionally analyzed through a meta-analysis. For the meta-analysis, random effect models were used. Per included study, we calculated the effect size d and its standard error using suggested formulas for mean differences and odds ratio ⁵² in MacOS Numbers. For studies that had multiple outcome measures, an average effect size d and standard error was calculated using suggested formulas⁵² and setting the multiple outcome correlation as r = .50. Heterogeneity was derived from the Q-statistic⁵³ and publication bias from funnel plots and Egger’s test⁵⁴. When publication bias was present, trim-and-fill analysis was conducted ⁵⁵. We conducted the meta-analysis for the five RCT studies including all outcomes, all objective outcomes, and all objective ART outcomes respectively (so excluding⁵⁶) with the dmetar, meta, and metafor packages in RStudio (version 2021.09.1) for MacOS.

Results

Overview of Studies

Table 1 provides a summary of the included studies (n = 31). Thirteen RCTs were identified, and the remaining utilized either pre-post or post- study designs. Social Cognitive Theory was cited most often as the underlying theoretical framework ^57–63, followed by the Information Motivation-Behavioral Skills model ^64–67 and Motivational Interviewing ^68–70. Seventeen out of 31 studies did not indicate an underlying theoretical framework.

Table 1. Overview of Included Studies.

Authors	Year	Study Design	Pilot	Health Eval	User Eval	Sample	Duration	Freq	Digital Platform	Initiator	Health Behaviour	Underlying Theoretical Framework
Amith et al.	2020	Post	No	Yes	Yes	Young adults	N/A	Once	Computer - App	User	HPV vaccine uptake	None
Amith et al.	2020	Post	No	No	Yes	PEP and PrEP users	N/A	Once	Computer - App	User	Increasing knowledge about PEP and PrEP	None
Amith et al.	2019	Post	No	No	Yes	Mothers of young children	N/A	Once	Computer - App	User	HPV vaccine uptake	None
Bonnevie et al.	2020	Post	No	No	Yes	16-25 year old Black and Hispanic women	18 months	N/A	Computer - Website	User	General sexual health	None
Chavez & Palfai	2020	Pre-post	Yes	Yes	Yes	Heavy Drinking MSM at risk	4 weeks	Thrice a week	Phone - SMS	System	Safe sex behaviours, reduction of drinking behaviour	Personalized Cognitive Counseling, Motivational Interviewing
Chernick et al.	2020	Pre-post	No	Yes	Yes	Adolescent (14-24 years) girls	10 weeks	Twice a week	Phone - SMS	System	Safe sex behaviours	Motivational Interviewing, Social Cognitive Theory
Christopoulos et al.	2018	RCT	No	Yes	Yes	HIV+ persons with history of poor retention/ART initiators	12 months	Thrice a week	Phone - SMS	System	ART adherence and retention in care	Behavioural Model for Vulnerable Populations, InformationMotivationBehaviour Skills model, Health Care Empowerment model, revised Stress and Coping Theory
Davis et al.	2019	Post	Yes	No	Yes	Black female adolescents	8 weeks	Daily	Phone - SMS	System	General sexual health	Social Cognitive Theory, Theory of Gender and Power
Dowshen et al.	2012	Pre-post	Yes	Yes	Yes	Youth living with HIV	24 weeks	Daily	Phone - SMS	System	ART adherence	None
Dworkin et al.	2019	Pre-post	No	Yes	Yes	Young HIVpositive African - American MSM	12 weeks	N/A	Phone - App	Both	ART adherence	Information- Motivation - Behavioural Skills Model of Antiretroviral Adherence
Dworkin et al.	2018	Post	Yes	No	Yes	HIV+ MSM aged 18-34 years on ART	N/A	Once	Phone - App	Both	ART adherence	Information- Motivation - Behavioural Skills Model of Antiretroviral Adherence
Fuchs et al.	2018	Pre-post	Yes	Yes	Yes	MSM PReP users	12 weeks	Weekly	Phone - SMS	System	PReP adherence	None
Garofalo et al.	2016	RCT	No	Yes	Yes	Poorly Adherent Youth Living with HIV (1629 years)	6 months	Daily	Phone - SMS	System	ART adherence	Social Cognitive Theory
Hardy et al.	2011	RCT	No	Yes	Yes	Poorly Adherent HIV+ individuals	6 weeks	Daily	Phone - SMS	System	ART adherence	None
Harris et al.	2010	Post	No	No	Yes	Individuals initiating or changing two medications on the HAART regimen	3 months	Daily	Phone - SMS	System	ART adherence	None
Ingersoll et al.	2015	RCT	Yes	Yes	Yes	Substanceusing nonadherent individuals	12 weeks	Daily	Phone - SMS	System	ART adherence	Information, Motivation and Behaviour Skills Model of Adherence, Social Action Theory
Jack et al.	2020	RCT	No	Yes	Yes	African-American women	12 months	N/A	Computer - Website	User	Safe sex behaviours	Motivational Interviewing
King et al.	2017	Pre-post	No	Yes	No	High-risk HIVpositive persons	12 months	Weekly	Phone - SMS	System	ART adherence	None
Liu et al.	2018	RCT	No	Yes	Yes	Young individuals at risk for HIV initiating PReP	36 weeks	Weekly	Phone - SMS	System	PReP adherence	None
Mendu et al.	2018	Post	No	No	Yes	Hispanic women in rural communities	N/A	Once	Computer - App	User	Cervical cancer education	None
Moore et al.	2015	RCT	No	Yes	Yes	HIV+ persons with cooccurring bipolar disorder	30 days	Daily	Phone - SMS	System	ART adherence	None
Moore et al.	2018	RCT	Yes	Yes	No	Persons living with HIV and meth use disorder	6 weeks	Daily	Phone - SMS	System	ART adherence, METH use	Health Belief Model, Theory of Planned Behaviour, Social Cognitive Theory
Moore et al.	2018	RCT	No	Yes	No	MSM and Transgender women	48 weeks	Daily	Phone - SMS	System	PrEP adherence	None
Naar-King et al.	2013	RCT	Yes	Yes	Yes	Youth living with HIV initiating ART	2 months	Once a month	Computer - App	User	ART adherence	Motivational Interviewing
Pot et al.	2017	RCT	No	Yes	Yes	Mothers of young girls	8 weeks	N/A	Computer - Website	User	HPV vaccine uptake	Theory of Planned Behaviour, Social Cognitive Theory
Rodrigues et al.	2015	Post	No	No	Yes	Treatment naïve people living with HIV	84 weeks	Daily	Phone - IVR	System	ART adherence	Social Cognitive Theory
Rodrigues et al.	2012	Pre-post	Yes	No	Yes	HIV+ individuals on ART	6 months	Weekly	Phone - IVR	System	ART adherence	Social Cognitive Theory
Salvadori et al.	2020	RCT	No	Yes	No	Individuals at risk for HIV (based on counsellor)	17.8 months	Daily	Phone - SMS	System	HIV retesting uptake	None
Swendeman et al.	2020	RCT	Yes	Yes	No	Persons living with HIV	6 months	Twice daily	Phone - IVR	System	ART adherence	None
van Heerden et al.	2017	Post	No	No	Yes	N/A	N/A	Once	Phone - IM	User	HIV Self Testing	None
Wells et al.	2015	Post	No	No	Yes	Hispanic women in rural communities	N/A	Once	Computer - App	User	Cervical cancer education	None

Note: RCT - Randomized Controlled Trial; PEP - Post-Exposure Prophylaxis; PrEP - Pre-Exposure Prophylaxis; HPV - Human Papilloma Virus; MSM - Men Who Have Sex with Men; HIV - Human Immunodeficiency Virus; ART - Antiretroviral Therapy

The most common target health behaviour was antiretroviral therapy (ART) adherence (15/31). In contrast to ART, only four studies focused on preventive treatment i.e. pre-exposure prophylaxis (PrEP) and post-exposure prophylaxis (PEP) knowledge, uptake and/or treatment adherence ^56,71–73. Other studies addressed general sexual health and safe sex practices ^{57,58,68,69,74}, HIV/STI testing ^75,76 and HIV risk factors such as heavy drinking ⁶⁸ and medication adherence for bipolar disorder ⁷⁷. HPV vaccine attitude and uptake was the subject of three studies, targeting either mothers ^59,78 or young adults ⁷⁹. Studies predominantly (24/31) took place in the United States. The populations of interest were broadly categorized as individuals at risk for HIV ^{56–58,68,69,71,72,74,76,78–81} and HIV-positive individuals who are initiating or taking ART, particularly those who have sub-par adherence ^{60–62,64–67,70,73,77,82–86}. Within these, groups which were targeted often were young people ^{56,57,62,65,66,70,79,84}, ethnic minorities ^{58,65,66,74,80,81}, sexual minorities^{65,66,68,71,73}, women ^{57,58,69,74,80,81} and substance users^64,68,73.

Mobile phone was the most common platform to deliver the interventions (22/31), which included short-message service (SMS)^{56–58,62–64,67,68,71,73,76,77,83–86}, interactive voice response (IVR)^60,61,82, instant messaging (IM)⁷⁵ and smartphone application (app)^65,66. The remaining interventions were delivered via computer, either through websites^59,69,74 or desktop/tablet applications^{70,72,78–81}. While all of the included systems permitted two-way interactions,seventeen out of 31 studies allowed multi-turn interactions i.e., either the CA or user was able to respond more than once within the same conversation^{57,58,62,65–70,72,74,75,77–81} and the remainder which were one-turn-only interactions ^{56,59–61,63,64,71,73,76,82–86}. Systems generally initiated interactions with users at fixed times e.g. daily ^{58,60–64,73,76,77,82,84–86}, twice a week ⁵⁷, thrice a week ^67,68, weekly ^56,60,71,83 and monthly ⁷⁰. Some were user-initiated ^59,65,69,74 while the rest were only used once to assess user acceptability outcomes.

Out of 31 studies, 26 looked at one or more user acceptability outcomes and 20 studies addressed intervention effectiveness through appropriate health outcomes.

Effectiveness

Out of 11 studies that assessed antiretroviral therapy outcomes, studies which used self-report measures^{62,65,70,77,82,84,86} found support for intervention effectiveness whereas those employing pill count^{63–65,77,82,86} and biomarker measures^{62,67,70,82–84} did not support this. Pre-exposure prophylaxis outcomes were assessed in three studies also using self-report⁷¹, pill count⁷¹ and biomarker measures^56,73, all of which indicated significant effects. Positive findings were observed for multi-dimensional attitude towards HPV vaccine in two studies^59,78, of which one found an additional near-significant effect for HPV vaccine uptake behaviour⁵⁹. For condom use, two studies found no significant effects for condom use behaviour ^57,68, of which one found an improvement in condom use attitude but not intention⁶⁸. One intervention was targeted at promoting a range of safe sex practices for women during pre-conception and was found to be effective for reducing the number of risks associated with (sexually transmitted) infectious diseases ⁶⁹.

Meta-analysis

The included RCTs looked at medication adherence for either antiretroviral therapy ^62,67,70,77 or pre-exposure prophylaxis⁵⁶. Table 2 summarizes the results for the conducted meta-analyses. For the five RCTs (n = 582), there was a small and significant effect on medication adherence, d+ = 0.23, 95% CI [0.037; 0.4213], p = .030. favoring the chatbot intervention (Figure 2). This effect was not present after repeating the analysis for only objective outcomes (d+ = 0.19, 95% CI [-0.096; 0.484], p = .137) (Figure 3), and with only ART studies (d+ = 0.10, 95% CI [-0.232; 0.424], p = .419) (Figure 4). The funnel plots did not indicate publication bias (see Figures 5, 6 and 7).

Table 2. Summary of Meta-Analysis Results.

	k	n	Effect Size			Heterogeneity			Egger’s Test
			d	95% CI	p	Q	I2	p	p
All RCTs (All Outcomes)	5	582	0.23	0.037 to 0.421	.03*	4.48	10.7%	.345	.405
All RCTs (Objective)	5	582	0.19	-0.096 to 0.484	.137	5.22	23.3%	.266	.144
ART-Only RCTs (Objective)	4	461	0.1	-0.232 to 0.424	.419	2.28	0.0%	.516	.714

Note: (1) All RCTs (All Outcomes) – included all RCTs addressing antiretroviral therapy (ART) and preexposure prophylaxis (PrEP), utilised self-report, pill count and biomarker measures; (2) All RCTs (Objective) – included all RCTs addressing ART and PrEP but only utilised pill count and biomarker measures; (3) ART-Only RCTs (Objective) – excluded 1 study conducted on pre-exposure prophylaxis use and only utilised pill count and biomarker measures

Figure 2. Forest Plot – All RCTs with all measures.

Figure 3. Forest Plot – All RCTs with objective measures only.

Figure 4. Forest Plot – RCTs addressing ART with objective measures only.

Figure 5. Funnel Plot - All RCTs with all measures.

Figure 6. Funnel Plot - All RCTs with objective measures only.

Figure 7. Funnel Plot - RCTs addressing ART with objective measures only.

User Acceptability

System Quality

Nine out of 26 studies evaluated system quality ^{60,65,69,72,75,78,79,85,86}. While ease of use was the most common measure (7/10), system quality was also evaluated through overall usability ⁷⁹, pragmatic quality ⁷² and response speed ^75,78. Users generally found the systems straightforward and easy to use. This was attributed to technological capabilities such as quick replies in instant messaging ⁷⁵ and the general familiarity of mobile phone interfaces which ensured that the chatbot understood them. Interestingly, response speed was evaluated negatively in two studies, whereby users found it to be either unrealistically fast ⁷⁵ or too slow and not efficient enough for a machine ⁷⁸.

Information Quality

Information quality was assessed in twelve out of 26 studies along content relevance ^58,66–69, content quantity ^58,66,85, clarity ^66,78,81, language style ^58,66,75, interaction intensity (depth, frequency and duration) ^58,61,62,68 and repetitiveness ^67,78,85. While participants largely found the content relevant and useful for the target behaviour ^57,58,68,69 (e.g. “these question sare things that all girls think about...made me think about my behaviours⁵⁷”), they voiced the desire for additional personalization of content^67,68. Across three studies that assessed content quantity ^58,66,85, two reported that users wanted additional content on other health topics (e.g. side effects of ART ⁶⁶) and on non-health topics (e.g. communication and relationships ⁵⁸). Language style was evaluated in three studies, two of which found that language style should be more appropriate. For example, it is important to avoid sensitive phrases (e.g. using AIDS interchangeably with HIV) as well as graphic images portraying sickness⁶⁶. For voice-based systems, users discouraged any harsh or judgmental intonation and wanted “straight talk” like from friend or relative^58,66. Van Heerden and colleagues ⁷⁵ instead found that users thought the language was too formal and incongruent with real life conversations. Qualitative feedback indicated that the clarity of some systems can be further improved by rewording and using visuals to complement the verbal and/or audio-visual dialogue ^66,78,81. Studies looking at repetitiveness ^67,78,85 found that some users were irritated when the system did not exhibit the variety that is characteristic of natural conversations. Users expressed room for improvement regarding interaction intensity in four studies ^{58,61,62, 68} - they wanted the conversations to occur more often i.e. daily or more and last longer ^58,61 while others would have appreciated more or less messages sent to them depending on their preferences ^62,68.

User Satisfaction

Nine out of 26 studies assessed overall user satisfaction. Two studies ^68,70 made use of the Client Satisfaction Questionnaire while others enquired about satisfaction ^{59,62,64,85,86} or acceptability ^57,81 using one or more survey items. Across all studies, users reported above-average scores and that they found their experiences with the technology satisfying or enjoyable.

Use

Intent to reuse the system in the future was assessed in eight studies and most users (around 78% across studies) responded positively, expressing that they would like to continue receiving the intervention after the study or be open to using such a system in their daily lives. Actual use was assessed quantitatively in 10 studies through message response rate for SMS-based systems ^{57,62,64,68,77,84,85} and through usage metrics for smartphone applications and websites ^59,65,74. The average message response rate was around 69%, ranging from 47% to 68% on the lower end ^84,85 to 92% on the higher end ^68,77. Usage metrics indicated that an average of 88% participants accessed the systems^59,65 at least once and each interaction lasted around 10 minutes, and another system⁷⁴ received 4,390 topic-relevant messages with an average of three questions per session, indicating reasonable use of the system.

Net Benefits

Perceived net benefits were evaluated in more than half of the studies (14/26) through perceived usefulness ^{56,57,60,61,64,68,71,75,78,80,84–86} and the likelihood of recommending the system to other individuals ^{56,57,62,64,65,67,81,84}. Qualitative feedback from users indicated that the systems were useful for promoting a range of sexual health behaviours such as condom use (“these questions are things that all girls think about...”), HIV testing (“it could save time not having to wait at a clinic for a counsellor”) and HPV vaccine uptake (“.provided useful information and reinforced important points”). Some studies found that the systems targeting treatment adherence were only useful if users were facing difficulties with adequate adherence ^61,71, consistent with the otherwise positive evaluations of perceived usefulness regarding systems targeting either individuals initiating treatment ^56,85 or exhibiting poor adherence ^62,64,85,86. Reminders were cited as being most useful feature by providing different strategies ⁶⁸, minimizing forgetfulness when they were busy or at work ^61,84 and that the reminders did not stop until they texted back ⁸⁶. Overall, users were also highly likely (86% across all studies) to recommend the systems they used to others who are HIV-positive, to a friend, or to others in general.

Additional User Acceptability Outcomes

There were constructs identified in the set of user acceptability outcomes that did not fall under any component of the D&M Model and are therefore summarized below. Constructs associated with privacy and anonymity were assessed in five studies^{56,60,61,66,75}, whereby a minority of users voiced a desire for additional measures (e.g. the ability to hide the application screen quickly, minimize attention from alerts and reminders) to avoid unintentional disclosure in three studies ^56,61,66. Two studies^69,81 assessed trust in the system and received positive feedback from users. Questions regarding general feelings of comfort and the lack of stigma were administered to users in five studies^{57,65,66,69,75} out of which four revealed that users indeed felt safe and comfortable in their interactions with the system. One study ⁶⁵ found that a small number of users faced instances of potential embarrassment and stigma when using the system in public and near their friends. In five studies, users were asked about the extent to which they felt emotionally supported, or cared for, by the system, all of which reported positive findings^{57,61,65,67,71}. Out of the seven studies that looked at social presence, users expressed desire for increased social presence or actual human interaction in three studies ^58,78,80 while the remainder, most of which utilized a static or embodied avatar, reported good or sufficient social presence ^66,75,81,85.

Discussion

Digital health interventions (DHIs) for sexual health promotion are becoming increasingly commonplace and are particularly attractive because at-risk groups are often unable to or are reluctant to seek out professional advice. In the spirit of adopting new technologies, DHIs using conversational agents (CAs) have begun to receive more attention for their added capacity to imitate natural interactions with humans. The CAs included in this review exhibited a marked variety in how the technology interacted with the users. Of interest were the relatively large number of CAs that allowed multi-turn interactions, which come across as more natural and are characterized by increased interactivity and feedback. These were particularly pronounced in the more recent years, likely explained by the rapid technological advances that have been made in the field of artificial intelligence⁸⁷. Understandably, the included studies were largely pilot studies which indicated the infancy of this growing field but resulted in the lack of rigorous study designs utilizing appropriate control groups that would have aided in more empirical analysis. As the interest in CAs for sexual health is evidently rising, the present review situates itself well in summarizing the available evidence of their effectiveness and acceptability.

Most of the studies targeted medication adherence, either antiretroviral therapy (ART) or pre-exposure prophylaxis (PrEP), and meta-analytic findings supported the effectiveness of CA interventions for adherence when considering both self-report and objective measures as well as both medications. However, this result became non-significant upon excluding self-report measures and the single PrEP study, suggesting the need to consider how these systems can result in more tangible improvements for ART adherence. In general, CAs targeting ART adherence fall under the umbrella of treatment⁸⁸ and were able to help individuals who already have HIV to manage their symptoms through interactive reminders and information provision. In contrast, there were markedly fewer interventions addressing prevention⁸⁸ of sexual health-related diseases, namely pre-exposure prophylaxis (PrEP), condom use, human papilloma virus (HPV) vaccine uptake and sexually-transmitted disease (STD) testing. Based on this limited number of studies, CA interventions resulted in positive outcomes for attitudes towards condom use and HPV vaccination uptake and testing behaviour. Given the difficulties in getting individuals to engage in precautionary behaviours^89,90, additional studies are needed to explore how CAs can effectively support such behaviours.

Through the lens of the DeLone and McLean Information System Success Model⁵¹, the CA interventions were found to be acceptable to users in terms of ease of use, perceived usefulness, satisfaction and intent to reuse, which is likely to translate into actual use according to the D&M model ^91–93 and other technology acceptance models^94–97. In addition to reminders, CAs were seen as being capable of providing on-demand emotional support and useful information in an anonymous manner without human contact, consistent with proposed drivers of CAs for healthcare⁴⁵. However, they were found to be lacking in aspects of information quality such that users desired additional, personalized, and clearer content communicated in an appropriate language style. While not included in traditional technology acceptance models as a separate construct, the heavy reliance on textual content and communication principles⁹⁸ in conversational systems lends to the importance of information quality, a point that is further supported by the number of included studies that evaluated this construct in some form.

The study identified an additional set of constructs that may play a role in user acceptability within this domain. Trust and privacy can be thought of as contemporary challenges that have permeated emerging technologies ^99–101 and have been discussed in other extended TAM models within^102,103 and outside of healthcare ^104–106. Users both expected and were largely satisfied with the degree of privacy and trustworthiness exhibited by the systems, although the demand for security appears abundantly strong that additional features may be needed to motivate long-term use. Social presence was found to be sufficient in only half of the studied systems, which could be attributed to the use of multimedia and embodiment through avatars^43,107,108. An interesting issue arises in incorporating social presence into a system that is often touted for its capacity to enable anonymous, “non-human” and therefore self-disclosing interactions^45,109–111, suggesting the need to achieve a delicate balance. While social presence has been implicated in the user acceptance of conversational agents in other areas ^{38,108,112 113,114,115,116 117}, its role in the sexual health domain remains to be disentangled. Comfort and emotional support have not received as much attention in the literature but these findings suggest that they may be important in specific domains such as domestic violence^118,119, sexual issues^120,121 and mental health¹²² where individuals need to feel safe and accepted while engaging with the system. While this review underscores the potential importance of these constructs, future studies can explore their role in user acceptance and inform their inclusion in extended TAM models for sexual health and related domains.

Conclusion

Despite the limited body of evidence, these findings support the notion that CAs for sexual health may not only be effective but that users also find these useful and acceptable for a range of sexual health behaviours. While CAs are already capable of supporting antiretroviral therapy adherence through simple two-way interactions, more studies are required to understand how the potential of CAs can be leveraged for more complex behaviours. This review also emphasizes the value of rigorous, holistic, and mixed-method evaluations of CA-based DHIs to gain deeper insight into how the intervention components are perceived by users as a driver of intervention efficacy. To that end, the findings serve as a good starting point for how we might go about enhancing the user experience for these interventions and highlight the need for theoretical developments regarding technology acceptance models which are more applicable to sensitive domains. The question still remains for further research as to whether and under what circumstances individuals would voluntarily adopt CAs outside the research context and in what way they can be reached in practice.

Data Availability

The authors confirm that the search strategy used in this study is available within the article and/or its supplementary materials. The data supporting this systematic review and meta-analysis are from previously reported studies and datasets, which have been cited. The processed data are available from the corresponding author upon request.