Crowdsourcing to improve HIV and sexual health outcomes: A scoping review

Abstract

Purpose of Review

This review synthesizes evidence on the use of crowdsourcing to improve HIV/sexual health outcomes.

Recent Findings

We identified 15 studies, including four completed randomized controlled trials (RCTs), one planned RCT, nine completed observational studies, and one planned observational study. Three of the four RCTs suggested that crowdsourcing is an effective, low-cost approach for improving HIV testing and condom use among key populations. Results from the observational studies revealed diverse applications of crowdsourcing to inform policy, research, and intervention development related to HIV/sexual health services.

Summary

Crowdsourcing can be an effective tool for informing the design and implementation of HIV/sexual health interventions, spurring innovation in sexual health research, and increasing community engagement in sexual health campaigns. More research is needed to examine the feasibility, acceptability, and effectiveness of crowdsourcing interventions, particularly in low- and middle-income countries.

Introduction

Despite remarkable advances in HIV prevention and treatment efforts, millions of people are infected each year. Expert-driven approaches to solving this complex public health challenge have been met with limited success. Such approaches often begin with a team of experts proposing an idea that they believe will reduce HIV transmission in a specified population (see Figure 1). These ideas often come from a variety of sources, including theory, previous literature, and formative research methods. Next, the experts, who are usually not members of the communities that they seek to serve, consult with local stakeholders to solicit feedback on the ideas that have been generated in hopes of increasing the likelihood of intervention acceptability, feasibility, and efficacy. While this approach has been effective for some HIV outcomes in the short-term, it has been less effective in producing the long-term behavioral changes necessary to end the global HIV pandemic.

Figure 1.

Comparison of crowdsourcing (right) and conventional (left) approaches.

In recent years, researchers, practitioners, and national and global health organizations have acknowledged the value of participatory approaches to HIV prevention, treatment, and care[1]. Crowdsourcing is such an approach and has the potential to engage difficult-to-reach populations in sexual health research. Crowdsourcing involves a group of non-experts and experts working together to solve a problem and then sharing solutions with the public [2]. It differs from conventional social marketing in several ways (see Figure 1). Crowdsourcing begins with the crowd, which includes key populations, experts, amateurs, and other individuals. The crowdsourcing organizers pose a question to the crowd, and the crowd provides potential solutions. In this way, crowdsourcing enhances community engagement, which may be less common in social marketing approaches. Social marketing approaches often rely upon the expertise of public health researchers and practitioners, which may be limited due to variability in education, training, and contextual experience. Unlike social marketing methods, crowdsourcing promotes innovation by reducing the likelihood of groupthink or cognitive inflexibility, in which novel ideas are inhibited due to experts’ tendency to use previous work to guide the direction of future projects.

Researchers have used crowdsourcing to engage experts and non-experts in a number of fields, including data processing [3], participant surveying[4], surveillance and monitoring [5, 6], and participant engagement [9]. While there are a number of crowdsourcing models, open contests are the most common. Open contests solicit innovative contributions from the community, judge submissions, reward finalists, and implement or share finalist ideas. The International AIDS Society, for example, has used crowdsourcing contests to solicit logo designs for their biennial conferences[7]. These contests are open to the public and enable a diverse group of individuals to play prominent roles in designing logos for the largest HIV conference in the world.

Crowdsourcing has several advantages. First, as a bottom-up approach, it provides a mechanism through which the public can contribute to intervention development by sharing their ideas about how to solve important problems affecting their communities[8]. Second, crowdsourcing has the potential to tap into established in-person and online social networks. In doing so, crowdsourcing can solicit input from a large number of people in a relatively short time period[9]. Third, crowdsourcing methods are often less expensive than traditional social marketing approaches [10], making them particularly feasible in diverse settings, including low-resource settings.

Despite these advantages, crowdsourcing is under-utilized in HIV/sexual health research and programming[11, 12]. The purpose of the current review is threefold. First, we summarize the results of previous randomized controlled trials (RCTs) aimed at evaluating the effectiveness of crowdsourcing methods in improving HIV/sexual health outcomes. Second, we describe studies that have used crowdsourcing methods to solicit ideas on ways to improve HIV/sexual health outcomes. Lastly, we provide suggestions for future research to increase awareness and use of crowdsourcing in HIV/sexual health research.

Methods

We used Arksey and O’Malley’s framework for conducting a scoping review, which consists of the following five stages: 1) identification of a research question; 2) identification of relevant articles; 3) article selection; 4) data charting; and 5) collating, summarizing, and reporting the results. Scoping reviews summarize the key literature on a topic to convey the breadth and depth of evidence in a particular field and seek to elucidate gaps in the literature. They are not intended to review studies systemically and are often used when the literature on a particular topic is sparse.

In this review, we synthesize the evidence on the use of crowdsourcing to improve HIV/sexual health outcomes. Specifically, studies were included in this review if they focused on: 1) RCTs evaluating the effectiveness of crowdsourced interventions in improving HIV/sexual health outcomes; and 2) observational studies using crowdsourcing to promote sexual health, solicit ideas to improve sexual health, or describe the process involved in implementing a sexual health-based, crowdsourcing intervention.

Our search included studies that were published prior to July 16, 2018, that focused on crowdsourcing, with an emphasis on HIV/sexual health outcomes. We searched the following databases: PubMed, EMBASE, CINAHL and the Web of Science. We used the following search terms: HIV, sexual health, sexually transmitted disease/infection, crowdsourcing, and open/challenge contests. Detailed searching algorithms are listed in supplement A. After removing duplicate publications, the titles and abstracts of the remaining articles were evaluated for relevance by two independent reviewers (WT and TR), who also assessed each full-text article. Discrepancies were assessed by a third reviewer (CW) for discussion and resolution. We also searched the reference lists of included articles, grey literature, government reports, policy documents, NIH RePORTER, and ClinicalTrials.gov.

Results

This scoping review synthesizes studies that used crowdsourcing to improve HIV/sexual health. Our initial search generated 413 articles. A total of 15 studies were selected, including four completed RCTs[10, 13–15], one planned RCT, nine completed observational studies[16–24], and one ongoing observational study.

Evidence from randomized controlled trials

We included four completed RCTs [10, 13–15] and one planned RCT. These studies assessed the effectiveness of using crowdsourced videos[15, 10, 14], messages[15, 13], and health services[13] to improve HIV/sexual health outcomes. Table 1 provides a summary of included studies, along with additional details regarding participant recruitment and outcomes.

Table 1.

Summary of four included randomized controlled trials that aimed to evaluate the effectiveness of crowdsourced interventions in promoting HIV/sexual health services.

Year	Location	Participants and participant recruitment	Crowdsourced interventions, and how the crowd contributed to intervention development	Control	Results (primary outcome)
2014	China	721 MSM who have not tested for HIV. The participants were recruited by putting recruitment ads on three Chinese MSM web portals (in the northern, southern, and eastern regions of China).	Crowdsourced video. The crowdsourcing contest solicited one-minute videos from 11 community-based organizations focused on gay communities in China. Videos were judged by stakeholders with ties to the gay community.	Health marketing video	HIV testing uptake was similar between the crowdsourced arm (37%, 114/307) and the social marketing arm (35%, 111/317). The estimated difference between the interventions was 2.1% (95% CI: −5.4% to 9.7%)
2015	China	1173 MSM who engaged in condomless sex in the past three months. The participants were recruited via banner advertisements on a gay dating app, WeChat and Weibo platforms.	Crowdsourced video. The study team solicited videos using the group website, and through brief presentations during course lectures and by hosting interactive feedback sessions on college campuses, without giving any examples or other resources. The crowd submitted entries online.	Social marketing video	At three months, 196/376 (52.1%) and 206/415 (49.6%) individuals reported condomless sex in the crowdsourced and social-marketing arms (estimated difference: +2.5%, 95%CI: −4.5 to 9.5%).
2016-2017	China	1381 MSM who have not tested for HIV in the past three months. The participants were recruited through a gay dating app, by sending a survey invitation to registered users in the eight selected cities.	Multicomponent. A multimedia HIV testing campaign, an online HIV self-testing distribution model, and a local community-based testing promotion campaign. Members of the crowd participated in one of three ways: 1) submitted entries to the crowdsourcing contest, 2) participated in a team-based, 72-hour designathon, and/or 3) served as judges for the national wide crowdsourcing contest.	Standard of care	The crowdsourced intervention package improved recent HIV testing (in three months) uptake among Chinese MSM, with an increasing proportion of 8.9% (95% CI: 2.2 to 15.5). The intervention specifically increased recent HIV self-testing, with a risk ratio of 1.89 (95% CI: 1.50-2.38).
2018	China	556 MSM without HBV/HCV testing or HBV vaccination. Participants were recruited using online advertisements placed on multiple MSM social media platforms.	Crowdsourced images and one-minute videos (2 each). The images and videos were solicited using a crowdsourcing contest. Participants were asked to provide suggestions on ways to improve images and videos voted as top entries as part of a previous open contest.	Standard of care	The confirmed HBV and hepatitis C virus (HCV) test uptake was similar between the intervention and control arms. The two groups also had similar rates of HIV testing.

One study used a crowdsourcing contest to solicit videos from the public aimed at promoting first-time HIV testing among Chinese men who have sex with men (MSM) and transgender women [10]. The researchers posted a call for videos on three Chinese MSM web portals used for social networking, partner finding, news sharing, and advertising and hosted a teleconference to increase contest awareness. They convened a multisectoral panel of judges to evaluate video entries, which were limited to one minute. In this crowdsourcing contest, the crowd was composed of MSM community-based organizations in China. A total of 721 MSM who never tested for HIV were randomly assigned to watch either the crowdsourced video or a social marketing video created by a local public health experts for the same purpose [10]. The results of the RCT indicated that the crowdsourced video was just as effective as the social marketing video in encouraging first-time HIV testing among Chinese MSM and transgender women. Moreover, cost data showed that the crowdsourcing intervention was 45% less expensive than the social marketing intervention [25]. Exceptional videos were disseminated through MSM community-based organizations.

In another RCT, the researchers evaluated the effectiveness of utilizing an open contest to improve condom use among Chinese MSM in 2015 [14]. As in the trial described above, the researchers posted an open call for one-minute videos on a Chinese MSM web portal and convened a judging panel to evaluate the videos using a standardized set of criteria. In this crowdsourcing contest, the crowd included individuals who could create short videos in Chinese. The single video with the best score was then formally evaluated in an RCT. A total of 1173 MSM who engaged in condomless sex in the past three months were randomly assigned to either watch the one-minute crowdsourced video or a social marketing video developed by a marketing company in China. Results indicated that the crowdsourced video was non-inferior to the social marketing video in promoting condom use among Chinese MSM. Moreover, the per unit cost of the crowdsourced video was less than the social marketing video ($58 vs. $84).

One study used a stepped wedge RCT to evaluate the efficacy of a multicomponent crowdsourcing intervention in promoting HIV testing among Chinese MSM. The intervention consisted of three components: 1) a national open contest soliciting images and concepts to promote HIV testing through social media and in-person events; 2) a 3-day regional designathon in which five-member teams developed an HIV testing strategy; and 3) local contests soliciting stories about HIV testing experiences to promote community engagement and HIV testing [13]. These contests were open to anyone to participate but intended to particularly engage MSM in the eight selected cities. To recruit participants, the study team partnered with a large social networking application for Chinese MSM and sent a study invitation to all registered users seeking men who had not been tested for HIV in the past three months. A total of 1381 MSM from eight Chinese cities (clusters) were eligible for the study and randomly assigned to one of four groups that received the same intervention at four different time periods [13]. The results of the trial suggested that the crowdsourced intervention package led to improved uptake of HIV testing (estimated RR 1.43, 95% CI 1.19 – 1.73) among Chinese MSM. This study also found that crowdsourcing helped to facilitate HIV self-testing and community engagement in promoting HIV testing among Chinese MSM.

One ongoing, web-based study aims to evaluate the effectiveness of a crowdsourced intervention on hepatitis B and C testing among Chinese MSM[15]. During the intervention development stage, crowdsourcing was used to solicit images and one-minute videos for Hepatitis B and C testing and ideas to reduce stigma. Then, 556 men who had never been vaccinated for the hepatitis B virus (HBV) were recruited and randomized into either the crowdsourced intervention group or the control group. The intervention included two images, two videos, and a participatory activity to provide feedback on the images and videos. The control group did not receive an intervention [15]. Analysis of this RCT is ongoing.

Lastly, we identified one planned study aimed at evaluating the effectiveness of crowdsourcing interventions in improving HIV self-testing. This proposed study uses open contests and apprenticeships to develop new HIV self-testing services for Nigerian youth (UG3HD096929). This crowdsourcing project is focused on Nigerian youth aged 14-24 years old. This study seeks to convene a panel of judges to select two HIV self-testing strategies to pilot in an effort to assess trial readiness and evidence of efficacy. The specific aims are to (1) use open contests and apprenticeship to develop new HIV self-testing services; (2) evaluate the effectiveness of two participatory interventions promoting HIV prevention services among at-risk youth; and (3) determine the effectiveness of an integrated participatory, crowdsourced intervention on HIV prevention outcomes using a stepped wedge pragmatic RCT design.

Evidence from observational studies

We identified ten observational studies, including one ongoing study. These studies used crowdsourcing to inform policy, research, and/or intervention development [16–24].

We identified one study that used crowdsourcing to inform sexual health policy. Specifically, the purpose of this study was to develop a strategy to better engage youth in national, regional and global decision-making processes on HIV [17]. To do this, the UNAIDS Secretariat launched CrowdOutAIDS, a web-based, participatory policy project that used social media and crowdsourcing to help youth identify barriers to youth leadership and engagement in the AIDS response and create solutions. The participatory process included four steps: connect (reach out to youth), share (facilitate discussion), find solutions (analysis and voting), and collective action (establish a drafting committee). CrowdOutAIDS reached more than 5000 youth from 79 countries. This process identified three major priorities and six strategic actions regarding HIV-related programming for youth and informed the development of UNAIDS policy priorities. Moreover, this project also identified youth networks and organizations that could be leveraged to implement the proposed programs.

We identified five observational studies that used crowdsourcing in sexual health research. One research group conducted two studies in Australia that used online contests to solicit public input on herpes simplex virus (HSV) research [19, 16]. The first study sought videos on strategies to destigmatize HSV and evaluated whether the videos successfully created effective messages [19]. Participants were asked to submit 30-second videos presenting their ideas. A total of 103 entries were submitted, and after judging, six finalists were selected and awarded prizes. Thematic analysis indicated that participants used five strategies to destigmatize HSV: education, normalization, promoting disclosure, challenging negative perceptions, and articulating moral indignation[19]. The study concluded that using crowdsourced videos to reduce HSV-related stigma could be ineffective and possibly do more harm than good[16]. Another study retrospectively analyzed 63 videos from the same contest to identify lay perspectives on HSV disclosure[16]. Specifically, they analyzed three components of disclosure: rationale, approach and setting, and timing based upon the type of personal relationship. These public perspectives identified many creative methods for HSV partner disclosure and could be used by physicians responsible for partner services[16].

In a U.S. study, contestants submitted videos or images that embodied the significance of HIV cure research through an online platform[22]. Most participants were African American youth aged 18-23 years. Of the 144 potential contestants, 39 people submitted 32 entries. Social media analytic data showed that 684 people followed the website; there were 2233 unique visits to the online platform; there were 585 unique views of the videos, and the site reached 80,624 unique users. The themes for the contest entries emphasized the importance of community engagement in promoting HIV cure research. Based on this work, the authors concluded that crowdsourcing could increase community engagement in HIV cure research.

A Chinese study described two open contests: “Testing Saves Lives” and “Sex+Health”[21], “Testing Saves Lives” solicited one-minute videos from community-based organizations (CBOs) focused on promoting HIV testing in China. The video contest was publicized using the study website, direct emails to the leadership at CBOs, and through two open Skype calls. Entries were judged based upon their ability to: generate interest in HIV testing, reach untested individuals, and engage the community. The video contest was intended to: 1) empower CBOs to more effectively utilize social marketing to promote HIV testing; 2) promote multisectoral collaboration; and 3) appeal to youth from key populations. “Sex+Health” was developed to encourage Chinese youth up to age 30 to develop images aimed at sparking conversations about sexual health. Entries were judged based upon their relevance to sexual health and ability to engage Chinese youth[21]. The contest was promoted using a combination of in-person events at four high schools and universities, the US Consulate in Guangzhou and via social media. A total of 96 images were submitted for the “Sex+Health” contest over 39 days. Entries in the top five were displayed on the study team’s website for six days to enable members of the public to select the winning entry.

A Ugandan study used crowdsourcing in a nationwide mass media campaign aimed at promoting safe sexual practices among adolescents. Messages focused on abstinence, condom use, and reducing one’s number of sexual partners[23]. During “Hits for Hope,” musical acts were invited to participate in a competition that required them to compose and perform an original song focused on HIV prevention. A total of 80 groups from 10 districts performed and a panel of judges comprised of Ugandan youth selected the winning song. This song was recorded and distributed to taxi drivers, youth centers, and sold commercially. Surveys of 1,681 Ugandan youth revealed that, after the campaign, condom use increased from 46% to 69%. Moreover, fewer youth reported that they were unsure where to find condoms (from 42% to 31%) [23].

We identified a study that used crowdsourcing to inform HIV/ sexual health programs. In a multi-country open contest entitled, “Scenarios from Africa,” the researchers solicited storylines from young Africans for short fiction films intended to educate their communities about HIV [20]. Between 1997 and 2005, more than 100,000 young people from 37 countries participated in the contest. In 2005 alone, a total of 22,894 stories were submitted by 63,327 contest participants. After judging, 30 stories were selected as finalists. These stories were used to inform the scripts for short fiction films and translated into 25 languages. Resulting films were then disseminated through sub-Saharan Africa countries that contributed submissions [26, 27].

Crowdsourcing methods were used to maintain an internet-based interest/advocacy sounding board that was developed to respond to parents who shared apparent instances of group B streptococcal (GBS)-linked reproductive infections, along with comments and suggestions regarding GBS disease policies and prevention [18]. Participation in the program was associated with improved knowledge of the risk of sexually-transmitted GBS and the unreliability of intrapartum antibiotic prophylaxis strategies.

We identified one completed and the ongoing observational study aimed at improving PrEP uptake among MSM [24]. In Bangkok, Thailand, a crowdsourcing contest was used to generate messages and to promote PrEP among young, high-risk MSM and transgender women. During the judging phase of the contest, one single crowdsourced video was viewed and judged by 22,779 people, and the majority of the surveyed MSM and transgender women indicated that this video was designed for them [24]. Another crowdsourced PrEP promotion study is underway in Baltimore [28, 29]. This study uses an open contest and a neuroimaging technique to develop and evaluate pre-exposure prophylaxis (PrEP) promotion messages for high-risk MSM in Baltimore, MD.

Policy and research implications

Our scoping review identified a number of policy and research implications for using crowdsourcing methods to improve HIV/sexual health outcomes.

First, our scoping review suggested that most crowdsourced HIV/sexual health studies were conducted in high-income or upper-middle-income countries. As a bottom-up approach[21], with minimal costs[10] and strong community engagement[30], crowdsourcing is a promising method that could lead to improvement in current HIV/sexual health services in LMICs. However, policies and strategies for promoting this method in LMICs are lacking. People residing in LMICs are disproportionately affected by HIV and other sexually transmitted infections. As such, crowdsourcing could be used to identify and implement policies that are responsive to community needs, while also building upon their assets.

Second, crowdsourcing can be further used to promote other HIV/sexual health services, including HIV self-testing and pre-exposure prophylaxis (PrEP). The existing crowdsourcing studies mainly focused on promoting HIV testing [10, 13], condom use [14] and other sexual health services[16, 26]. However, crowdsourcing was rare to be used for promoting new HIV prevention tools, such as HIV self-testing and PrEP. In a study conducted in China, crowdsourced HIV self-testing was one of three intervention components of a stepped wedge trial to improve HIV testing among MSM in China, and the study found that the intervention specifically increased recent HIV self-testing [13]. Additional studies aimed at promoting and evaluating the use of crowdsourcing HIVST and PrEP are needed.

Third, research aimed at promoting the dissemination and scale-up of crowdsourcing methods are needed. The majority of our knowledge regarding outcomes associated with using crowdsourcing to promote sexual health messages is based on studies that were either underpowered due to small sample sizes, focused on MSM, or observational. Thus, more research is needed to examine the effectiveness of such interventions through RCTs with diverse populations. Moreover, all studies cited in this review focused on open or challenge contests to solicit ideas. Other forms of crowdsourcing, such as hackathons or open online systems, may be used to solicit ideas. To promote the scale-up of crowdsourcing, the Special Programme on Training and Research in Tropical Diseases, TDR, published a practical guide on crowdsourcing in health and health research [9]. This provides a number of practical tools for implementing and evaluating contests.

Fourth, more effective methods for evaluating crowdsourced interventions are needed. Although several RCTs have been conducted to evaluate the effectiveness of such interventions in improving HIV outcomes, the development of metrics requires further consideration. For example, most of the existing studies relied on self-reported data collected via online surveys. A research checklist providing guidance on how to evaluate and standardize reporting for crowdsourced interventions may be useful.

Lastly, more strategies for engaging members of key populations or marginalized groups in crowdsourcing research are needed. Most of the studies identified in this review focused on MSM, suggesting the need for research among other groups and key populations. Several contests focused on recruiting key populations or subsets of key populations through including them on steering committees, directing promotional messages to them, and engaging specified groups.

While there are many benefits to using crowdsourcing methods in HIV/sexual health research and service delivery, there are also limitations. First, crowdsourcing methods often require some degree of comfort or skill in using technology to express oneself. For example, several of the crowdsourcing contests cited in the review required the public to create short videos. However, this requirement necessarily constrained participation to those who have access to appropriate equipment and training. Crowdsourcing contests could encourage people to submit multiple types of messages. Additionally, open contests are associated with greater innovation and opportunity to integrate community voice in developing solutions to key public health challenges; however, open contests are often more time consuming than closed contests.

Conclusion

In summary, crowdsourcing can be an effective tool for informing the design and implementation of HIV and sexual health interventions. More research and community collaboration on how to use crowdsourcing to improve HIV/sexual health services, including HIVST and PrEP uptake, are needed.

Table 2.

Summary of completed observational studies that used crowdsourcing to promote HIV/sexual health services.

Field	Year	Program name	Role of crowd	HIC or LMIC*	Outputs
Policy	2011	CrowdOutAIDS	Contribute policy suggestions	LMIC	Suggested three major priorities and six strategic actions to engage youth in HIV-related policy
Research	1995	Hits for Hope	Write a song	LMIC	Adolescents reported safer sexual practices after the mass media campaign
	2011§	HSV perceptions	Create a video	HIC	The solicited videos discussed the motivation of HSV disclosure, the value of disclosure, and the strategies to reduce stigma, which including
	2013	Testing saves lives	Create a video	LMIC	One-minute videos to promote HIV testing among MSM in China
	2015-2016	2BeatHIV	Create images or videos	HIC	Increased community engagement for HIV cure research
Program	1997-2005	Scenarios from Africa	Write a story	LMIC	Stories were selected and transformed into short fiction films in 25 languages and were further used as an educational resource at the community level in sub-Sahara African countries
	1998-2014	GBS vaccine and patient education	Contribute comments/suggestions	HIC	The crowdsourced platform increased parental knowledge that GBS can be sexually transmitted
	2018	Ready PrEP Go	Create images and videos	LMIC	Around three-quarters of the MSM from the community reported that the finalists are designed for them and can grab their attention.

^*Note: High-income countries or Low- and middle-income countries, HSV= herpes simplex virus, GBS = group B streptococcal

^§two included studies were from the same crowdsourcing contest.