Comparing the effectiveness of a crowdsourced video and a social marketing video in promoting condom use among Chinese men who have sex with men: a study protocol

Abstract

Introduction

Crowdsourcing has been used to spur innovation and increase community engagement in public health programmes. Crowdsourcing is the process of giving individual tasks to a large group, often involving open contests and enabled through multisectoral partnerships. Here we describe one crowdsourced video intervention in which a video promoting condom use is produced through an open contest. The aim of this study is to determine whether a crowdsourced intervention is as effective as a social marketing intervention in promoting condom use among high-risk men who have sex with men (MSM) and transgender male-to-female (TG) in China.

Method

We evaluate videos developed by crowdsourcing and social marketing. The crowdsourcing contest involved an open call for videos. Entries were judged on capacity to promote condom use, to be shareable or ‘go viral’ and to give value to the individual. 1170 participants will be recruited for the randomised controlled trial. Participants need to be MSM age 16 and over who have had condomless anal sex in the last 3 months. Recruitment will be through an online banner ad on a popular MSM web page and other social media platforms. After completing an initial survey, participants will be randomly assigned to view either the social marketing video or the crowdsourcing video. Follow-up surveys will be completed at 3 weeks and 3 months after initial intervention to evaluate condomless sex and related secondary outcomes. Secondary outcomes include condom social norms, condom negotiation, condom self-efficacy, HIV/syphilis testing, frequency of sex acts and incremental cost.

Ethics and dissemination

Approval was obtained from the ethical review boards of the Guangdong Provincial Center for Skin Diseases and STI Control, UNC and UCSF. The results of this trial will be made available through publication in peer-reviewed journals.

Trial registration number

NCT02516930.

Strengths and limitations of this study.

• This will be one of the few randomised controlled trials evaluating crowdsourcing.

• The use of a large men who have sex with men (MSM) platform will allow us to reach a large number of MSM who do not disclose their sexual orientation to doctors or others.

• No biomarker data will be collected, and there are inherent limitations associated with behavioural outcomes.

Introduction

Male sexual health

Male condoms have long been recognised as an effective method for reducing the risk of HIV and other sexually transmitted diseases (STDs),^{1 2} but men who have sex with men (MSM) infrequently use condoms in China.^3–6 The resulting high incidence of HIV and STDs among MSM suggests the need for novel health promotion campaigns. One systematic review⁷ and one literature review among MSM⁸ demonstrate that social marketing campaigns are effective in promoting condom use, but the persistence of these behavioural changes over time is unclear. We propose that crowdsourcing may substantially improve on existing methods for developing condom promotion campaigns.

Crowdsourcing

Crowdsourcing is the process of giving individual tasks to a large group, often involving open contests and enabled through multisectoral partnerships. While the process originated in the private sector,⁹ intended to aid research, development and dissemination, it has since been widely adopted. In 2010, the Executive Office of the President of the USA urged federal agencies to use crowdsourcing as a method to develop innovative approaches to governmental initiatives.¹⁰ A crowdsourcing method differs from a social marketing method in several ways.¹¹ Crowdsourcing is a bottom-up approach, using the community for idea generation through implementation rather than relying on the expertise of public health experts. This ensures a higher degree of community engagement than approaches using social marketing do, which tends to be a top-down approach. Crowdsourcing promotes innovation because it removes cognitive fixation, in which innovation is hampered due to new ideas being strongly influenced by prior examples.^12–16 By engaging more people with less experience, this phenomenon is avoidable and allows for a more creative process.¹⁷ Our team has previously used crowdsourcing successfully to develop an effective HIV testing promotion video and images promoting sexual health.¹⁸

Objectives

Aims and hypotheses

Specific aim 1: To compare the effect of a crowdsourced 1 min video to a social marketing 1 min video in promoting condom use among MSM and transgender male-to-female (TG) in China. This will be evaluated using data from follow-up surveys at 3 weeks and 3 months postvideo.

Hypothesis 1: Crowdsourced videos are not inferior to social marketing videos to promote condom use among MSM and TG in China.

Specific aim 2: To compare the cost of using crowdsourcing compared to social marketing methods for developing short videos focused on promoting condom use among MSM and TG individuals in China.

Hypothesis 2: A crowdsourced video is cost saving compared to a social marketing video for promoting condom use.

Specific aim 3: To compare the effect of a crowdsourced 1 min video to a social marketing 1 min video in changing condom use self-efficacy and self-reported behaviour among MSM and TG individuals in China.

Hypothesis 3: Crowdsourced videos are not inferior to social marketing videos in changing condom use self-efficacy and self-reported behaviour among MSM and TG in China.

Methods

Trial design

This study will be a pragmatic, non-inferiority, randomised controlled trial comparing two groups—MSM who watch a crowdsourced video and MSM who watch a social marketing video. Allocation to each arm will be carried out with a 1:1 ratio using a computer-based algorithm. The study is projected to run from November 2015 to February 2016.

Setting

This study survey will be made available to MSM across China through a popular online portal, Danlan and gay mobile dating app, Blued. Danlan.com is an online gay community that allows MSM to connect with each other for relationships, events and communication. The website is maintained by a private corporation, Danlan, which also developed the for-profit app Blued. Blued has become very popular among the MSM population, recently reaching 15 million users.¹⁹ User personal information is protected and secure. Studies have shown that the internet has become a popular method for MSM to find partners, with a reported 28.3–88.4% of MSM using the internet to seek sexual partners.²⁰ While internet-based interventions have yet to be widely dispersed in mainland China, early studies show that such e-technology-based approaches would be well received.²¹

Recruitment

Participants will be recruited using a banner link on a popular MSM app ‘Blued’ (Danlan, Beijing, China), as well as through announcements sent via Danlan's social media (Weibo, a microblogging platform; WeChat, a messaging platform and QQ, a messaging platform). Blued is China's most popular social networking mobile application among MSM. Blued has 15 million followers with 24% (3.6 million people) daily activity rate.¹⁹ Danlan has over 17 000 followers on social media platform Weibo and forwards news via WeChat and QQ to over 429 000 followers.²²

Eligibility

The survey is voluntary, and to be eligible, participants must state that they were born biologically man, had anal sex with men at least once during their lifetime, have had condomless anal/vaginal sex in the past 3 months, are at least 16 years of age and able to complete an online written survey in Chinese. All participants must agree to an online informed consent and provide their cell mobile number. Participants who do not meet these criteria will not be allowed to proceed with the survey.

Formative work

Prior to survey development, we will interview key informants specifically about conducting an internet survey among MSM in China. Survey development will be performed drawing on previous surveys and a review of existing literature, focusing on English and Chinese language studies. The survey will be developed in English and Chinese but conducted entirely in Chinese. The Chinese version of the survey will be piloted online with 150 volunteers to gauge postintervention condom usage rates and to estimate the necessary sample size for the non-inferiority study. The survey will also be piloted with Danlan to ensure there are no problems with distribution. Feedback will be solicited online regarding question wording and interpretation. Pilot data will not be included in the final analysis. The purpose of this extensive formative research is to ensure that the online survey is simple and easy to complete. The CONSORT-EHEALTH checklist for online surveys²³ will be used to ensure completeness. The online survey will be created using Qualtrics Survey Software (Qualtrics, Provo, Utah, USA), and the videos will be hosted on Tencent Video (Tencent, Shenzhen, China).

Interventions

The development of the crowdsourcing video was publicised via open contest. We posted a public call on social media platforms (Weibo, WeChat) for videos promoting condom use awareness. For further promotion, we hosted in-person events at several different college campuses in Guangzhou, China, and worked with local community-based organisations to publicise the contest. In-person events included didactic sessions, interactive feedback sessions and community-driven events. Ten judges, including community health leaders, doctors, business leaders and researchers, evaluated the videos. Each judge scored the video entries on a scale of 1–10 (10 the highest score), and a single winner was identified. The winning video will be included in the survey as the intervention arm of the randomised control trial (RCT). The 1 min video depicts a group of men dressed as cartoon villains attempting and failing to break down a wall, followed by an image of condoms. Our team will delay public announcement of the contest winner to allow time for adequate intervention implementation and comparison. The winning video will be publicly announced 2 weeks after the intervention is evaluated using the 3-month follow-up survey.

The social marketing video was commissioned from a working group in Jinan. This 1 min video contains audio of two men about to engage in intercourse, but stopping to discuss condom use and sexual health as a symbol of love. The script of the video was written by experts in San Francisco and modified by experts and the gay community in Jinan and Qingdao. The video was shot by an advertising company based in Jinan.

Data collection

A survey will be developed using the Qualtrics survey tool. Participants will answer 150 questions on sociodemographic information, sexual behaviour, social norms, condom self-efficacy, HIV testing and community engagement. At the end of the survey, participants will be randomly assigned to one of two intervention arms, the crowdsourcing video or social marketing video, and will view the appropriate video. Participants will not be informed of the video options on randomisation and will not see the alternate intervention video. Participants will provide mobile telephone numbers and will receive text message reminders 3 weeks after initial survey completion to complete the 3-week follow-up survey. After completion of the 3-week survey, participants will be compensated for the first portion of the study (about US$15.87). Three months after completion of the initial survey, participants will again receive a mobile telephone reminder to complete the 3-month survey. After completion, participants will receive the second portion of their compensation (about US$7.93).

Participants will register for our survey using a mobile number. Following completion of data collection, data entries will be screened for duplicate mobile numbers, and the second entry will be excluded. Entries with invalid mobile numbers will also be excluded.

A data monitoring committee will not be required as this study employs low risk behavioural interventions. All participants will provide consent prior to taking part in the study.

Measures

Data from survey items on sociodemographics and sexual behaviours will be collected using standardised survey instruments immediately before video watching, at 3 weeks after video watching and at 3 months after video watching. Sociodemographic characteristics include participants' age, place of residence, highest level of education completed, annual income, marital status, sexual orientation and sexual orientation disclosure. Behavioural variables include number of sex acts in the past 3 weeks, condomless sex with men, condomless sex with women, condom self-efficacy and other secondary outcomes (see online supplementary file 1).

Outcomes

Primary outcomes

The primary outcome will be any condomless vaginal or anal sex (with any sex partner) among MSM and TG individuals following the video intervention. A participant is counted as having had condomless sex if they participated in any act of sexual intercourse (vaginal or anal) that has taken place without use of a condom. Using a postintervention survey, participants will be asked with what frequency they have used condoms since watching the video: all, most, some or none of the time (see online supplementary file 2). The 3-week follow-up survey will ask about the 3 weeks following the intervention, and the 3-month follow-up will cover the 3 months following the intervention. Individuals who have not had sex in the interval will be classified as having no condomless sex.

Secondary outcomes

• Postintervention sex acts

• Condom use social norms

• Condom self-efficacy

• Condom use negotiation

• HIV testing and self-testing

• Syphilis testing and self-testing

• Incremental cost of intervention associated with respective video interventions per individual reporting increased condom use or no sex since intervention. Other cost-related data from organisations involved in making the intervention videos will be collected. Detailed information on incremental costs can be found in table 1

More detailed explanations of secondary outcomes can be found in online supplementary file 1.

Table 1.

Incremental costs associated with social marketing and crowdsourced arms

Phase	Financial costs Inputs to be capture, can all directly be found in the project financial accounts, main challenge is to allocate across components and to allocate SESH overhead costs	Economic costs Extra inputs not already captured by financial costs
Contest development
Video contest (including production)	Money paid for planning and implementation	For social marketing arm: Personnel of CBOs/CDC (director of movie, actors, film editors) Rental of professional video equipment (if applicable) Building cost (office renting) for CBOs/CDC* Equipment and software cost (if applicable)* For crowdsourced arm: Personnel of SESH (although all volunteer) Judging opportunity cost (volunteer) Steering Committee planning meeting (three 1-hour meetings) Building cost (office renting)* In-person promotion costs
Survey start-up	Money paid to launch the survey (start-up)	SESH personnel costs, to design and maintain the programme Equipment cost of SESH (computer and other items)* Software (Qualtrics)*
Survey implementation and intervention	Money paid to the participants (implementation) Money paid for the software used for follow-up (implementation)	SESH personnel costs
Testing		Cost for condoms (from CDC)

*The cost will be annualised and we will calculate a proportion of the cost to account for items only being used the study time frame. The key idea is that some of these phases are like capital goods, where they only need to be performed once but have benefits for longer (thus requiring annualisation of costs), while the implementation phase has a life only as long as the survey is running.

CBO, community-based organization; CDC, centers for disease control.

Sample size calculation

Sample size for this non-inferiority trial was determined assuming an equal probability of reporting condomless sex in the crowdsourced video and social marketing video arms. Assuming a 50% probability of condomless sex in each arm, a one-sided significance level (α) of 2.5%, a non-inferiority limit of 10% and loss to follow-up of 10%, a total sample size of 1170 individuals was required (585 in each arm) to have 90% power (1−β). The sample size was calculated using the formula:²⁴

where and are the true probabilities of reporting condomless sex in the social marketing video (standard) and crowdsourced video (experimental) intervention groups, respectively, d is the non-inferiority limit and f(α,β)=(Φ⁻¹(1−α)+Φ⁻¹(1−β))², where Φ denotes the cumulative distribution function of the standard normal distribution. More information on sample size calculation can be found in table 2.

Table 2.

Sample size for 90% power and one-sided 0.025 significance level

Probability of primary outcome in the control group*	Probability of primary outcome in the experimental group*	N evaluable per arm	Total sample size for RCT
0.50	0.50	526	1170
0.45	0.45	521	1158
0.40	0.40	505	1124
0.35	0.35	479	1066
0.30	0.30	442	984

*On the basis of the pilot study, 9 of 25 participants (95% CI: 18% to 57%) had condomless sex at least once in the 3-week period immediately following the video intervention. According to a similar RCT we conducted in 2014, the loss to follow-up rate was about 10%; adjustment for loss to follow-up required (N evaluable per arm)/(1–0.1) to be enrolled. A non-inferiority limit of 0.1 was used for all calculations.

RCT, randomised control trial.

Randomisation and allocation

Participants will be randomly assigned to one of the two intervention videos using an electronic randomiser tool available through Qualtrics. Randomisation will occur independently of any other data collected, with participants allocated in a 1:1 ratio to one of the two arms. Participants will not be informed of which video (crowdsourcing or social marketing) they are assigned to.

Data analysis

Primary analysis

The primary analysis will evaluate the non-inferiority hypothesis comparing the two interventions, as well as the superiority hypothesis. The difference in proportions having condomless sex (crowdsourced–social marketing) will be computed, with a corresponding two-sided 95% Wald CI. The crowdsourced intervention will be declared non-inferior to social marketing if the upper confidence limit is below 10%. If the upper confidence limit is below 0%, then the crowdsourced intervention will be declared superior to social marketing. The recruitment methods, survey instrument and video length will be the same between in the two study arms.

Effect modification analysis

Effect modification analyses will be under taken based on prior exposure to the condom promotion video viewed by the participant to assess whether this exposure modified the effect of video intervention arm on the primary condom use outcome. A linear probability model will be used to evaluate effect modification by testing for an interaction between intervention and prior video watching.

Missing data plan

If the primary outcome is missing for <11% of participants, then the primary analysis will use a complete-case approach. If the primary outcome is missing for 11 to <20% of participants, then a sensitivity analysis using multiple imputation based on the PROC MI procedure in SAS (Cary, North Carolina, USA) will also be used. If the primary outcome is missing for ≥20% of participants, then multiple imputation will be used in the primary analysis.

Secondary analysis

Comparison will be made between the two trial arms with respect to each of the secondary outcomes enumerated above and in online supplementary file 1. Non-inferiority comparisons will also be made between study arms for the subset of individuals who reported sex during the follow-up period (3 weeks and 3 months, respectively), and causal inference methods will be employed to account for postrandomisation selection bias.

Ethics and dissemination

Ethical review

IRB approval was obtained from the Guangdong Provincial Center for Skin Diseases and STI Control, University of North Carolina at Chapel Hill and University of California San Francisco.

Informed consent

All participants will be provided an online consent form immediately prior to survey initiation. This online informed consent describes personal data to be collected, explaining that data will be used for research purposes. Contact information is provided to participants to address further questions. Participants will be required to sign the consent and provide a mobile telephone number as agreement to proceed with the survey.

Confidentiality

Data will be collected through the Qualtrics survey tool (Provo, Utah, USA). Data will be transmitted securely using Secure Sockets Layer (SSL) (Transport Layer Security (TLS)) 128 bit encryption across the internet (HTTP) and located in a secured Qualtrics server in the USA. The server is configured with redundant hard drive array to ensure reliability. Access to the data will be password protected within the server's firewall. Survey responses will be kept separately from participants' email addresses; the two files will be linked with a non-descript, unique, randomly generated identifier.

Participants will provide mobile telephone numbers, which will be kept separately from data containing answers to survey items. These telephone numbers will be accessible only to two researchers solely for the means of sending reminders, follow-up surveys and mobile top-up incentives.

Dissemination

The results of this study will be prepared and submitted for publication in a peer-reviewed journal. Study findings will also be shared through conference abstracts and presentations, workshops, and to our partnering organisations.