Broadening community engagement in clinical research: Designing and assessing a pilot crowdsourcing project to obtain community feedback on an HIV clinical trial

Abstract

Background/aims:

Community engagement is widely acknowledged as an important step in clinical trials. One underexplored method for engagement in clinical trials is crowdsourcing. Crowdsourcing involves having community members attempt to solve a problem and then publicly sharing innovative solutions. We designed and conducted a pilot using a crowdsourcing approach to obtain community feedback on an HIV clinical trial, called The Acceptability of Combined Community Engagement Strategies Study (ACCESS). In this work we describe and assess the ACCESS crowdsourcing activities in order to examine the opportunities of crowdsourcing as a clinical trial community engagement strategy.

Methods:

The crowdsourcing engagement activities involved in ACCESS were conducted in the context of a phase 1 HIV antibody trial (ClinicalTrials.gov identifier NCT03803605). We designed a series of crowdsourcing activities to collect feedback on three aspects of this clinical trial: the informed consent process, the experience of participating in the trial, and fairness/reciprocity in HIV clinical trials. All crowdsourcing activities were open to members of the general public 18 years of age or older, and participation was solicited from the local community. A group discussion was held with representatives of the clinical trial team to obtain feedback on the utility of crowdsourcing as a community engagement strategy for informing future clinical trials.

Results:

Crowdsourcing activities made use of innovative tools and a combination of in-person and online participation opportunities to engage community members in the clinical trial feedback process. Community feedback on informed consent was collected by transforming the clinical trial’s informed consent form into a series of interactive video modules, which were screened at an open public discussion. Feedback on the experience of trial participation involved designing three fictional vignettes which were then transformed into animated videos and screened at an open public discussion. Finally, feedback on fairness/reciprocity in HIV clinical trials was collected using a crowdsourcing idea contest with online and in-person submission opportunities. Our public discussion events were attended by 38 participants in total; our idea contest received 43 submissions (27 in-person, 16 online). Facebook and Twitter metrics demonstrated substantial engagement in the project. The clinical team found crowdsourcing primarily useful for enhancing informed consent and trial recruitment.

Conclusions:

There is sufficient lay community interest in open calls for feedback on the design and conduct of clinical trials, making crowdsourcing both a novel and feasible engagement strategy. Clinical trial researchers are encouraged to consider the opportunities of implementing crowdsourcing to inform trial processes from a community perspective.

Introduction

Community engagement is crucial for the ethical design and conduct of medical research,^{1, 2} particularly clinical trials.^{3, 4} We define community engagement as the processes by which input is sought from community stakeholders to inform the design or conduct of clinical trials.⁵ Insufficient engagement with wider community concerns may contribute to adverse outcomes in clinical trials, including nonadherence to interventions,⁶ community mistrust of research,⁷ and halted trials.^8–11 Several guidance documents highlight the importance of good-quality community engagement for clinical trials,^12–15 including the UNAIDS/AVAC Good Participatory Practice guidelines.⁵ The Good Participatory Practice guidelines note that many strategies can facilitate community engagement, including both formal (e.g. community advisory boards) and informal (e.g. focus groups) mechanisms.

One novel strategy with the potential to extend community engagement in clinical trials is crowdsourcing. Crowdsourcing is an approach that involves an open call for members of the public to come together as a group to solve a problem, and then publicly sharing selected solutions that emerge.¹⁶ It has been successfully used to engage community members in health-related research, including studies to design HIV testing campaigns^{17, 18} and to investigate the meaning of HIV cure for local lay community members.¹⁹ However, few studies have used crowdsourcing to inform community engagement in clinical trials,²⁰ resulting in a lack of literature to guide the implementation of crowdsourcing engagement activities for community feedback on clinical trials.

To advance our understanding of crowdsourcing as a community engagement strategy for clinical trials, we assess a series of novel crowdsourcing activities through a pilot study called ACCESS (The Acceptability of Combined Community Engagement Strategies Study). This study involved the design and conduct of multiple crowdsourcing activities to obtain community feedback on aspects of a phase 1 HIV antibody trial. The purpose of this paper is to present the context and design of the crowdsourcing activities used in ACCESS, to assess the levels of community engagement in each of these crowdsourcing activities, and to examine the opportunities of crowdsourcing as a means of extending community engagement in clinical trials.

Methods

We used crowdsourcing to obtain community feedback on an HIV clinical trial at the University of North Carolina at Chapel Hill called the VOR-07 study (ClinicalTrials.gov identifier: NCT03803605). This phase 1 clinical trial examined the safety and effectiveness of combining a dose of an HIV antibody (VRC07–523LS) with vorinostat, a drug used to reactivate latent HIV virus in HIV-infected individuals. Conducting crowdsourcing alongside the VOR-07 trial presented us with several advantages for evaluating crowdsourcing as a method for HIV clinical trial community engagement. As VOR-07 was a phase 1 clinical trial, we were reasonably certain that members of the local community had not yet heard of this trial, and thus the feedback obtained would represent community participants’ first impressions of the trial’s design and procedures. This is particularly important given the influence that myths about HIV clinical research can have on community members’ willingness to engage in HIV research.²¹ Additionally, by partnering with the VOR-07 team we were able to access the trial’s informed consent form and protocol to develop our engagement activities. This allowed us to obtain feedback on a ‘real life’ example of a clinical trial rather than relying on a fictional example or the abstract concept of clinical trials in general. Asking for community members’ feedback without a concrete example of a trial would have been particularly difficult given that trial literacy is a major barrier to community engagement.²²

Finally, examining crowdsourcing in the context of an actual trial allowed us to identify community feedback that could potentially be used to inform the design and conduct of HIV clinical trials. Importantly, it should be noted that the ACCESS study was conducted in parallel with the VOR-07 clinical trial as a separate social science study without direct influence on the trial’s protocol and processes.²³ Indeed, the engagement activities of ACCESS were only launched once the protocol for the VOR-07 trial had already been approved, due to the need to use the trial’s materials (i.e. consent form and description of trial processes in the protocol) to develop our engagement activities. Thus, while the feedback collected through our ACCESS engagement activities would not be able to impact early processes of the VOR-07 trial (e.g. informed consent, recruitment), it could potentially impact future HIV clinical trials.

Crowdsourcing involves prompting participant feedback to focus on solutions to specific, predefined problems.²⁴ We identified three aspects of the VOR-07 trial where crowdsourcing engagement strategies could potentially provide community input: 1) the process of obtaining informed consent, 2) the experience of participating in the clinical trial, and 3) concepts of fairness and reciprocity in HIV clinical trials (i.e., how clinical trial researchers could give back to the community). These topics cover the duration of a trial from initial design through to trial conduct and results dissemination/follow-up.⁵ We operationalized these topics into a series of three separate engagement activities, each requiring their own unique tools for soliciting participant feedback.

Social media engagement metrics (i.e. Facebook and Twitter) were used to assess the extent of engagement with ACCESS crowdsourcing activities. Ethical approval for ACCESS was obtained from the Institutional Review Board at the University of North Carolina at Chapel Hill. Written informed consent was obtained from all participants prior to their participation in each crowdsourcing activity.

After the ACCESS study, preliminary findings were presented to representatives of the VOR-07 clinical team followed by a guided group discussion. This discussion focused on obtaining participants’ reactions to both the findings of ACCESS and the engagement methods used in ACCESS, specifically whether and how our engagement strategies could be useful for informing VOR-07 and/or future HIV clinical trials. This group discussion with the clinical team was audio recorded and transcribed verbatim, and written informed consent was obtained from all group discussion participants.

Results

Crowdsourced feedback on the process of obtaining informed consent

For our first crowdsourcing activity, we transformed the 20-page VOR-07 informed consent form into a series of interactive, plain-language, animated video modules. The modules were created by a local community-based organization that specializes in translating clinical science to lay audiences through creative products (Community Expert Solutions).²⁵ Graphics and narration were used throughout the modules to decrease cognitive load and increase readability for a lay audience. We hosted an open public viewing and discussion of the modules at the Durham County Department of Public Health, a public health services venue that can be accessed among disabled individuals (elevators and ramps), is accessible by public transportation, and is familiar to local residents. The event was advertised via the Facebook (295 followers) and Twitter (329 followers) accounts of our research group, searcHIV (the Social and Ethical Aspects of Research on Curing HIV). We also distributed physical copies of the event flyers at both a local infectious disease clinic and a clinical research center. We additionally sent the event information to local HIV community engagement advocates, including the UNC Strategic Community Engagement Education Dissemination office. Importantly, we solicited the assistance of a community member known to our research group as a highly-engaged local HIV advocate who had previously participated in crowdsourcing research conducted by our group (i.e. the 2BeatHIV project).²¹ This advocate provided crucial assistance to our recruitment efforts by sending the event information through her extensive networks in the community via phone/text messages and email. Having the direct involvement of a well-known community advocate in our recruitment processes helped to build trust in our research activities and served as a bridge to potential participants. Finally, we also sent the event information to three local HIV-related community advisory boards (with board members encouraged to circulate the event flyer among their wider community networks). Anyone aged 18 years or older was eligible to participate, with no requirements to have familiarity with HIV clinical trials.

After screening the modules, participants were asked to discuss which parts of the informed consent modules were easiest and most difficult to understand, what additional information they thought a trial participant should know (i.e. what was missing from the modules), and what additional formats would be useful for conveying informed consent information to potential trial participants. This discussion was guided by a facilitator, who emphasized for participants that this event was not a clinical trial recruitment session but rather a community feedback opportunity – an important distinction given that participants were viewing a video version of a clinical trial informed consent form. To obtain a sufficient participation to benefit from multiple perspectives, we offered attendance incentives of a $15 gift card per attendee and also provided lunch.

In total, we made 4 posts on to our Facebook account and 2 posts to our Twitter account advertising this event (see Table 1). Social media engagement metrics showed that our Facebook posts reached (i.e. were viewed by) a total of 584 unique users, garnering 121 engagements (i.e. ‘likes’, comments, or sharing of the event posts) and 40 post clicks (i.e. clicking on the post to expand the image/text of the post). Our 2 Twitter posts advertising the event garnered a total of 2362 impressions (i.e. the number of times users are served a posted Tweet in their timeline or search results) and 36 total engagements (i.e. the number of times users interact with a Tweet in any way). Ultimately a total of 20 people participated in-person at the event. We collected 1 hour of audio-recorded feedback from the participant discussion following the module screening.

Table 1.

Engagement metrics for social media posts announcing crowdsourcing events and sharing of results

Event/Sharing Posts	Number of Postsa and Total Engagement Metrics by Social Media Platform
	Facebook				Twitter
	# of Posts	Reachb	Reactionsc	Clicks	# of Posts	Impressionsc	Total Engagementsd
Event 1 (Informed Consent)	4	584	121	40	2	2362	36
Event 2 (Experience of Trial Participation)	3	152	112	7	2	1947	22
Event 3 (Idea Contest)	4	489	62	16	2	2204	17
Sharing of Idea Contest Results	4	311	61	14	4	2014	39

^aThe number of posts made by our team on a social media platform to advertise the event/share the results, including initial announcement and (for events) subsequent reminders.

^bReach is defined by Facebook as the number of unique users who viewed a post.

^cImpressions are defined by Twitter as the number of times a user is served a Tweet in timeline or search results.

^dTotal engagements are defined by Twitter as the number of times a user interacted with a Tweet in any way, including clicks anywhere on the Tweet, re-tweeting, replying, and ‘likes’.

Crowdsourced feedback on the experience of participating in the trial

For our second crowdsourcing activity, we developed a series of three vignettes depicting fictional characters’ experiences with participating in the VOR-07 trial. The vignettes were developed on the basis of the VOR-07 trial’s protocol, with the clinical team providing review and input to ensure that the vignettes accurately reflected trial processes. Each vignette depicted characters experiencing different stages of trial participation: initial recruitment into the trial, going through clinical trial procedures, and completing the trial. The vignettes were then transformed into three animated videos which visualized the characters’ experiences and contained voiceover narration. We hosted an open public viewing of the videos (again at Durham County Department of Public Health), followed by a discussion with participants about their reactions to these (fictional) trial participants’ experiences and the various barriers and facilitators to trial participation that might exist in their own (or others’) lives. We used the same recruitment strategies as those implemented for the previous crowdsourcing activity on informed consent, and participant incentives included a $15 gift card and breakfast.

We made 3 posts on Facebook advertising this second crowdsourcing event, and 2 posts on Twitter. Social media engagement metrics demonstrated slightly less reach with this second event compared to the first; our Facebook posts reached a total of 152 unique users and garnered 112 reactions and 7 clicks, while our Twitter posts received 1947 impressions and 22 engagements. A total of 18 people participated in this event. We collected approximately 1.5 hours of audio-recorded feedback from the group discussions following presentation of each animated vignette.

Crowdsourced feedback on fairness and reciprocity in HIV clinical trials

For our final crowdsourcing activity, we launched an idea contest with both in-person and online submission opportunities. Idea contests are a form of crowdsourcing that involves an open call for submissions in response to a challenge or prompt, evaluation of submissions, and celebration and sharing of the winning ideas.^{19, 26} As the prompts for the idea contest, we posed questions to participants on how HIV clinical trial researchers could give back to the community. This topic included how to make it easier for community members to learn about HIV clinical trials, how to better communicate eligibility criteria to potential participants, how clinical trial researchers could help participants post-trial, and how to communicate trial results. Participants were asked to submit creative ideas in response to these prompts, with the goal of identifying the most helpful answers. The in-person portion of the contest was launched as a part of a public event in celebration of HIV Cure Research Day on December 14^th, 2018 at a well-known community clinic easily accessible by public transit. Event attendees were invited to contribute their ideas on colorful submission forms, which were then put on display as part of the event’s festivities (which also included music, food, a presentation by an HIV cure researcher, and resource tables operated by HIV community organizations). We offered small bags of candies as both an incentive to participate and a token of appreciation to anyone who filled out an idea submission form.

Additional contest submissions were subsequently collected via an online submission form hosted on our research group’s website (http://searchiv.web.unc.edu/). Both in-person and online portions of the contest were advertised using the same strategies as the previous two events (social media, flyers at an infectious disease clinic/clinical research center, and outreach to local HIV advocates/ community advisory boards). For the online portion of the contest, we also circulated flyers with the contest information and URL among popular student message boards across the University of North Carolina at Chapel Hill campus, and informed participants of our previous crowdsourcing events of this additional opportunity to be involved in the project. In total we made 4 posts to our Facebook account and 2 posts to our Twitter account containing the link to the online contest submission page. Social media engagement metrics showed that these posts were reached 489 unique users and garnered 62 reactions and 16 post clicks on Facebook; our Twitter posts received 2204 impressions and 17 engagements.

We received a total of 43 idea submissions, including 27 entries submitted in-person and 16 online submissions. Forty submissions were eligible and evaluated by a panel of judges, which was comprised of a mix of community engagement experts and HIV researchers. Criteria for judging included feasibility of implementation, creativity/uniqueness of the idea, and potential for the idea to have a positive impact on the relationship between clinical trial researchers and trial participants and/or community members. Judges were provided with instructions on how to assess and assign a score to each idea, as well as descriptions of what would constitute exceptional, good, fair, and poor scores. Judges’ assessments were used to identify the top three ideas as finalists based on their final scores. Individuals who submitted these winning ideas were awarded gift cards ($200, $100, and $50 for first, second, and third place, respectively) and their winning ideas were announced and celebrated on our research project’s website. We also created images with abbreviated versions of the winning ideas to post on our social media accounts (i.e. Facebook and Twitter) containing links to the full text of the ideas hosted on our website, and a general ‘thank you’ image acknowledging all contest participants (see Figure 1). Collectively these posts reached 311 unique users on Facebook, garnering 61 reactions and 14 post clicks; the Twitter posts received 2014 impressions and 39 engagements. Importantly, the identities of the individuals who submitted the winning ideas were not publicly shared in order to focus attention on the ideas themselves. Keeping finalists’ identities confidential was also specified in the informed consent language of the contest submission forms (both in-person and online) in order to encourage participation.

Figure 1.

Images created to celebrate and share the crowdsourcing contest finalists’ ideas and to thank contest participants.

Feedback and discussion with the clinical team

Three representatives from the VOR-07 clinical team participated in the post-ACCESS feedback and discussion session. Participants agreed that using video modules for informed consent (as was used as an engagement tool in the first ACCESS crowdsourcing event) may be useful. The clinical team suggested that including additional visual elements on trial recruitment websites may enhance communication of trial procedures (e.g., a video explaining leukapheresis).

The clinical team also pointed to the opportunities and potential challenges of using crowdsourcing as an engagement strategy to inform clinical trials. Crowdsourced feedback on strategies and spaces for expanding recruitment opportunities (i.e. outside of doctors’ offices) was viewed by the clinical team as useful information and indicative of the value of crowdsourcing to further understand where and how to reach potential trial participants. However, the clinical team also noted that there were some limitations to the ability to implement crowdsourced feedback on trial processes. For example, researchers would not be able to change the frequency of follow-up or timing of blood draws. Clinical team members also suggested that sometimes feedback from community members may not be as useful for informing trial processes because of a limited understanding among crowdsourcing participants of what is feasible to implement in clinical research. Finally, the clinical team raised some concerns about using crowdsourcing as an engagement strategy for high-risk clinical research. For example, communicating trial compensation amounts to the broader public through crowdsourcing (including people who would not necessarily be eligible to participate in the trial) may create confusion among people living with HIV who are not eligible to participate. As such, the clinical team identified the specific context of a clinical trial and the risk-benefit ratio to potential trial participants as important factors to consider in the decision to implement crowdsourcing as a part of a trial’s community engagement.

Conclusions

This study has several implications for community engagement related to clinical trials. First, we found that crowdsourcing is a feasible method to conduct community engagement activities related to clinical trials. While trial literacy has been identified as a challenge for engaging community stakeholders,²² our plain-language video modules and animated vignettes made it possible for community members without prior knowledge of the VOR-07 clinical trial to attend discussion groups on topics related to the trial processes (informed consent and experiences of trial participation) and offer feedback on trial design and the conduct of clinical HIV research, resulting in a total of 2.5 hours of audio-recorded discussion.

Second, there is sufficient community interest in research to allow open calls for community feedback on clinical trials. This suggests that crowdsourcing could be used as a way to enhance engagement in other clinical trials; however, as the clinical team noted in follow-up discussion, some trials may be more amenable than others to the use of crowdsourcing as a community feedback mechanism. Additionally, social media engagement metrics demonstrate that Facebook and Twitter posts advertising crowdsourcing events and sharing of contest finalists’ ideas can generate substantial levels of engagement with users. Fostering sustained relationships with key community leaders in combination with promotion on social media platforms can be effective for reaching community members for participation in crowdsourcing activities to inform clinical trials.²⁷

Third, since a crucial element of crowdsourcing is the sharing of solutions contributed by the crowd,²⁴ the community feedback we obtained will be disseminated to a broader audience than just the VOR-07 clinical team. Our crowdsourcing idea contest allowed us to share finalists’ winning ideas with the broader public via social media and our project. Additional results from analysis of feedback obtained in our crowdsourcing events will also be shared with community advisory boards, HIV advocacy networks, and public information channels – including not only academic publications, but also blog posts, social media to ensure our findings can reach the lay audiences who made our crowdsourcing possible.

Fourth, ACCESS demonstrates the importance of sustained relationships with community leaders and local community-based organizations as crucial elements in the success of community engagement research to inform clinical trials. Our team has had previous experience with conducting HIV-related crowdsourcing research events in our local community,^{19, 21} which helped us to design engagement activities with community members’ interests in mind. Additionally, our pilot study’s recruitment processes benefitted from local community members’ familiarity with our team’s past community engagement efforts as this helped to establish trust in the legitimacy of ACCESS among participants. This experience highlights the importance of approaching community engagement as an ongoing process of building sustainable relationships between trial researchers and the local communities in which clinical trials are embedded.

A limitation to the engagement activities of the ACCESS pilot are that these activities were conducted separately from and in parallel with the VOR-07 clinical trial itself, and thus the feedback from this pilot is unable to directly impact the design and conduct of this particular trial. We also acknowledge that the possibility of altering trial conduct by crowdsourcing is limited due to the fixed nature of certain aspects of clinical trial protocols. However, this limitation is not specific to crowdsourcing as an engagement method, and suggests further reflection on the rigid nature of clinical trials itself, particularly in an era where the benefits of adaptive trial design are becoming aparent.²⁸ Finally, ACCESS was conducted by a research team trained in social science research techniques (e.g. focus group facilitation, qualitative data collection). Clinical teams may not necessarily have the training or time to implement similar crowdsourcing engagement strategies themselves; however, this highlights the potential added value of collaborating with social scientists as well as the need to dedicate sufficient resources to supporting community engagement activities.

The design of our crowdsourcing pilot provides insights on the opportunities of using this method for obtaining community input on clinical trials. None of these crowdsourcing activities are meant to replace the use of clinical teams’ existing engagement strategies (e.g. community advisory board feedback), but rather extend additional opportunities for discussions about important aspects of a clinical trial (e.g. informed consent forms, trial protocols) into the broader, local community in which a trial is taking place. While there is some evidence that it is feasible to use crowdsourcing to inform clinical trial design,²⁹ it is not currently a widely-used method for engagement in clinical trials.²⁰ Crowdsourcing thus represents a novel strategy for meeting recommendations to implement additional stakeholder advisory mechanisms in clinical trials.⁵