Centralized Research Recruitment—Evolving a Local Clinical Research Recruitment Web Application to Better Meet User Needs

Abstract 

Recruiting volunteers into clinical research remains a significant challenge for many clinical research study teams, thus the Michigan Institute for Clinical and Health Research (MICHR) at the University of Michigan developed UMClinicalStudies (http://www.UMClinicalStudies.org)—a Web application that links the community to a single gateway for clinical research. UMClinicalStudies (formerly named “Engage”) is an integral piece of MICHR’s efforts to increase clinical research participation in order to advance medical discoveries. Despite the initial success of the application, barriers to research participation remain, including the applications accessibility for potential research volunteers and study team members. In response, new initiatives were instigated to identify user needs, in order to broaden the ability to simultaneously assist researchers in recruitment activities, while also aiding potential volunteers in the exploration of and participation in clinical research opportunities. To do this, improvements to the interface and functionality were identified and implemented for both the public and the research audiences through extensive system analysis, and through the application of human computer interactivity processes, resulting in significant improvements in usability and ultimately research volunteerism, indicating that utilizing such technology is pivotal in reaching broader audiences for clinical trial participation. Clin Trans Sci 2011; Volume 4: 363–368

Introduction

Improving medical care relies on new knowledge gained from research. Yet, recruiting volunteers into clinical research remains a significant challenge for many clinical research study teams, particularly as limited funding necessitates shorter timelines to engage participants. With 50% of clinical research sites enrolling one or no patients in their studies, ¹ it is clear that there is a great need to improve recruitment practices.

Data show that there are over 80,000 clinical trials conducted each year in the United States, however, less than 2% of the population participates. ² It is well established that most of the public is either not informed of the need for volunteer participation in clinical research, or there is a fear or mistrust of the medical and clinical research enterprise. ³ , ⁴ , ⁵ Clearly, institutions have much room for improvement in communicating the need and opportunity for clinical research participation. Typically, clinical research information tends to be broadly dispersed, creating a difficult and inefficient environment for patients and community members to find studies. Thus, the Michigan Institute for Clinical and Health Research (MICHR) at the University of Michigan developed UMClinicalStudies (http://www.UMClinicalStudies.org),—a Web application that links the community to a single gateway for clinical research.

UMClinicalStudies (formerly named “Engage”) is an integral piece of MICHR’s efforts to increase clinical research participation in order to advance medical discoveries. The site consists of several components—Study Database—commonly referred to as the Bulletin Board, Community Information and Education, and a Health Insurance Portability and Accountability Act of 1996 (HIPAA) and IRB compliant Registry of research volunteers.

The Bulletin Board component was released in July 2005, and offers research teams the ability to add and update their studies, provides multiple search tools for the public, provides details on studies actively recruiting, and contact information for each recruiting study. In addition, it assures compliance by automatically checking that each recruiting study has a valid Institutional Review Board approval. Currently, there are over 437 active studies in the Bulletin Board, with an average of over 10,000 visitors per month.

The Research Volunteer Registry component was released in January 2007, and consists of a database that allows patients and community members to sign up for a single study, or to be included in the general registry. Once registered, volunteer information is matched daily to the eligibility requirements in each active study posting, providing volunteers a list of current studies in which they may be eligible to participate, and providing research study teams a list of potential volunteer research participants. Volunteers who are University of Michigan Health System (UMHS) patients may also grant access to their medical record (for screening purposes only), which provides research study teams a database of study candidates with crucial disease information, allowing for more efficient recruiting. The ability for researchers to contact registrants if they are a potential match for their study has streamlined enrollment processes considerably. At present, over 8,000 volunteers have registered at the time of this writing; an increase of over 700% since the implementation of MICHR’s Clinical and Translational Award (NIH grant UL1RR024986).

The Community Information & Education component of the site contains a mini‐tutorial about clinical research—information focused on diversity and multicultural research participation, a health glossary containing links to specific health conditions and preventative health information, and a glossary of clinical research terms to guide users in the language that is often used by researchers. These pieces were crucial components developed in partnership with community partners, designed to narrow the gap between academic and lay language as it pertains to research participation.

System Overview

The Bulletin Board and Registry are java‐based Web applications, running on resin application servers on Linux, using Oracle as the backend database. The systems were designed as Web applications to ensure ease of use for end users, and greater accessibility for a broader audience. The Community and Education component was developed as static html Web pages. CSS style sheets are used for visual presentation and shared between all three components allowing the presentation to appear as one application to the end user.

Potential research volunteers can either search for studies that are posted, and contact research study team members directly, or, after completing an electronic informed consent process, the volunteers can enter basic information about their health interests through the Research Volunteer Registry. UMHS Patient volunteers can also provide (or deny) permission for researchers to view medical records for the purpose of prescreening. Potential volunteers can specify which researcher may or may not view their information. They may electively choose certain research studies from the Bulletin Board and allow those research study teams to view their information (Figure 1). Alternatively, they can specify that any research study team with a study in a relevant, specified disease area can view their information, or they can specify that their information be accessible to any research study team for any study that they appear to be eligible for. If a match seems probable based on the information in the registry, researchers can contact registry volunteers for further discussion. The volunteers can specify whether and how they can be contacted. Volunteers can remove their profile from view at any time. Extensive measures are also in place to ensure the security of data stored in the registry—including using a separate database, referred to as the Honest Broker (Figure 2). The Honest Broker is an information system hub that can work with clinical information systems to support clinical research while keeping volunteer and patient confidentiality secure and HIPAA‐compliant. This is the central piece of software, which handles the components responsible for receiving, storing, maintaining, and distributing deidentified protected health information (PHI).

Figure 1.

UMClinicalStudies.org is a research study information portal, allowing potential study participants to search for current studies and register through the UMClinicalStudies Registry, a secure, password‐protected database that matches volunteers to studies based on their profile and the study eligibility criteria. Likewise, research teams are provided a list of potential volunteers that appear to be eligible based on general volunteer profi le data.

Figure 2.

The fi gure above shows the many sources of data are used to provide content for the clinical studies posting components, which are denoted in gray. In addition, the clinical studies components feed data to other sites such as the U of M Health patient Web site, http://www.uofmhealth.org/. The connections in black show where data is retrieved either directly from a database or via a Web service. The connections in red show where URLs are used to link to components together.

Understanding that engaging the public in clinical research cannot be completely reliant upon Web interface applications—and that many volunteers will want more personal assistance, MICHR also employs staff to operate a toll‐free phone‐line and an email response team. Research team members, as well as patients and potential research volunteers, have a “live person” to respond to inquiries and issues. Similarly, MICHR’s team has had a proactive approach to gathering input and assessing the effectiveness of UMClinicalStudies for research teams and potential research volunteers—taking a “ground up” approach to ensure that the system is designed in collaboration with its intended users. It is from these measures that the UMClinicalsStudies Web site underwent significant reconstruction, as described below.

Discussion—The Need to Keep Evolving

Despite these undertakings to develop an elaborate yet streamlined system that aims to educate, inform, and empower volunteers in the area of clinical and translational research, the continuing struggle to recruit research participants remains. Though building awareness of the need for clinical research volunteers requires a robust communication, outreach, and marketing plan, it was deemed essential that any traffic that was successfully driven to the Web site portal needed to be greeted by a more intuitive and user‐friendly site. Thus, in the interest of reducing barriers further, new initiatives were instigated to identify user needs, in order to broaden the ability to simultaneously assist researchers in recruitment activities, while also aiding potential volunteers in the exploration of and participation in clinical research opportunities. To do this, improvements to the interface and functionality were identified and implemented for both the public and the research audiences.

Since the Internet allows for expansive forms and sources of information to an increasingly broad audience, ⁶ maintaining relevancy and updates that meet user expectations requires a hypermedia rich with concise but informative content, and a multimedia presence going beyond hyperlink navigation. ⁷ By watching user patterns, remaining cognizant of the average Web consumer’s connection speeds, the Web site and application redesign strived to strike a balance between the expectations of consumers, and exploring the boundaries of current technology. ⁸ As such, through these explorations, it was determined that current expectations and capabilities include a need to offer a personalized service to users, and perhaps most exciting is the opportunity to facilitate a two‐way dialogue with the community.

Current analysis and information drawn from individual users, as well as focus groups (investigators/study coordinators/general public/multicultural task forces), have led the UMClinicalStudies team toward developing an environment that offers enhanced features and tools, and improved usability. By continuing to integrate the knowledge of study teams and community members in the development of an improved research recruitment Web application, the aim is to ensure that UMClinicalStudies evolves to remain useful and relevant to an increasingly broad audience. This article reports on the process utilized and design objectives to evolve a local clinical research recruitment Web site to better meet user needs and recruitment outcomes.

Methods—The Reconstruction Process

The principles of human computer interaction apply to any public Web resource, but perhaps especially so in one that has the potential to elicit charged emotions or concerns. Thus, MICHR worked with UM’s School of Information’s Human Computer Interaction graduate students for the first two phases of the reconstruction through a course “Evaluation of Systems and Services,” which covered the key concepts of evaluation and a variety of methods used to determine the goals of a system or service.

The reconstruction process consisted of the following steps:1.

• 1System Evaluation

  In an effort to create an effective tool for both researchers and volunteers to work with, the reconstruction process began by first analyzing the existing system in detail using various evaluation and analysis techniques. Detailed below are techniques that were used and the experiences with them:

  • • Generalized Transition Network (GTN)—This network can be used to describe complex systems, with the key advantage being that it provides the ability to easily describe hierarchies of modes or states of a system. ⁹ We utilized a GTN process to help in understanding the system, its loops paths, features, user interactions, etc. This allowed a high‐level view of the system, allowing our team to become more familiar with the system intricacies, while also identifying some low‐level issues to be resolved. This was a time‐consuming process, and required skills in page layout and photo editing software.
  • • Comparative Evaluation—This involved looking at similar systems to gather information about what was being offered—including the functionality, usability, and esthetics of sites that were similar to UMClinicalStudies.org. This was beneficial in identifying the strengths of the system, while also identifying features that could be included or improved upon. Identifying pros and cons was essential at this step to begin determining the scope of work of the reconstruction. Access to information for each comparative site varied, as it was difficult to conduct detailed analysis within sites that required authentication.
  • • Heuristic Evaluation—The goal of heuristic evaluation was to find the usability problems in the design so that they could be attended to as part of an iterative design process. Our heuristic evaluation was a relatively quick and straightforward process, involving a small set of evaluators who examined the interface and judged it’s compliance with recognized usability principles, ¹⁰ , ¹¹ though it was noted that because there was no participation of the end users in this process, the information obtained revolved around consistency of interface design.
  • • Vocabulary Analysis—A large portion of UMClinicalStudies is text‐based. Hence, the language and textual elements used in the interface play a very important role in how well the users understand and use the system—particularly for a complex topic such as clinical research. For this reason, the team conducted a vocabulary analysis to analyze the appropriateness of the vocabulary and grammar used on the Web site. This was incredibly useful, as it revealed vocabulary‐related issues that would not have otherwise been revealed through other usability inspection models.2.
• 2Assessment of User Needs

  It was intended for this site to evolve cooperatively between researcher, volunteer and community needs. First, the team set out to better understand what information the public wanted UMClinicalStudies to provide and how such a site would be most useful to them. The following approaches were used to gather a host of feedback from people about the usability and effectiveness of the Web site:

Focus groups

The team employed a qualitative methodology utilizing focus groups in order to understand and identify:

• • General attitudes toward clinical research (concerns, fears, perceived risks or benefits, questions).
• • What would motivate individuals (besides getting paid) to participate in research.
• • Individuals expectations of a Web site where they could go to learn about clinical research—for example, site name, how they would find it, what it would be like, and what they would expect to be able to do there.
• • Ways to improve public outreach regarding clinical research studies.

The focus groups included an even mix of men and women participants, of various ethnicities, between ages 20 to 65. Groups were facilitated by an external moderator to help ensure that participants could be spontaneous in their discussion without feeling pressured or persuaded to relay favorable opinions or alter their initial attitude.Similarly, focus groups and interviews were conducted with researchers and study team members. By gaining community and researcher input at this stage, we set out to further tailor UMClinicalStudies to meet the needs of both the researchers and the public for research projects.

User interviews

User Interviews were conducted with end users and various stake holders to identify specific user needs, user demographics, and the ways in which users use the system in its current form—and identify any issues within the system that required “work around” efforts due to potential system limitations. An effort was made to ask open‐ended questions so that interviewees themselves could lead the team through important information with regard to usage, problems faced, system improvement suggestions, and other such commentary.

User testing

User testing was performed concentrating on efficiency, accuracy, recall, and emotional response. Project evaluators spent approximately 1–2 hours each with individual system users, observing usage of UMClinicalStudies, and then interpreting the results to identify patterns and how the actions related to the interface of the Web site. The results from this exercise were rich in quality and provided significant feedback about the system, and guided the prioritization of our findings. Because of the difficulty in aligning volunteers, researchers and study team members to be involved in the user testing, this level of analysis required substantial planning and management. In consideration of time, and limited resources, we were careful to define our goals and design a testing process and analysis that would produce the most consequential results. Careful thought went into formulating tasks, as they were providing the basis for revealing key information.

Surveys

Surveys were distributed to both research volunteers and researcher study team members regarding UMClinicalStudies and its interface, as well as the success they had with the system. The survey helped to reach a wide user population, though the information obtained was more quantitative than qualitative. This process gave a good sense of the attitudes and feelings that users were having toward the system.

Personas and scenarios

Personas and Scenarios were the tools utilized which aided in understanding the system’s users and to imagine them using the system under various conditions. Three personas were developed from the user information, clearly identifying and differentiating the three groups of users that the system had. Developing user scenarios helped establish a vocabulary in both identifying users’ current needs, and establishing the validity/basis for a particular task or function, and focusing on the activities that need to be supported by the system, rather than making the users conform to the systems functionality. ¹² Questions like “Would our persona use or need this feature?” helped in thinking analytically and broadly about the various issues that the system faced.

• 3Developing Project Objectives

The original version of UMClinicalStudies was very text‐heavy, and, judging from the results of user observations/testing, not intuitive. The interface of UMClinicalStudies was broadly construed to include not just the visual/auditory display and interaction dialog, but the situation in its entirety, including interacting with a public that has many cautions already in place regarding clinical research. We also realized, through our work with the users that the system was considerably difficult to navigate for both researchers and volunteers—and action steps and paths were not clearly defined. Logging in to the registry (for new user registration or revisits) seemed to be a major problem for users since it seemed to be buried deep inside the Web site pages. Many key components of the site were several clicks away for the users. Because of the unintuitive nature of this earlier version of UMClinicalStudies (then called “Engage”) the bounce rate (the percentage of single‐page visits or visits in which the person left the site from the entrance page) within the Web site was extremely high—average of 72.46%—which caused great concern.

From all the evaluations and user observations conducted, a list of priorities was generated that defined the project objectives for the new release. The underlying aim of the reconstruction process was to improve the usability and user experience of UMClinicalStudies for both the researcher and the volunteer audience. The top criteria for this phase of the redesign included, among other things:

• • Improving the site architecture to have the Web site appear as one application though it consisted of several different components.
• • Improving entry points and accessibility into the system, particularly making the process of logging into the Registry simpler and straightforward, both for researchers and volunteers.
• • Improving navigation within the Web site.
• • Improve the search functioning within the Web site.
• • Redesigning the study posting section—concentrating on improving the workflow for researchers and study teams to post and edit studies.
• • Improving ‘matches’ between research participant volunteers and open studies.
• • Making the Web site more intuitive and inviting, especially for the volunteers.
• • Improving the interactions and the overall experience that users have with the system.
• • Improved communication with users, including email notifications to researchers and volunteers when a match is made (a relevant study for volunteers, and potential research participants for researchers).
• • Reducing the bounce rate for the Web site to below 40%.

Once the project objectives were defined, the reconstruction team went about designing the new release of UMClinicalStudies. Brainstorming sessions within the team led to the creation of new site mockups that were then tested and a final interaction design was created for the new release. Once the new design was validated, a branding treatment was applied to improve the visual appeal of the Web site so it would appear warm and inviting to users.

We found it important to design with mindfulness to the tension that exists between the two main user populations—researchers and volunteers. Both user groups have different demographics, computer experience, ways of using the system, requirements, etc., and great care and attention to detail was undertaken in order to design for solutions that would meet both their needs.

• 4Implementation of Findings

Once the wire‐framing and Web site architectural designs were complete, the mockups and design specifications were handed over to the developers to begin the implementation stage. The implementation stage consisted of developing the tool in iterative cycles with testing and analysis being done after every cycle. All team members, including the project manager (50% dedicated time), usability specialist (0.75 Full Time Equivalent [FTE]), Web designer (0.25 FTE), application developers (1.5 FTE), and users (approximately 200 hours), were involved in analyzing and evaluating every sequence.

Some key findings during analysis and evaluation were that the login links/entry points to the system for both volunteers and researchers were buried too deep within the Web site and difficult to find. As a volunteer, it was difficult to move from finding a study to actually registering an interest for that study. For a researcher, it was difficult to move from posting a new study to the site, then over to the registry to look for volunteers for the study. Other issues discovered included broken links, and dead ends. This drove the need to make logging in (or registration for first‐time users) and moving between the components a top priority.

Other focus areas for site development were more specific to the interaction design and usability. We realized that the navigation for the site was not intuitive, despite the fact that it provided more features and content compared to peer sites. It also became a priority to provide additional and more consistent “Help” throughout the site—providing guidance specific to the page that people were located on, and placing consistent action buttons with standardized labels and icons.

Through the Heuristic evaluation process, we derived that the registry enrollment process was too lengthy for user expectations—also hindered by the fact that the site at that time required people to register in one session (not allowing a “save and complete later” option), which would cause some potential volunteers to abandon the process midway. Thus, a “save and complete later” option was implemented.

Confusing terms and vocabulary like “Posting Form” to indicate editing a study posting, “Public Posting” to view a posting, etc., were also remedied by using clearer terminology. The Web site was also heavily medically texted, and was well above an eighth‐grade reading level (the recommended level for consumer‐centric health literacy), ¹³ and nonintuitive terms like “Bulletin Board” were debated upon and eventually replaced by more action‐oriented terms like “Edit a study” for researchers, and “Find a study” for potential volunteers, etc.

Furthermore, by reviewing page analytics it was determined that a large percentage of potential volunteers utilized the system to browse open clinical trials—but not sign up within the registry. Because the study postings listed on UMClinicalStudies have the research teams contact information for each study, in addition to a staffed phone line to UMClinicalStudies staff, there are still direct routes for participation made available without requiring an account. This validated the importance of having various search strategies for volunteers such as by condition, or by studies recently added. Of significance to the improved search, the Google Mini was embedded within the application—allowing users to customize their search in the various file types, yet will only see the content the individual user is authorized to view.

From the surveys that were circulated to the researchers, one of the key components that required addressing was the number of research participants that were able to be recruited through the system—it was very low for some programs, particularly the University of Michigan’s Comprehensive Cancer Center. Studies that seemed to do well were those that were seeking a large portion of “healthy volunteers” (volunteers that did not indicate a specific disease or medical condition). The effectiveness of the match between study eligibility criteria and volunteer personal information was improved by ensuring that the match was including both the previous/current condition of the registrant as well as whether they were interested in studies seeking healthy volunteers—which made a huge impact on the effectiveness of the match (due to the fact that while an individual may have asthma, they may still be considered a “healthy participant” for other studies). Staffing solutions were also identified to assist groups like the Cancer Center in contacting and screening the volunteers that expressed interest in cancer studies by connecting with professional oncology nurses.

Another key issue identified by research teams was streamlining their workflow. Thus, we integrated study entry information into eResearch—the Web‐based system that centralizes the review and approval process for Human Subjects Research Applications at UM. Information is pulled from the protocols entered into eResearch and prepopulates the posting information into UMClinicalStudies. This process ensures that research teams only need to enter study information into only one place.

By incorporating an optional email notification system—where volunteers are notified if new studies are matched to them, and researchers are notified when potential research participant volunteers have requested contact or are matched to their study based on eligibility criteria—the system has increased not only the awareness of opportunities, but also instigated a swell in user interest by just this simple transaction. Volunteers have been more proactively seeking studies and researchers have reported an increase in participants through the system. As such, the number of studies posted on UMClinicalStudies has increased by over 25% since this new version has been released, and the site bounce rate for all users was reduced to fewer than 40% as of June 2010.

A complete cycle of testing was performed once the development was complete. An alpha test within the teams and the organization was followed by a beta test within selected user groups. The feedback received from these tests was incorporated back into the release to make it more user‐friendly.

Continuing Assessment of User Needs and New Initiatives

We continue to analyze and evaluate the system and try to align it with the user needs. Some new initiatives that are under current development include:

• • Further usability improvements—particularly to the volunteer registration process.
• • Further improving search capabilities to assist volunteers in finding studies more easily.
• • A medical and drug taxonomy that both volunteers and researchers can use to provide improved matching.
• • Building a query tool for researchers so they can search the large database of volunteers that already exist on other selection criteria beyond those specified in their study to allow more efficient identification of eligible volunteers.

Conclusion

It is now more important than ever with the emphasis on improving outcomes of research, and speeding the timeline from discovery to bedside, that research recruitment be effective for both volunteers and researchers. This requires constantly reviewing research recruitment methods and tools for ease of use and understanding not only for volunteers but for the researchers that are hoping to post their studies and obtain participants. Through this evaluation and restructuring process, it became clear that we must continue to offer multiple methods (beyond Web) to connect the public to clinical trial participation opportunities, and that we must constantly evolve with community and researcher needs and expectations if we are to remain relevant to both user groups. As UMClinicalStudies grows to include more studies, and to attract larger numbers and more diverse volunteers, we aim to ultimately help researchers produce even more useful results to improve health and health treatment options.

Conflict of Interest

The authors report no conflict of interest.