The Brain Health Registry: An internet-based platform for recruitment, assessment, and longitudinal monitoring of participants for neuroscience studies

Abstract

INTRODUCTION

Recruitment, assessment, and longitudinal monitoring of participants for neuroscience studies and clinical trials limit the development of new treatments. Widespread Internet use allows data capture from participants in an unsupervised setting. The Brain Health Registry (BHR), a website and online registry, collects data from participants and their study partners (SPs).

METHODS

BHR obtains self and SP report questionnaires, and neuropsychological data including the Cogstate Brief Battery, Lumos Labs Neurocognitive Performance Test, and MemTrax Memory Test. Participants provide informed consent prior to participation.

RESULTS

Baseline and longitudinal data were obtained from over 53,000 and 24,000 participants, respectively. Over 13,700 participants were referred to, and over 1700 were enrolled in, clinical Alzheimer’s and aging studies, including 5 observational studies and 7 intervention trials.

DISCUSSION

Online assessments of participants and SPs provide useful information at relatively low cost for neuroscience studies and clinical trials, and may ultimately be used in routine clinical practice.

1. Introduction

The cognitive impairments and loss of function associated with brain aging and neurodegenerative diseases are a huge and growing problem for human society, with Alzheimer’s disease (AD) as the most common of the neurodegenerative diseases. AD has a prevalence of 5.4 million in the USA, and is associated with $236 billion in patient care costs [1]. Another major source of costs is caregiving for those with dementia, which totals many billions of hours of unpaid care and is associated with increased healthcare costs for caregivers due to the adverse effects of caregiving. In the absence of the development of new, disease-modifying treatments, the number of people age 65 and older with AD is expected to nearly triple by 2050 due to the aging population, with total annual costs projected at more than $1 trillion [1]. Thus, AD and other dementias are some of the most devastating and costly unmet public health needs [2].

AD is associated with the development of amyloid (Aβ) plaques and tau tangles, and is generally thought to be an (A β) facilitated tauopathy. Although no treatments have been demonstrated to slow the progression of AD, an increasing number of clinical trials are aimed at either reducing brain (A β) load with immunotherapy [3–6], or inhibiting production of (A β) with secretase inhibitors [7, 8]. A large number of possible therapeutic agents have been developed using animal models, but a major barrier to AD drug development is the cost and time of conducting clinical trials, especially costs associated with recruitment and screening, with 70–80% screen-fail rates [9, 10]. The average time to get an AD drug to clinic is estimated at 8.6 years [11], and the average cost of getting a successful disease-modifying AD drug to clinic (including the cost of failures) is estimated at $5.7 billion [12].

What can be done differently to accelerate and reduce costs of AD clinical trials? One approach is the establishment of “registries” of well-characterized candidate participants [2, 12, 13]. A number of registries have recently been developed to feed participants to a few, pre-defined clinical trial sites. Most of these registries have a narrow geographical focus, while some, such as the Dominantly Inherited Alzheimer’s Network (DIAN) [14] have a more widespread reach. There are advantages and disadvantages to local versus national registries. The major benefits of local registries are proximity of participants to relevant clinical trial sites and the greater value participants place in local academic institutions, which may motivate them to participate in research. On the other hand, the major benefits of a national registry are the greater economy of scale in maintaining the website and IRB protocol, and the ability to rapidly apply lessons learned to a large cohort. National registries also have the potential to more broadly facilitate AD clinical research by referring participants to many different studies.

A few national Internet-based registries have been established and have referred participants into clinical studies. These include the Alzheimer’s Disease Education and Referral Center (ADER Center), hosted by the National Institute on Aging (nia.nih.gov/alzheimers/clinical-trials); TrialMatch, hosted by the Alzheimer’s Association (trialmatch.alz.org); and the Alzheimer’s Prevention Registry, hosted by Banner Alzheimer’s Institute (endalznow.org). These registries collect limited participant data, such as contact information, demographics, basic medical information, and family history of AD or other dementia, to reduce burden on the participants and to facilitate ease of enrollment [15, 16]. The efficacy of using national registries to facilitate AD clinical research has not been established.

In 2012, we began to plan an online registry and database, the Brain Health Registry (BHR), with the overall goal to accelerate the development of effective treatments and preventative interventions for AD and other brain disorders. BHR recruits, screens, and longitudinally monitors cognition and function in participants, and additionally gathers information from their study partners (SPs) and caregivers. The BHR is unique in its size, geographic reach, longitudinal data collection, inclusion of online neuropsychological tests, and enrollment of participant-study partner dyads. Our website, www.brainhealthregistry.org, went “live” in early 2014 and has since amassed nearly 57,000 participants. This paper describes the development of the project, its current capabilities, the enrolled participants, and the results of efforts to refer BHR participants to clinical studies.

2. Methods

2.1. Overview of the Brain Health Registry

The BHR consists of a public website, registry, participant portal, investigator portal, and database. Anyone age 18 and over can join. Participants register by creating a username and password, sign online consent, and can perform tasks including questionnaires and neuropsychological tests (NPTs). The BHR study is classified by the UCSF IRB as “non-significant risk” with the major risk to participants being loss of privacy in the unlikely event that identifying information is inadvertently released. An external advisory board, comprised of experts and advocates in the field of Alzheimer’s disease research, meets annually to provide scientific guidance.

2.2. Registration

The registration page on the BHR website is used to collect the minimum information necessary (first and last name, email address, username, password, and month and year of birth) to determine the eligibility of the potential participant for the BHR study. Customized registration pages can be created with different branding, look and feel, and information fields.

2.3. Informed Consent

After registration, participants are directed to an informed consent/information page, which requires the participant to either “agree” or “decline” participation. The BHR consent is an information sheet (not requiring a physical signature) version of the UCSF IRB-approved informed consent. It is obtained electronically through the BHR website and does not require a digital signature or online agreement; a novel approach to obtaining consent that is rapidly growing in popularity. A waiver of signed consent was granted because the BHR meets federal regulation 45 CFR 46.117(c)(2) which states that the research presents no more than minimal risk of harm to subjects and involves no procedures for which written consent is normally required outside of the research context. A Health Insurance Portability and Accountability Act (HIPAA) form is not required for BHR participation, as current study procedures are self-report and do not fall under HIPAA’s Privacy Rule.

If a person declines the consent form, they are not considered enrolled into the BHR. The consent/information page may be printed before and after a participant agrees. Participants are able to withdraw their consent at any time by clicking the “withdraw” button in their BHR profile, or by contacting BHR directly.

In the event that an updated consent form is required, participants are automatically redirected to the updated consent form upon logging into the BHR website. They are provided with an explanation of the changes and the requirement to agree to a new consent. If a participant does not agree to the terms of the new consent, they are not allowed to participate further in the BHR.

2.4. Online Self-Report Questionnaires

Questionnaires currently in use on the BHR website are based on well validated instruments that are used either verbatim or adapted for an online setting. These questionnaires include measures of family history of AD, everyday cognition (ECog), early developmental history, sleep, diet, medical history, TBI/concussion, satisfaction with life scale, depression history, Geriatric Depression Scale (GDS), short-form health survey (SF-36), medication stability, caregiver experience, hoarding and cluttering [17–26].

2.5. Online Neuropsychological Tests

The BHR collects data on cognitive function across multiple domains, including processing speed, attention, learning, and memory, from self-administered, online neuropsychological tests (NPTs). Currently, the BHR includes the following online NPTs: 1) the Cogstate Brief Battery (CBB) [27]; 2) Lumos Labs NeuroCognitive Performance Tests (NCPT) [28]; and 3) MemTrax Memory Test (MMT) [29]. All NPTs are owned and provided by third party vendors.

2.6. Caregiver and Study Partner Portal

The Caregiver and Study Partner Portal (CASPP) allows a SP of a BHR participant to separately register, consent, and complete questionnaires. The data linked between the SP and participant includes 6 questionnaires, each of which takes approximately 3–10 minutes to complete. The questionnaires can be broadly characterized as gathering information about the participant and about the SP him/herself. Questions about the participant include a short health screener, ECog and FAQ adapted for online use, and questions about affective symptoms and disruptive behaviors associated with brain illness. Questions about the SP him/herself include demographics, a short health screener, stress, and SP relationship to participant. SP’s who identify as caregivers also complete the Caregiver Experience questionnaire within the CASPP. The CASPP was launched in the summer of 2016, in a beta testing phase, during which time feedback from participants, investigators, and other stakeholders was collected and used to optimize content and user experience.

2.7. Information Technology (IT) Platform

The BHR’s IT platform, Ebisu, is web-based software designed by Derek Flenniken, BHR’s Director of IT, to manage observational studies of human participants. Ebisu was developed using C#, JavaScript, ASP.NET MVC, SQL Server and Azure and is designed to run entirely in the Cloud. Several integrations have been developed with third-party systems. Ebisu uses two-factor authentication and claims-based authorization allowing precise control over data access and modification. Ebisu uses both on-the-wire and at-rest encryption to protect data. To date, there have been no major security breaches.

A select few members of the BHR team are responsible for the administration of the database server and have implicit access to all information stored. BHR staff are trained and certified annually on information security, Good Clinical Practice and HIPAA regulations. BHR staff members do not have permissions to access or use collaborator data unless explicitly granted through an independent data sharing agreement, or for the purpose of general metrics on the usage and performance of the BHR platform.

Ebisu has multiple capabilities, including (1) Administering tasks to participants. Currently, self-report questionnaires (some of which use branch logic) are administered to participants and SPs using Qualtrics. Participants are also directed to online NPTs administered by Cogstate, Lumosity, and MemTrax within the BHR wireframe. After completion of NPTs, participants are directed back to the BHR website in a seamless fashion; (2) Participant tracking. Ebisu can be used to track task completion and other online activities of participants, such as BHR website visits; (3) Participant communications. BHR routinely communicates with participants by email. Ebisu allows study staff to implement email communications and send them to all or only a specific subset of participants. The timing (date, day of the week, time of day) of email communication can also be customized; (4) Customized study design. Multiple sub-studies can be implemented within BHR. Sub-studies can be customized in terms of the tasks (questionnaires and NPTs) that participants complete and the schedule of task implementation. For example, tasks can be added, deleted, or the content can be modified; and the order of tasks presented to participants can be changed. Ebisu allows multiple investigators and study coordinators to design and implement sub-studies within the BHR platform; (5) Data management. Ebisu offers tools that can be used to query the BHR database based on all collected data. Data sets can be created based on queries and exported for analysis.

2.8. The Investigator Portal

An independent portal within Ebisu allows investigators and study coordinators outside of the BHR staff to register, log in, and perform tasks, including all Ebisu tasks listed above. The Investigator Portal facilitates information exchange between BHR staff and outside investigators. Investigators using BHR for clinical trial recruitment use the Investigator Portal to provide information to BHR staff about whether BHR participants referred to them enrolled in and were randomized into specific trials.

2.9. Dashboards

BHR data, collected on the Ebisu platform, is routinely visualized using a set of dashboards. Data concerning geographical location, age, gender, family history of AD, and whether participants meet typical inclusion/exclusion criteria for preclinical, prodromal, or AD dementia studies are queried from the BHR database and reported continuously on a dashboard. Dashboards also aid in tracking progress and examining results of various recruitment efforts, and customized dashboards are also available for various sub-studies. Figure 1 shows an example of a dashboard.

Figure 1.

Example of a BHR dashboard. The BHR dashboard offers up-to-date information to the BHR research team. The top navigation bar allows switching between baseline and follow-up visits. Total numbers of consented participants and subtotals of participants enrolled in the previous day/week/month are represented by the circles beneath. Completion of online questionnaires and cognitive tests are tracked bars at the bottom, and these are also available on an individual level. Also available, but not depicted in this figure, are demographic information and referral sources.

2.10. Recruitment

Participants are recruited to join the BHR from a variety of sources, including owned, paid, and earned media (Table 1). Owned media is published and released through BHR controlled communication channels, while paid media is distributed through paid sources. Earned media, also referred to as free media, is media that earns recognition with BHR influences.

Table 1.

Examples of various media used to facilitate recruit in BHR

Media Type	Examples
Owned	BHR website, www.brainhealthregistry.org BHR online social networks (i.e., Facebook page and Twitter account) BHR press releases Brochures or fliers
Paid	Online advertising, including: ○ Search (Google AdWords) ○ Social networks (Facebook ads) Direct mail, traditional or electronic Sponsorships
Earned	Publicity in news outlet (printed, television, radio, or internet) Endorsement from influencer (i.e., organization or group that promotes BHR to their network) Traditional word-of-month Organic sharing of BHR related digital content ○ Forwarding an email ○ Sharing via social media (i.e., shares, mentions, retweets, reports, reviews)

To quantify the number of participants per recruitment source, we use a combination of methods:

1. Self-reported recruitment source: All participants are asked how they learned about the BHR at registration. Participants are provided with a list of active recruitment sources, which is managed by the BHR study team and updated monthly. Participants are only able to endorse one recruitment source. Approximately 6% of participants do not report a recruitment source and about 14% report an unknown source, “Other”.

2. Trackable links: Whenever possible, trackable links that redirect to the BHR website are included in digital communication, such as digital advertisements, email, online articles, and social media posts. Trackable links can provide granular information that is unknown to the BHR participant. Like the self-reported recruitment source, information from trackable links is collected once, at registration. Trackable links are not always used, as is sometimes the case when BHR study team is not involved in the initial digital communication with the participant.

2.11. Retention/Engagement Strategies

To facilitate engagement of existing participants and study partners, the BHR sends regular emails and electronic newsletters to participants, which include announcements of new BHR features, and general educational resources focused on brain health. More information on newsletters is located in Appendix A.

2.12. Landing Pages

The BHR collaborates with partners, including companies and other organizations with a web presence. These partners may post a BHR advertisement on their website or promote BHR via emails to their mailing lists. This advertisement includes a link that takes the interested party either directly to www.brainhealthregistry.org or to a landing page that connects advertisements/partners to the BHR website. Landing pages may feature a collaborator’s logo or messaging that is an extension of the advertisement; they will also include some minimal information about the BHR and a “join now” button.

2.13. Types of BHR Collaborations

A major and unique goal of BHR is to facilitate studies other than those conducted by BHR Investigators. Collaborators include other investigators within UCSF, academic investigators outside of UCSF, advocacy organizations, and private sector entities conducting clinical neuroscience studies. Similar to research projects in general, it not possible for the BHR to respond to all requests for collaborations and referrals to studies. Therefore, the BHR team reviews each request. Decisions are based on a variety of factors including scientific merit, complexity, burden to BHR participants, value of data coming back to BHR, cost and other factors. The BHR does charge for referring subjects to some studies, depending on the nature of the study, whether requests come from for-profit or nonprofit institutions, value of data coming back to BHR, and the cost to BHR for performing the service. There are multiple ways to collaborate with BHR. Main categories of collaboration are: (1) Comprehensive referrals; (2) Direct to site referrals; (3) Co-enrollment; (4) Cooperative study; (5) Software as a Service; and (6) De-identified data sharing. These are described in detail below and in Table 2.

Table 2.

Responsibility and Feature Matrix of BHR Collaborations

	Comprehensive Referral	Direct-to-Site Referral	Co-Enrollment	Cooperative Study	Software as a Service (SaaS)
IRB of Record	UCSF	Collaborator	UCSF/Collaborator	UCSF	Collaborator
Data Owner	UCSF	–	UCSF	UCSF	Collaborator
Branding	UCSF	Collaborator	UCSF/Collaborator	UCSF/Collaborator	Collaborator
Tasks/Protocol	Full BHR	–	Customized	Customized	Customized
Data Linkage	Yes	–	Yes	No	No
Data Sharing	Yes	Yes	Yes	Yes	Yes
Participant Support	UCSF	UCSF	UCSF	UCSF	Collaborator

2.13.1. Comprehensive Referrals

Comprehensive Referrals means that appropriate participants who are registered in BHR are referred to clinical trials. BHR is IRB-approved to refer participants to studies outside of UCSF and outside of the BHR protocol. In order to be referred to studies, BHR participants must opt in to learning about future research opportunities. BHR participants selected for outside referral are emailed by BHR, with information which describes the study of interest and provides further instructions for participating in the study. The content of this referral email is generated jointly between BHR and the referring organization. In the referral email, the participant may be asked to contact the study site directly, or to login to their BHR account for next steps. In this model, BHR participants retain control of their contact information and opt-in to outside studies on a case-by-case basis. All referral programs are subject to the collaborators’ local IRB and UCSF IRB. Comprehensive referrals originate from the pool of enrolled BHR participants who have completed self-report questionnaires and/or NPT’s. This allows for screening prior to site referral, to identify likely candidates for a collaborators’ study. Appendix B shows a diagram of this type of referral.

2.13.2. Direct to site referrals

Direct to site referrals means that individuals who come to the BHR website or a specially designed recruitment landing page are directly referred to studies, without requiring enrollment in the BHR (and therefore without completion of questionnaires/cognitive tests). In this case, BHR refers candidates directly to a study site from a web-based form and/or landing webpage. An optional limited study specific online eligibility screener may also be included. Participation in any referral program requires collaborators to report enrollment progress, ideally using the Investigator Portal, so that referral programs can be evaluated and modifications can be made to optimize BHR for success. Participants who are directly referred to sites for studies are often invited to join BHR as an additional option. Appendix C shows a diagram of this type of process. Direct to site referral is currently functioning for ADNI 3. The ADNI 3 recruitment website ADNI 3.org, was developed and is operated by the BHR team. This website directly refers interested participants to ADNI 3 sites [30]. The website includes a map of all ADNI 3 sites and a 1–800 number. More details concerning this approach will be reported in the future.

2.13.3. Co-enrollment

Co-enrollment means that participants are enrolled in the BHR (sign the BHR consent) and are also enrolled in another study, with data linkage between the two studies. BHR offers the ability for collaborators to invite their research participants to join the BHR, with the goal of linking study data collected by both research studies to create a more enriched dataset for analysis, and to inform future research projects. Some collaborators may request that their co-enrolled participants be presented with a co-branded and/or modified BHR experience to better suit their study needs. The BHR has the flexibility to offer these co-enrolled participants a subset or alternative set of questionnaires and/or cognitive tests. The BHR can also tailor participant communication and study visit frequency to the specific needs of the collaborator. The BHR registration page may include a field to enter a unique identification code that is provided to co-enrolled participants, via email and other recruitment communications from a specific recruitment source. This code would identify the participant as associated with a specific source. In co-enrollment collaborations, the BHR is responsible for obtaining each participant’s consent to share BHR data with collaborating investigators; consent is required because collaborators will receive identifiable BHR data. The Co-Enrollment Process Flow is detailed in Appendix D.

2.13.4. Cooperative study

A Cooperative study means that the BHR platform is used for a collaborative study, but the Cooperative study is not primarily branded as the BHR. A Cooperative study is branded to suit the collaborator. BHR originated in the San Francisco Bay Area and operates under UCSF IRB. Although UCSF is a well-known and well-respected academic institution, it may not have the same influence in other geographic areas than a local academic institution or national foundation. Therefore, BHR offers the option to create a Cooperative study. A Cooperative study allows the BHR experience be branded with a local or national name in order to enhance credibility and visibility at a specifically targeted audience. Participants who join a cooperative study will still sign a UCSF BHR consent and be considered BHR participants. Similar to co-enrollment, cooperative studies can be tailored to either include the full BHR study protocol or be modified to include fewer or alternate study tasks. Cooperative studies are operated by BHR staff, who handle all participant communication and engagement activities. Cooperative study data is pooled with the full BHR dataset. Depending on the preference of the collaborator, participants recruited into Cooperative studies may be sequestered from comprehensive referral programs. Additionally, de-identified data from a Cooperative study is provided to collaborator for analysis and publication. In summary, a cooperative study is a custom branded version of BHR with data sharing under UCSF IRB.

2.13.5. Software as a Service (SaaS)

SaaS means that a collaborator uses the BHR software, but has complete responsibility for all aspects of the study. The web-based platform that the BHR has developed to manage online studies and registries is now being made available to other groups for managing their own online studies and registries. As this platform is provided as a service, the BHR is responsible for the development, maintenance, and management of the software and servers upon which it runs. Clients of the service are able to design their own studies, customized to include their own questionnaires, tests, consent forms, colors and logos. Clients who utilize SaaS are responsible for obtaining all necessary approvals prior to beginning their research, as research outside the BHR study protocol is separate. The de-identified data from a SaaS project could, or could not, be pooled with the de-identified BHR data, depending on the wishes of the client.

2.13.6. De-identified data sharing

BHR allows and encourages sharing of the entire, de-identified BHR database with qualified investigators outside of the BHR research group, governed by a Data Use Agreement (DUA), as well as all collaborators using BHR services. Investigators who are approved for data use will be provided the data directly from the IT team. The DUA requires that the applicant agree to terms including but not limited to: (1) Will not attempt to establish the identity of, or contact any BHR participants; (2) Will not further disclose these data beyond the uses outlined in the agreement; (3) Will req okay uire anyone who utilizes these data, or anyone with whom they share these data, to comply with the data use agreement; (4) Will comply with any rules and regulations imposed by their institution and its IRB in requesting these data. BHR recommends that all collaborators with BHR, including those using SAAS, agree to de-identified sharing of their data. Sharing of de-identified data also allows those collaborators who are using BHR to build cohorts for comparing their results with other BHR studies.

3. Results

3.1. Recruitment

Figure 2 shows daily cumulative enrollment as of January 15, 2018. Since its inception, over 500,000 users have visited the BHR website. 66,322 individuals registered, meaning created a unique account within our system, and of those, 56,982 consented to participate, thereby enrolling in the study, and 4,372 consented to participate as a study partner. BHR participants have been recruited from a variety of sources, such as a news story, word-of-mouth, Internet ads, social media, other registries/research studies, email, and advocacy groups. Approximately 6% of participants do not report a recruitment source and about 14% report an unknown source, “Other.”

Figure 2.

Cumulative enrollment of BHR participants. Daily cumulative totals of enrolled BHR participants until January 15, 2018.

3.2. Initial Launch

The initial launch of BHR was on April 8, 2014, and was spearheaded by Cater Communications, a bipartisan firm specializing in communications and public policy strategies. Using traditional advertising and earned media activities, Cater was successful in securing several front page news stories in Bay Area publications, including the San Francisco Chronicle; San Francisco Examiner; and the San Jose Mercury News, as well as obtaining public service announcements by professional baseball player, Sean Doolittle of the Oakland Athletics, and professional basketball player, Draymond Green of the Golden State Warriors. April 8, 2014 was proclaimed as Brain Health Registry Day by The City and County of San Francisco, California; The City of Oakland, California; and the Marin County Board of Supervisors. Additionally, a brief digital advertising campaign was conducted to bolster the traditional public relations strategies related to the initial launch. These activities resulted in approximately 4,000 new participants in the week of the launch.

3.3. First Landis Communications Campaign

In late 2014, BHR began working with Landis Communications, a full-service public relations and communications agency. After a brief ramp-up period, Landis began a robust digital advertising campaign in the Bay Area, which immediately doubled the number of new participants who enrolled in BHR each month. In addition to significantly increasing the number of BHR participants through digital advertising and mounting an earned media campaign, Landis also worked with BHR staff to train them in media relations and bolster their public speaking abilities.

3.4. B. Smith Public Relations Campaign

On March 13, 2015, BHR was featured on the Today Show by newly secured spokesperson, B. Smith. B. Smith, a former model, restaurateur, and TV host, had recently been diagnosed with AD, and shared her powerful story with the national audience of the Today Show. Landis Communications secured the Today Show segment with B. Smith and combined the publicity with a designated digital advertising campaign. The entire campaign brought in almost 6,000 new BHR participants and elevated the Brain Health Registry to national recognition. The B. Smith campaign also increased minority representation in BHR, leading to enrollment of 872 African American participants (13.7% of total cohort recruited) and 1571 total non-Caucasian participants (24.7% of the total cohort recruited). Altogether, the B. Smith campaign resulted in an increase in the percentage of African Americans in the BHR cohort from 1.5% to 4.9%.

3.5. Global Alzheimer’s Platform Foundation (GAP) campaign

BHR continued to work with Landis Communications in collaboration with the Global Alzheimer’s Platform Foundation (GAP). The GAP campaign, like the B. Smith campaign, utilized a robust digital advertising and earned media campaign in several US cities, including but not limited to Boston, Massachusetts; Las Vegas, Nevada; Los Angeles, California; Providence, Rhode Island; Atlanta, Georgia; and Delray Beach, Florida. That GAP campaign also included partnership between BHR and UsAgainstAlzheimer’s Networks as well as GAP affiliated sites, such as Cleveland Clinic Lou Ruvo Center for Brain Health (Las Vegas, NV), Butler Hospital (Providence, RI), and Brigham and Woman’s Hospital (Boston, MA). Landis also initiated new advertising strategies with this campaign including a contest to have lunch with actress Linda Gray, from CBS-TV’s “Dallas,” called #GrayMatters. Landis also secured public service announcements with singer and entertainer Paula Abdul and Ronald Reagan, Jr. To date, the GAP campaign has brought in over 9,600 new BHR participants.

3.6. Partnerships and Cross-Promotion

In addition to working with public relations agencies, BHR partnered with non-profit and advocacy organizations focused on Alzheimer’s research and support, to help increase public awareness about BHR and assist with study enrollment. These organizations promoted BHR to their networks, via email campaigns, newsletters, social media, and endorsement on their websites. In 2014 and 2015 Lumos Labs, providers of the NCPT, emailed over seven million of their members in support of BHR. BHR also partnered with other research groups and hospitals to cross-promote BHR and studies taking place at their centers. Printed BHR recruiment materials we placed in waiting rooms, at reception desks, and on bulletin boards, and links to the BHR website were shared on partners’ websites. Finally, BHR is listed in AD-centered internet-based registries, ADEAR Center, TrialMatch, and Alzheimer’s Prevention Registry, as well as the Parkinson’s focused internet registry Fox Trail Finder, hosted by the Michael J. Fox Foundation (foxtrialfinder.michaeljfox.org).

3.7. Additional Recruitment Strategies

BHR study staff attended community events and scientists gave talks, including presentations before fellow UCSF faculty and staff. In 2015 BHR obtained access to a UCSF patient list and emailed over 100,000 patients asking them to participate in BHR; approximately 6% enrolled between February 2015 and November 2015.

3.8. Costs of registry development and maintenance

The overall investment in the conception, design, programming, management, testing up to the initial launch of the BHR took 2.5 years and was approximately $2,300,000 million in direct costs. This includes $350,000 in advertising costs and $450,000 in IT development costs. Since the launch more than $5,000,000 was spent over 3 years to fund the activities described in this report. The ongoing annual operating costs are approximately $2,000,000 to operate our registry, and support existing referral, co-enrollment and software-as-a-service programs. Therefore more than $7,000,000 was spent to create and operate the BHR registry with approximately 53,000 participants, including its associated programs.

3.9. Demographics of Participants

As of January 15, 2018, there were 56,982 participants enrolled in BHR, including 35,901 adult volunteers over age 55. Of this cohort of older participants, 1735 self-report MCI and 212 self-report AD. 93% of the cohort indicated they are willing to be contacted for future studies. Participants report residence throughout the United States (Figure 3); 22% of participants report residence within one of the nine Bay Area counties. Characteristics of the total cohort are described in Table 4.

Figure 3.

Location of BHR participants. Heat map of the geographical distribution of BHR participants who claim residence in the United States based on self-report data.

Table 4.

Summary of Referral Programs

Study Type/Description	Referred	Contacted site or BHR	Interested	Screened by Site	Eligible	Ineligible	Enrolled
Interventional
Phase 2, randomized, placebo-controlled drug treatment intervention for mild AD	33	6	2	1	0	1	0
Phase 3, randomized, placebo-controlled drug treatment intervention for prodromal AD	139	48	28	0	0	0	0
Phase 3, randomized, placebo-controlled drug treatment intervention for prodromal AD	129	15	11	0	0	0	0
Phase 4, randomized, placebo-controlled drug treatment intervention for cognitively normal adults	284	70	28	3	0	3	0
Phase 3, randomized, placebo-controlled secondary AD prevention trial	1212	340	159	62	21	41	21
Randomized trial evaluating rTMS vs. sham treatment on cognitive function	269	57	15	4	1	3	2
Caregiver study, investigating tablet intervention effects on attentional awareness in AD caregivers	14	4	1	1	1	0	0
Total	2080	540	244	71	23	48	23
Percent of total referred	100.0%	26.0%	11.7%	3.4%	1.1%	2.3%	1.1%
Non-Interventional
Observational
Olfaction and cognition study, participation online & in home	3236	1841	1673	1148	1147	1	1148
Aging/cognition study for cognitively normal adults	1221	140	96	75	54	21	38
Aging/depression study for older adults with Major Depressive Disorder	2176	366	208	223	20	203	25
Study of AD biomarker in cognitively normal, MCI and AD adults	5359	1059	740	23	22	1	22
Site Referral
Multiple studies conducted at this site	531	131	105	32	22	10	22
Multiple studies conducted at this site	475	78	64	30	23	7	0
Validation
Study is conducted to determine validity of online cognitive measures compared to standardized, cognitive measures, obtained in clinic	3509	903	591	576	526	50	503
Study is conducted to determine validity of psychiatric instruments collected online for use among individuals with hoarding disorder. Online data will be evaluated in comparison to clinical interview.	267	61	52	28	17	11	17
Total	16774	4579	3529	2135	1831	304	1775
Percent of total referred	100.0%	27.3%	21.0%	12.7%	10.9 %	1.8%	10.6%
Total, across all studies	1 8854	5119	3773	2206	1854	352	1798
Percent of total referred	100.0%	27.2%	20.0%	11.7%	9.8%	1.9%	9.5%

This table includes aggregate date from referral programs.

Screened by site includes initial prescreen and study screen.

Enrolled in study indicates that the referred participant signed the referral study consent

3.10. Online Self-Report Questionnaire Completion Rates

At baseline, 98% of participants completed at least 1 self-report questionnaire: 36% completed less than half of questionnaires, 22% completed more than half of questionnaires, and 40% completed all questionnaires. The baseline initial questionnaire is presented to participants as the first online procedure in their protocol, and was completed by 96% of all participants that completed a questionnaire. Apart from this, questionnaires have similar completion rates, between 53% and 61%. Participants that identify as Caucasian have a higher completion rate of all questionnaires (43%) compared to the total cohort, as well as a higher rate compared to participants that identify as African American (23%), Asian (34%), and Latino (30%).

3.11. Online NPT Completion Rates

At baseline, 61% have taken at least 1 NPT: 23% have completed 1 NPT, 22% have completed 2 NPT’s, and 16% have completed all 3 NPT’s. Caucasian participants have a higher completion rate of all 3 NPT’s (17%) compared to the total cohort, and also compared to African American (7%), Asian (12%), and Latino participants (11%). Forty-three percent of all BHR participants have taken the NCPT, 30% have taken MMT, and 41% have taken CBB at baseline. At baseline, increasing age (β= −0.009, z=−10.24, p < .001), education (β= −0.327, z=−10.63, p < .001), and having a parent with a memory problem (β= −0.077, z=−3.13, p < .01) were all associated with increased likelihood to complete online NPT’s. In contrast, women (β= 0.310, z= 10.46, p < .001) having a memory concern (β= 0.097, z= 3.85, p = 0.001), and having a diagnosed memory problem (β = 0.180, z= 2.22, p = 0.001) were associated with decreased likelihood of completing online NPT’s. At follow-up evaluations, similar results were seen.

3.12. Longitudinal Data Collection

The BHR invites participants to return at 6-month intervals to complete online follow-up questionnaires and to repeat online NPT’s. 27,957 participants have longitudinal data. To date, the BHR return rates for 6, 12, 18, 24, 30, 36 and 42-month follow-ups are 41%, 32%, 28%, 25%, 23%, 28% and 26% respectively. At 12-months, Caucasain participants have a higher return rate (35%), compared to the total cohort, as well as African American (13%), Asian (26%), and Latino participants(20%). Similarly, Caucasian participants have higher return rates at 24-months (28%) and 36-months (30%) compared to the total cohort, and in comparison to African American (12% and 18%), Asian (20% and 18%), and Latino participants (26% and 19%). The cohort of returning participants is slightly older (average age = 58.2, standard deviation = 13.2) compared to participants who only complete a baseline visit (average age = 55.0, standard deviation = 14.4). This data demonstrates the feasibility of using BHR to assemble a large cohort of adult volunteers for clinical neuroscience studies, and to follow them longitudinally.

3.13. Study Partner Caregiver Assessment Portal

As of January 15, 2018, a total of 8,951 BHR participants have invited a SP to join. Of those invited, 4,400 (49% of those invited) have enrolled and signed online consent. Of all enrolled SPs, 2,589 (59%) completed all tasks, and 2,655 (60%) SP’s completed the Caregiver Experience questionnaire. Of SPs who completed the Caregiver Experience questionnaire, 504 (19%) self-identify as the caregiver of their associated participant [31].

3.14. Comprehensive Referral Collaborations

Participants that we predict to be eligible for participation in collaborators’ clinical research were identified using BHR data. While referral criteria varied depending on the study design and site recruitment goals, participants were generally identified through self-reported data: (i) aged 55-85, (ii) memory concerns, (iii) no current major neurological disease or other exclusionary diagnosis, (iv) not currently taking anti-AD drugs, (v) no recent history of drug/alcohol abuse and (vi) within range of a site. In some cases, referral criteria also included cognitive data targeting those who scored lower on a test or outcome. Participants identified as referral were emailed with study information and contact information of the site recruiter. Nearly all collaborators/site staff used the online Investigators Portal to report screening and enrollment on BHR participants who made contact. Table 4 summarizes status of referral collaborations as of January 25, 2018.

3.15. Software as a Service (SaaS)

BHR has created a flexible framework which allows others to use the BHR software for their own research purposes (described in Methods). Currently, BHR has partnered with an academic group in the Netherlands to launch a Dutch version of BHR (BHR-NL) in early fall of 2017. BHR-NL will be regulated by and operating under a Dutch IRB and will contain a single neuropsychological test with a few standardized questionnaires. The purpose of BHR-NL is to establish a Dutch research registry for comprehensive referrals to European clinical research studies and industry-sponsored trials. This collaboration will be described in future manuscripts.

3.16. Adverse Events

Several reportable events and incidents have occurred. The BHR has received emails from participants expressing serious concerns about their health, or have answered questions from participants that suggested there may be threats of violence to themselves or others. In these cases, each participant was informed that the BHR does not provide clinical care and were provided with various alternative resources. In some instances, BHR participants have received inadvertent emails from BHR collaborators inviting them to join non-approved activities. In these cases, participants were sent emails apologizing for these non-approved solicitations. All adverse events were reported to the UCSF IRB. These experiences have been beneficial to the BHR team for developing strategies to manage the study in the future.

4. Discussion

This report demonstrates the feasibility of several novel approaches: 1) development of a website, the BHR, that captures self-report, SP report, and NPT information; 2) use of BHR to enroll participants and obtain informed consent approved by the IRB; and 3) recruitment of large numbers of participants using the BHR website. Our study has also demonstrated that large numbers of participants in BHR provide longitudinal data. Collection of longitudinal data is a unique and powerful feature of the BHR that allows us to identify signs of cognitive decline, and enables us to maintain a long-term relationship with participants that may increase their willingness to participate in future clinical studies. Today, to our knowledge, no other website features the unique combination of longitudinal self-report, SP report, and NPT data obtained by the BHR. These results show that a properly designed and appropriately approved research website can effectively capture valuable data from large numbers of participants, an approach that will likely find increasing use for research, and ultimately clinical practice.

When the project was conceived 5 years ago, serious doubts were expressed about the feasibility of an online registry of this nature. It was unclear whether the UCSF IRB would approve of the use of online methods for obtaining informed consent. A further concern at the outset of the project was whether the registry would be scalable beyond a small number of participants. Currently, the BHR has nearly 57,000 participants who have provided electronic consent, and is growing at a rate of approximately 500 registrants per month, clearly demonstrating its scalability. The major limitation to the growth of BHR are funds required for public relations and advertising.

One of the unique and powerful features of the BHR approach is the creation of a long-term, centralized “pool” of potential participants for clinical research. Importantly, the online data collected within the BHR can be queried using study-specific inclusion and exclusion criteria to identify participants that are likely qualified for referral to other studies or trials. This approach has the potential to reduce screen fails, an important advantage of using the BHR for “prescreening.” Longitudinal data collection and regular communication with participants facilitates long-term engagement, and if participants are found to be ineligible for a specific study, they do not need to be recruited de novo for future studies.

Our results demonstrate that we can successfully refer large numbers of participants into neuroscience clinical research studies, and emphasizes the challenge of recruiting participants from registries into randomized treatment trials. The vast majority of participants referred to a collaborator’s study joined a non-interventional study; the success rate for enrollment into treatment trials was only 1%. There are several explanations for this, which we discuss below, and will be more fully explored in future studies.

Although BHR has a very large overall number of participants, they are spread out over the entire USA and beyond. This creates a challenge in recruiting to a specific trial site. This challenge can be addressed in the future by more targeted advertising and public relations in specific geographical areas, as well as establishment of partnerships with local universities and organizations that may lend brand name recognition. For example, the use of custom landing pages with local university branding may increase local recruitment success. In support of this idea, we had greater success enrolling in the San Francisco Bay Area as approximately 22% of BHR participants report residence within on the nine Bay Area counties; this may be because of UCSF brand recognition and recruitment efforts that were specific to the Bay Area.

The BHR online model presents unique challenges in successful screening for intervention studies. The AD intervention studies we were referring participants into had more restrictive entry criteria, for example age 60–80 for inclusion, compared to the observational studies. This resulted in a larger number of participants eligible for observational versus intervention studies. We believe enrolling more adults into BHR overall will enable us to identify even more participants who meet stricter entry criteria for AD interventional trials. Increased longitudinal data collection can also help future screening and referral efforts. Participants who did not initially meet eligibility criteria may convert to referral-eligible while providing longitudinal follow up data to BHR. Collection of longitudinal NPT data will also allow us to better identify those undergoing cognitive decline, who are likely to be at risk for future decline and conversion to MCI and AD dementia.

The online recruitment model of BHR is likely to select for participants who are likely not treatment seeking and/or are less willing to participate in clinical research that requires multiple in-clinic visits, invasive procedures, and greater time commitment. This idea is supported by our referral data: less than 30% of participants referred to a study successfully initiated contact with the study site, a surprising finding given that 93% of participants said they’d be interested in learning about other research studies. Moreover, BHR participants that were referred and interested in participation in a collaborator’s study needed to contact study sites, and this may help explain low rates of contact. We are currently exploring the option of transferring the burden of initial contact from the participant to the study site, which may increase response and study enrollment rates. Lastly, we also acknowledge that the level of commitment needed from participants to enroll in the online registry and complete screening assessment maybe too high. In response to this concern, we now offer collaborators direct to site referrals, where individuals are directly referred to studies, without requiring enrollment in the BHR and completion of questionnaires/cognitive tests. Future studies will report insights regarding this approach.

Constraints on participation from trial sites may also be a contributing factor to low referral rates into intervention studies. In all current referral programs to intervention trials, BHR was not written into trial protocols and additional steps had to be taken by the study to support BHR referrals, including protocol amendments, which require time and other resources. Ideally enrollment from BHR would be built into a study’s protocol at outset of a trial, and all necessary steps taken so BHR is able to receive sensitive information, including reason for screen fail, thereby allowing us to better assess referral programs.

Another limitation with current referral programs is that BHR is reliant upon collaborating investigators/their staff to report enrollment information. There have been delays in receiving this enrollment information and sites have little incentive to provide updates in real-time. In the future BHR will be pursuing collaborations where BHR is more integrated into the referral study so that we may access enrollment and screening status without causing burden to the site. We conclude that online registries can be used to populate clinical neuroscience studies, but the utility of such an approach to facilitate treatment trials is not clear and requires further analysis of new referral strategies.

The major limitations of this project concern validity and generalizability. Although the BHR has collected information on nearly 57,000 participants, we have no way to directly determine whether all individuals actually exist, and whether the data is valid. Furthermore, it is unknown whether cognitive measures obtained unsupervised and online, via the BHR are comparable to cognitive measures obtained in-clinic, under supervision, using validated measures. The BHR is currently engaged in a funded validation project in which individuals are contacted and brought in for in-clinic assessments. The results of this effort will be reported in future publications.

Generalizability is also a concern. The number of participants from minority populations including African Americans, Latinos, and Asian Americans is considerably lower than US census data. In addition, the education level of BHR participants is greater than US census data. Furthermore, approximately 74% of the all BHR participants are female. Underrepresentation of traditionally-underserved groups is a limitation of many in-clinic studies which occur at research institutions. However, targeted public relations efforts have yielded promising results. For example, The B. Smith campaign described resulted in recruitment of cohort comprised of 13.7% African Americans and 24.7% total non-Caucasians. This campaign brought to BHR 872 new African American participants, 455 of whom were age 55 and over. At the time of the campaign, this resulted in an increase in the percentage of enrolled African Americans in BHR from 1.5% to 4.9%. Unfortunately, the BHR team has been unable to replicate the B. Smith campaign in another way, and we note that no similar successful campaigns have been performed in the AD field. We believe this is a major problem in the AD field and that it will take substantial financial resources to mount a major marketing, public relations and advertising campaign to increase minority and low SES enrollment.

Some may view the inclusion of adults across the age span as a limitation in terms of facilitating AD studies, which of course focus on older individuals. However, we do not view this as a limit but rather a benefit of our approach. BHR was never designed to focus solely on AD studies or studies of older people. From the onset our intention was to avoid a strong “Alzheimer’s Disease” brand, and attract a broad population including many cognitively-normal participants and individuals with various problems across the age spectrum. This has allowed us to study age effects, sleep problems across the age spectrum, traumatic brain injury, and to refer participants for many types of clinical studies. Nonetheless, the BHR cohort includes over 27,000 people age 60 and older, and approximately 10,000 people age 70 and older. This is likely due to our recruitment efforts specifically targeted older adults, such as co-branding with AD advocacy groups and Internet advertising about older adult brain health. We hope to follow younger participants for many years, where they eventually would transition into this older age bracket. Younger family members enrolled in BHR may eventually be used to recruit older adults into studies.

Finally, it is obvious that only individuals who have access to the Internet with appropriate devices can participate in the BHR, and individuals without such access are excluded from participation. Additional selection biases are also likely, such as biases for cognitively healthy individuals over those with cognitive problems or cognitive decline. Selection biases are important issues that must be addressed in future analyses of participant characteristics associated with missing data and retention.

As we have gained experience and better understood the value of our approach, we have expanded the goals of this project beyond the original focus of facilitating AD treatment trials. The future plans for the BHR include development of improved methods to recruit SP’s and caregivers, implementation of the BHR SaaS, additional validation studies of online methods, and enhancement of the user experience to improve engagement.

The significance and impact of this project lies in the demonstration that a properly designed and appropriately approved research website effectively captures a range of valuable demographic, and neuropsychological data, from large numbers of participants from a wide geographical area. We believe that the broad scope and reach of the BHR has the potential to address many of the problems associated with both traditional methods for the recruitment of participants in clinical studies, and with existing registries for this purpose. By demonstrating the feasibility of this approach, we expect that online registries of this nature have the potential to develop into an indispensable tool to facilitate clinical trial enrollment, recruitment, and prescreening, to facilitate screening in many different healthcare settings, reduce trial costs, and accelerate the development of new treatments.

Table 3.

Demographics of the complete cohort

Demographics	N	%
Total	56,982
Age
<50	15,772	27.7%
50–59	14,504	25.5%
60–69	17,255	30.3%
70–79	8,285	14.5%
>80	1,611	2.8%
Female	42,117	73.9%
Race/Ethnicity
African American	2,584	4.5%
Asian	1,810	3.2%
Caucasian	46,109	80.9%
Hawaiian/Pacific Islander	104	0.2%
Hispanic/Latino	3,041	5.3%
Native American	242	0.4%
Other	1842	3.2%
More than 1 race	2,051	3.6%
Not Collected	1,367	2.4%
Declined to report	873	1.5%
Education
High School or less	6,413	11.3%
Some College	10,057	17.6%
2 Yr College Degree	4,685	8.2%
4 Yr College Degree	16,934	29.7%
Advanced Degree	18,893	33.2%
Memory data
Memory concern	24,267	42.6%
Family history of AD	14,267	25.0%
Diagnosed with MCI	2191	3.8%
Diagnosed with AD	251	0.4%
Diagnosed with dementia	349	0.6%
Medical Condition
Parkinson’s	1,146	2.0%
Movement Disorder	1,496	2.6%
Motor Neuron Disease	204	0.4%
Stroke	933	1.6%
Schizophrenia	99	0.2%
Heart Disease	2,281	4.0%
Blood Pressure	10,524	18.5%
Cholesterol	12,458	21.9%
Diabetes	2,610	4.6%
Cancer	5,015	8.8%
Alcohol abuse	3,964	7.0%
Drug Use	2,465	4.3%
Smoking	13,373	23.5%

Research in context.

1. Systematic review: the authors reviewed the literature by traditional sources (Pubmed and Google Scholar), and by personal communication with authors.

2. Interpretation: the BHR has successfully enrolled over 53,000 participants through its website, and characterized both participants and study partners through self-report questionnaires, and participants through online neuropsychological tests. A substantial proportion of participants have been followed longitudinally, and over 1700 participants have been enrolled in neuroscience studies demonstrating the feasibility of an online registry for providing a pool of characterized potential participants clinical trials of Alzheimer’s treatments

3. Future directions: continued longitudinal tracking of participants, and enrollment of new participants and study partners will provide an even greater pool of potential participants in Alzheimer’s clinical trials, reducing costs and time associated with enrollment of participants in these studies.

Appendix A

Summary of newsletters sent by the BHR to participants, SPs, and registrants.

Newsletter Date	Number of People Newsletter Sent To	Topic Summary	Unique Open Rate
Sep 2014	–	Enrollment update: 9,000 participants Information on longitudinal visits Call to action: Help spread the word. Invite friends and family to BHR	–
Dec 2014	8,207	Enrollment update: 10,000 participants Calls to action: ○ Help spread the word. Invite friends and family to BHR ○ Men under-represented in the BHR. Ask male family members and friends to join Participants return in huge numbers BHR researchers presenting at conferences	65%
Mar 2015	15,929	B. Smith partners with BHR and records PSA Ethnic communities often under-represented in clinical trials Call to action: Help spread the word. Invite friends and family to BHR	57%
Apr 2015	19,033	1 Year Milestone Return Visits BHR partners with B. Smith Additional Research Opportunities Calls to Action: ○ Continue participation by returning for follow-up visits ○ Help us grow. Tell friends and family to BHR	65%
Jul 2015	22,164	Enrollment update: 26,000 participants Participate in additional research Call to Action: BHR Challenge Continues, forward email to one family member/friend	55%
Nov 2015	26,560	Enrollment update: 30,000 participants New Features: BREEZE Health History, Head Injuries and Concussion questionnaire, MemTrax Memory test Learn about associations between cardiovascular and brain health and participate in UCSF’s Health eHeart Study Call to Action: Invite friends and family to BHR	52%
Feb 2016	30,222	Healthy activities with positive impact on cognition (i.e., nutritious diet, exercise, social interaction) Happy Valentine’s Day Return to the BHR Global Alzheimer’s Platform Foundation Partners with BHR	38%
Apr 2016	32,750	BHR celebrates two years Enrollment update: 40,000 participants Calls to Action: ○ Encourage friends to BHR ○ Enter raffle to win lunch with Linda Gray Remembering Nancy Reagan	45%
Jun 2016	34,488	Alzheimer’s and beta amyloid BHR as an Add-On Study for the IDEAS Study Happy Father’s Day Call to Action: Tell friends and family to BHR	50%
Aug 2016	36,074	BHR researchers and other highlights from 2016 AAIC BHR partners with Monell Chemical Senses Center for olfaction study	43%
Oct 2016	37,266	Potential AD treatments in the news Information on clinical trials	44%
Dec 2016	41,574	BHR gives thanks to caregivers Calls to Action: ○ Caregivers please complete the new Caregiver Experience questionnaire ○ Invite Caregivers to join BHR	37%
Mar 2017	41,796	BHR launches two new features: Caregiver and Study Partner Initiative (CASPP) and Hoarding & Cluttering questionnaire	44%
May 2017	41,481	Special message of gratitude from BHR Principal Investigator BHR celebrates three years	39%
Jul 2017	43,435	Importance of Brain Donation in research Examples of national tissue or brain banks and other resources	34%
Sep 2017	44,077	Cognitive Test: traditional verse computerized versions Overview and summary of results from a sub-study of 200 Brain Health Registry participants conducted to determine reliability and validity of a computerize cognitive assessment develop by an industry collaborator	38%
Dec 2017	41,197	Preliminary results for analysis of study partner reported data Announcing the launch of longitudinal visits for Study Partners Call to action: ○ Study partners continue participation by returning for follow-up visits	34%

Appendix B

Appendix C

Appendix D

Co-Enrollment Process Flow: BHR offers the ability for interested groups to invite and track their research participants by having their participants join BHR.

Appendix E

Brain Health Registry team organizational chart