Digital Endpoints: Definition, Benefits, and Current Barriers in Accelerating Development and Adoption

Abstract

The assessment of health and disease requires a set of criteria to define health status and progression. These health measures are referred to as “endpoints.” A “digital endpoint” is defined by its use of sensor-generated data often collected outside of a clinical setting such as in a patient's free-living environment. Applicable sensors exist in an array of devices and can be applied in a diverse set of contexts. For example, a smartphone's microphone might be used to diagnose or predict mild cognitive impairment due to Alzheimer's disease or a wrist-worn activity monitor (such as those found in smartwatches) may be used to measure a drug's effect on the nocturnal activity of patients with sickle cell disease. Digital endpoints are generating considerable excitement because they permit a more authentic assessment of the patient's experience, reveal formerly untold realities of disease burden, and can cut drug discovery costs in half. However, before these benefits can be realized, effort must be applied not only to the technical creation of digital endpoints but also to the environment that allows for their development and application. The future of digital endpoints rests on meaningful interdisciplinary collaboration, sufficient evidence that digital endpoints can realize their promise, and the development of an ecosystem in which the vast quantities of data that digital endpoints generate can be analyzed. The fundamental nature of health care is changing. With coronavirus disease 2019 serving as a catalyst, there has been a rapid expansion of home care models, telehealth, and remote patient monitoring. The increasing adoption of these health-care innovations will expedite the requirement for a digital characterization of clinical status as current assessment tools often rely upon direct interaction with patients and thus are not fit for purpose to be administered remotely. With the ubiquity of relatively inexpensive sensors, digital endpoints are positioned to drive this consequential change. It is therefore not surprising that regulators, physicians, researchers, and consultants have each offered their assessment of these novel tools. However, as we further describe later, the broad adoption of digital endpoints will require a cooperative effort. In this article, we present an analysis of the current state of digital endpoints. We also attempt to unify the perspectives of the parties involved in the development and deployment of these tools. We conclude with an interdependent list of challenges that must be collaboratively addressed before these endpoints are widely adopted.

Contextualizing the Benefits of Digital Endpoints

To understand the potential of digital endpoints, we must first examine the deficiencies of traditional endpoints. The assessment of Parkinson's disease has long relied on subjective and rater-dependent methods of in-clinic measurement [1, 2]. Consequently, clinical judgment of disease burden is limited, and the clinical trials that rely on traditional measures are expensive and prone to false positives or negatives [2, 3]. The motor symptoms typically present in patients with Parkinson's disease however can be objectively and efficiently captured by smartphones and other low-cost sensors [4, 5]. Initial efforts to use these data for disease measurement have been promising. For example, using smartphone data to measure voice, finger tapping, gait, balance, and reaction time, researchers trained a machine learning model to construct an objective Parkinson's disease severity score that measures a patient's burden with much greater frequency and more accurate context than gold standard methods [6].

Heart failure is another prominent example in which traditional endpoints are often insufficient. The traditional primary clinical endpoints for heart failure are cardiovascular death and hospitalization with heart failure [7]. While these are often appropriate endpoints, they are also coarse and only allow physicians to assess pathophysiology as discrete variables.

While there are alternative heart failure endpoints such as clinical symptoms, jugular venous pressure, laboratory values like NT-proBNP, and imaging methodologies, they typically require a patient to visit an office or laboratory; thus, data arrive in a series of snapshots. Consequently, traditional endpoints only allow the assessment of the patient's disease in a clinical setting; they do not offer insights into the patient's daily burden of symptoms or physical constraints. This is particularly problematic for diseases with persistent and limiting symptoms. It is therefore predictable that patients with advanced heart failure often emphasize quality of life over duration of life [8]. Surrogate endpoints such as the Kansas City Cardiomyopathy Questionnaire, the Minnesota Living with Heart Failure Questionnaire, and the New York Heart Association Classification attempt to elucidate the disease's effect on a patient's daily life; however, these patient-reported data are typically sensitive to extreme developments in symptom severity but are often insufficient to indicate more subtle shifts to a patient's quality of life [9].

Digital endpoints offer benefits beyond allaying the shortcomings of traditional endpoints. For example, digital endpoints can provide efficient and accurate screening, enable improved analysis regarding whether to proceed with a drug's development, allow for earlier disease diagnosis, and enable continuous monitoring of a patient's disease state [10].

Importantly, utility cannot be derived from all data; evidence must be of high quality and relevance. Data used to improve drug trial success rates should allow researchers to glean insights into a therapy's effect on the way a patient feels, functions, or survives. This is not a contemporary position. In fact, since 1990 − when the FDA required the first pulmonary hypertension randomized clinical trial to use a primary endpoint that measured participant symptoms, fitness, or survival − an evaluation of fitness via the 6-Minute Walk Test has served as the primary endpoint for nearly every clinical trial, evaluating pulmonary vasodilators in pulmonary hypertension [11]. The FDA continues to fortify its guidance that patient experience must be primarily considered in drug discovery [12]. The 21st Century Cures Act, signed into law on December 13, 2016, and the Prescription Drug User Fee Act Reauthorization explicitly describe efforts to elevate the patient's voice in drug development [13, 14].

While the FDA's position on this matter has been unchanged, the availability of data that more fully elucidate the way a patient feels, functions, and survives outside of a clinical setting has only become available recently. As described before, traditional efforts have required patient evaluation in a clinical setting, outside a patient's free-living environment. These data however offer only a limited perspective about a patient's disease burden and may not properly characterize the effect of a disease on a patient's daily life [15]. A bona fide examination of disease burden is significantly improved with context of a patient's typical day. By continuously collecting data, sensors and other widely available technology permit this sort of analysis; thus, digital endpoints can offer previously unavailable insights into the disease.

For example, the heterogeneity of multiple sclerosis's disease course has complicated traditional clinical assessment of the disease state [16]. For decades, the primary clinical endpoints − the Expanded Disability Status Scale and relapse rate − have remained unchanged, despite their considerable deficiencies [17]. In recent explorations however, digital endpoints have used an array of sensors, from infrared cameras to commonly accessible accelerometers, for the purpose of demonstrating the burden of a patient's disability [18]. The use of accelerometer data, for instance, can allow clinicians to characterize the walking impairment of patients with multiple sclerosis more precisely than can traditional endpoints [19]. Understanding a patient's mobility impairment allows clinicians to most appropriately treat the disease burden. In the case of multiple sclerosis, this improvement is particularly meaningful as 70% of patients reported gait impairment to be the most challenging aspect of the disease [20].

There is a reason for optimism beyond multiple sclerosis. Similar efforts have been undertaken for Huntington's disease, autism spectrum disorder, Parkinson's disease, diabetes management, Duchenne muscular dystrophy, and heart failure with preserved ejection fraction [6, 21, 22, 23, 24, 25]. The Clinical Trials Transformation Initiative, a public-private partnership organized to improve the quality and efficacy of clinical trials, has presented approaches for digital endpoints to measure several of these diseases. For example, the organization suggests that physical activity measured by an accelerometer could provide an endpoint for heart failure, accelerometer data could be used to improve the objectivity and responsiveness of existing gold standard Parkinson's disease endpoints, continuous glucose monitoring could be used to predict severe hypoglycemia for patients with diabetes mellitus, and commonly accessible sensors could be used to measure physical activity levels for patients with Duchenne muscular dystrophy [26, 27, 28, 29]. The Digital Medicine Society maintains an ever-evolving library of digital endpoints employed in industry-sponsored studies [30]. An abbreviated list of recent efforts is available in Table 1.

Table 1.

Abbreviated list of digital endpoints [30]

Study phase 4	Indication Sickle cell anemia	Endpoint Change in total physical activity from baseline to weeks 10–12 and 22–24	Technology Activity monitor	Measurement Activity count
4	Sickle cell anemia	Change in mean nocturnal hemoglobin oxygen saturation percentage from baseline, week 10–12, and week 22–24	Pulse oximeter	Overnight pulse oximetry
1	Menopause, depression, and anxiety	Change in wake after sleep onset from beginning of study to weeks 4, 8, 12, and 16	Activity monitor	Wake time after sleep onset
3	Parkinson's disease	Change in PD symptoms as assessed by the Parkinson's KinetiGraph/personal KinetiGraph wearable device from baseline to week 12	Activity monitor	Tremor, bradykinesia, dyskinesia, and daytime somnolence
4	Diabetes mellitus	Percent time in euglycemia by a continuous glucose monitor during the final 14 days of each treatment	Continuous glucose monitor	Glycemic variability
2	Diabetic foot ulcer	Change in physiologic stress response	Heart rate monitor	Heart rate variability
2	Alzheimer's disease	Change in speech features from baseline to week 48	Microphone (audio recordings)	Acoustic and linguistic language features
2	COPD	Average change in pre-bronchodilator FEV1 from week 16 to week 24	Home spirometer	FEV1
3	Chronic cough	Percentage of participants with a ≥30% reduction from baseline in 24-h coughs per hour at week 24	Chest contact sensor with audio recording	Cough count
4	Asthma	Percentage of rescue-free days between month 4 and month 6 as determined by the rescue medication sensor	Ingestible sensor	Medication adherence

These examples were selected to demonstrate the broad set of contexts in which digital endpoints are employed and for their relatively recent initiation dates (between December 2017 and August 2020) [30].

Regulators across the world recognize the benefits of employing nonclinical data to better design and conduct clinical trials and have thus established guidance for real-world data. As defined by the FDA, real-world data are “the data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources [31].” While real-world data comprise a broad spectrum of data including electronic health records and billing activities, the FDA also defines data gathered from sources that can inform on health status, such as by mobile devices, as real-world data. The introduction of these guidelines as law in 2016 and their subsequent expansion in 2018 is an indication of the FDA's willingness to facilitate the introduction of innovative approaches to disease measurement [32]. European regulators have similarly identified the utility of real-world data and have thus formulated a consortium to enable the development of novel solutions that leverage this class of data [33].

The utilization of machine learning models to diagnose and measure disease is a particularly compelling use for the real-world data produced by digital tools. Data generated by mobile devices, wearables, and other biosensors have been growing rapidly. These data allow machine learning models to answer questions which were previously believed to be infeasible to address. Initial efforts are already underway. For example, an algorithm was designed to identify undiagnosed prodromal Alz­heimer's disease [34]. Separately, a machine learning approach was applied to smartphone data to quantify Parkinson's disease motor symptom severity [6]. Both of these studies produced promising results relative to their respective standard measures. The utilization of these sophisticated techniques in the analysis immense library of data generated by sensors will likely be a driver of innovation offered by digital endpoints.

Impediments to Digital Endpoint Adoption

The enthusiasm for digital endpoints has swelled as the continuously collected and objective data these endpoints apply to their assessment of disease state offer previously inaccessible accuracy and precision, expose previously unobtainable insights into disease burden, and can markedly reduce the cost of drug development. It is therefore natural to wonder why digital endpoints are not more frequently applied. There are several reasons.

Lack of Standardization and Collaboration

A review of published government initiatives indicates that there is a global consensus regarding the opportunity to apply digital tools to healthcare. In the USA, the FDA has established a Digital Health Innovation Action Plan to imagine the application of digital technologies at all levels of health care [35]. Health Canada has established a division within their Therapeutic Products Directorate's Medical Devices Bureau with the intention of improving their digital health review analysis [36]. An inter-council European Union group offered a statement in strong support of digital tools for building a healthier society [37]. The UK Medicines and Healthcare Products Regulatory Agency is actively seeking to engage with those working on digital health technologies [38]. Swissmedic is broadly applying digital tools to increase efficiency and effectiveness [39]. Japan's Health Care 2035 initiative emphasizes the use of digital tools [40].

Despite a unified vision, little international cooperation exists. Perhaps, it is ambitious to suggest nations should be more collaborative in their efforts; however, there is precedent. For example, the International Medical Device Regulators Forum is a consortium of medical device regulators from around the globe that attempt to coordinate regulatory requirements for medical products [41]. Continued international participation will be important to acceleration of digital endpoint acceptance; in fact, the Innovative Medicines Initiative, a public-private partnership between the European Union and the European pharmaceutical industry, has identified international collaboration as a critical variable to success to the adoption of digital endpoints [42].

In addition to collaboration across regulatory agencies, interdisciplinary participation is necessary for the successful development of digital endpoints. While all endpoints demand cross-functional cooperation, it is especially true for digital endpoints because these novel tools require software, hardware, and clinical validation [43].

On assessing the sensors and other hardware upon which digital endpoints rely for data generation, the FDA often considers every component of the device. If, for example, the accelerometer changes from one generation of a phone to the next, the device may have to be re-evaluated before regulators will allow for its data to be collected [44].

Software in the form of machine learning algorithms, mobile applications, and other types of device software are subject to the same scrutiny as physical devices; however, the assessment of software is often more complex [45, 46]. For example, while changes to hardware are characterized by long lead times, software changes rapidly and may be updated between time of submission and time of regulatory assessment [47]. Of course, even if regulators deem a digital endpoint's associated hardware and software to be fit for purpose, the endpoint must meet the same high standards for clinical validation as any traditional endpoint.

Despite these challenges, compelling progress continues to be made across disciplines. For example, efforts in computer science have used machine learning algorithms to measure levels of exertion in patients with chronic obstructive pulmonary disease using the data from a single ear-worn sensor [48]. Unfortunately, the application of machine learning models such as in the previous example occurs far too infrequently relative to the number of available algorithms. This is because the work of computer scientists is often isolated from the efforts of clinicians and medical researchers. Thus, progress can be obstructed by a lack of the domain knowledge requisite for applying machine learning models in a clinical setting [49]. Efforts to facilitate collaboration have been initiated such as by the Digital Medicine Society and even the FDA itself [50]. However, moving healthcare forward to a meaningful degree requires further unification of disjointed efforts.

Unproven Value

Even if collaborative efforts are established to develop digital endpoints that have a well-defined use case and gain regulatory approval, digital endpoints, as a category, must also allay any risk-averse participants such as pharmaceutical companies. While digital endpoints offer a promise of cheaper and more effective clinical trials because these endpoints have been applied so sparsely, this potential remains hypothetical [51, 52]. Given the cost of a clinical trial failure is immense, pharmaceutical executives exercise caution when presented an opportunity to significantly reshape their drug discovery processes [53]. Researchers have demonstrated that only 40% of phase 3 clinical trials are successful [54]. This is particularly concerning for drug makers as a significant investment is required to reach this phase. It has been hypothesized that the reason for the increasing number of late-stage failures may be due to a limitation in how traditional endpoints can represent changes in the disease state [55, 56]. If digital endpoints are better able to measure a change in a patient's condition, the likelihood of a late-stage failure could become far less likely. However, without a drug discovery paradigm shift, digital endpoints will only further complicate the pharmaceutical model, thereby contributing costs and complexity [57].

There is however some reason for optimism. In a 2018 Health Research Institute survey, 42% of pharmaceutical executives stated that they were aggressively investing in digital therapies. Of the 58% of respondents who had yet to invest in digital products, 42% suggested they would do so in the next 1–2 years, and the remaining 58% planned to work on this type of therapy within 3–6 years [58].

Physicians are also assessing the value of digital tools in a clinical context. In 2019, the American Medical Association surveyed 1,359 physicians regarding their interest and use of digital health tools [59]. Relative to the results from the same survey administered in 2016, a rising percentage of physicians reported using digital health tools to enable better care. However, physicians' primary questions regarding the use of digital tools did not change in the 3 years between the surveys − clinicians still require more evidence of the value of digital tools and want to understand their limitations as it relates to caring for their patients.

Fortunately, public-private partnerships such as the Clinical Trials Transformation Initiative, the Reagan-Udall Foundation for the FDA, and the Critical Path for Parkinson's have been established to drive the use of digital tools in a variety of clinical contexts. These multi-stakeholder initiatives are important pioneering efforts in demonstrating the value of digital endpoints with distinct clinical applications.

Nature of Health Care Data

Clinical trial data are difficult to manage, interpret, and reuse. These complexities have prompted the development of a set of principles known as the Findable, Accessible, Interoperable, and Reusable (FAIR) guiding principles for scientific data management and stewardship. They were published as a protocol for improving the experience of working with health data [60]. If these principles were to be applied to the vast quantities of data used by digital endpoints, the data could be further leveraged by researchers attempting to understand a spectrum of diseases [61]. For example, accelerometer data are commonly used to assess a patient's mobility or activity patterns. These are vital metrics for several unrelated diseases. If these data were available based on the FAIR protocol, it would significantly diminish a research effort's logistical challenges and would improve access to critical data.

The FAIR guidelines have received authoritative advocates including the European Commission which issued an action plan for “turning FAIR data into reality” [62]. The G7 has also offered its endorsement [63]. Despite this enthusiasm, the implications of adhering to FAIR principles in the context of digital tools in pharmaceutical therapy discovery must be considered carefully. For example, the adherence to these guidelines will require a shift in perspective from both pharmaceutical executives and researchers − executives will need to be convinced of FAIR guiding principles' economic benefit, while researchers will need to unburden themselves of the often tightly held belief that data generated in their laboratory should be accessible only to them [64]. Moreover, there are technical challenges that must be overcome to unlock the benefits of FAIR data. Generally, the acceptance of FAIR principles will require work, much of it significant, and any point of friction will only serve to delay the application of digital endpoints.

Finally, and most unfortunately, electronic medical record data have been fragmented by the proprietary nature of the EHR business model. New tools will need to be developed to access claims data which can provide distinctly valuable information. However, no set of principles has been adopted with the same support or enthusiasm as FAIR received for clinical trial data [65]. Given the disparate nature of EHR systems, a set of principles to govern the storage and transmission of these data seems unlikely in the near-term.

Conclusion

The collective confidence requisite for the broad adoption of digital endpoints continues to build. Researchers in and out of health care and physicians are applying their expertise to the exploration of these novel tools. There is tremendous optimism that digital endpoints will provide a platform to reduce the cost of therapeutic discovery while offering a more accurate and authentic portrayal of disease burden.

However, before digital endpoints can build meaningful acceptance and utility, genuine collaborative efforts are required. International regulators must continue to harmonize their guidance for digital tools and, more importantly, researchers and practitioners must offer their respective expertise so that the technical and regulatory ecosystems in which these endpoints will be employed can be efficiently and expeditiously developed.

To achieve the widespread adoption of digital tools, the steps required to successfully deploy digital measures across clinical research, clinical care, and public health must be established. The Digital Medicine Society has recently published a guide for digital clinical measure development [66]. The authors are actively seeking feedback about their recommendations. While these sorts of initiatives are principal to the broad acceptance of digital measures, they are most likely to succeed with interdisciplinary collaboration; thus, it is important that experts across domains contribute to the development of these fundamental guides.

Additionally, the public-private partnerships that direct the implementation of digital endpoints for various diseases must continue to be supported. Each of these initiatives offers opportunities for interested parties to support their mission. These partnerships also often publish resources that researchers and clinicians can use to facilitate the use of digital endpoints. Not only are these efforts important for driving our understanding of disease but they also demonstrate the clinical utility that clinicians are eager to more fully understand before more adopting digital tools in their practice.

The successful deployment of digital endpoints will also continue to build regulators' confidence in these tools. Regulatory agencies across the globe have expressed their support for the promise of digital tools but are still discerning when evaluating these endpoints. Thus, researchers must leverage the programs regulators have established to facilitate collaborative efforts in assessing the benefits and limitations of digital endpoints.

Conflict of Interest Statement

S. Saria is a founder of and holds equity in Bayesian Health. The results of the study discussed in this article could affect the value of Bayesian Health. This arrangement has been reviewed and approved by Johns Hopkins University in accordance with its conflict of interest policies. S.S. is a member of the Scientific Advisory Board for PatientPing.

Funding Sources

The authors received no specific funding for this work.

Author Contributions

All the authors contributed equally to this article.