Digital health technology derived measures: Biomarkers or clinical outcome assessments?

Abstract

Digital health technologies (DHTs) present unique opportunities for clinical evidence generation but pose certain challenges. These challenges stem, in part, from existing definitions of drug development tools, which were not created with DHT‐derived measures in mind. DHT‐derived measures can be leveraged as either clinical outcome assessments (COAs) or as biomarkers since they share properties with both categories of drug development tools. Examples from the literature indicate a variety of applications for DHT‐derived data, including capturing disease physiology, symptom tracking, or response to therapies. The distinction between the categorization of DHT‐derived measures as COAs or as biomarkers can be very fine, with terminology variability among regulatory authorities. This has significant implications for integration of DHT‐derived measures in clinical trials, leading to confusion regarding the evidence required to support these tools' use in drug development. There is a need to amend definitions and create clear evidentiary requirements to support broad adoption of these new and innovative tools. The biopharma industry, the technology sector, consulting businesses, academic researchers, and regulators need a dialogue via multi‐stakeholder collaborations to clarify questions around DHT‐derived measures, to unify definitions, and to create the foundations for evidentiary package requirements, providing a path forward to predictable results.

INTRODUCTION

Use of digital health technologies (DHTs) in drug development began more than a decade ago. Here, we use the United States Food and Drug Administration (FDA) definition of DHT as “A system that uses computing platforms, connectivity, software, and sensors for healthcare and related uses”. ¹ Today, the benefits of DHT use in drug development are clear to the scientific community, regulators, the pharmaceutical industry, and patients: the collection of patient‐generated health data in the real world, as compared with episodic clinical visits. DHT‐derived data can facilitate the collection of patients' insights 24/7, enabling physiological and behavioral phenotyping. A more interesting focus for debate is why the use of DHTs in drug development remains substantially conceptual without broader adoption. Specifically, we focus on sensor‐based DHTs and corresponding data‐processing algorithms. Despite increased interest, only a small number of drugs have been approved where DHT data have been used to support labeling claims. ² To the best of our knowledge, limited information is available about DHT‐driven Go/No Go decisions for drug candidates. ³ , ⁴

When considering using data derived from a DHT in a drug development program, study sponsors need to determine what data would be required to support a specific hypothesis. Part of this decision‐making process includes the determination of what needs to be measured and how. This process entails defining the DHT‐derived measures by starting with meaningful aspects of health, identifying concepts of interest, then selecting a DHT with associated measures and corresponding endpoints according to the ‘fit‐for‐purpose’ principle to ensure that the level of evidence required is sufficient to support its proposed use as a clinical endpoint in the planned context of use (e.g., internal decision‐making or a label‐enabling endpoint). ⁵ One of the factors that stands out among a myriad of reasons why we have limited examples in the public domain where data derived from DHTs have been used to support drug development is the lack of clarity on definitions and related regulatory pathways to support the validation of DHT‐derived endpoints for regulatory decision‐making. While recent guidance from the FDA for DHT for remote data acquisition in clinical investigations ⁶ and the European Medicines Agency (EMA) Q&A on qualification of digital technology‐based technologies ⁷ are very useful as these guidance documents have clarified many questions around DHT‐derived measures and laid out fundamental principles for incorporating these tools into clinical trials, there are still areas that require further clarification. Part of the problem is a limited number of applications submitted by sponsors that would provide sufficient data to develop further the existing guidance documents and frameworks. Here, we analyze some of the areas that require further work and propose a potential path forward which, we believe, would enable better adoption of DHT‐derived measures.

In the drug development process, there are historically two main classes of assessments used to provide safety and efficacy data to support the approval of a drug: biomarkers and clinical outcome assessments (COAs). The term “biomarker”, an abbreviation of “biological marker”, was coined in 1980. ⁸ It refers to a marker of a biological process that can be normal, pathological, or a response to a therapy or exposure. A biomarker constitutes a measurement derived from objective assessments done either in vitro (e.g., from biological specimens using laboratory instrumentation) or in vivo (e.g., by imaging techniques or electrophysiological methods). Widespread adoption of biomarkers as tools for drug development began with a publication in 1998 by the National Institutes of Health (NIH) Biomarker Definitions Working Group, which created a definition of biomarkers and determined biomarker categories (e.g., diagnostic, prognostic, or response to treatment) as well as a definition of biomarker‐based surrogate endpoints. In 2004, the FDA recognized the importance of improving drug development efficiency and deemed biomarkers, and associated translational strategies, a high priority. Biomarker use has become mainstream in drug development, transcending therapeutic areas and development stages. ⁸

Clinical outcome assessments, unlike biomarkers, capture data depicting the treatment's impact on a clearly identified aspect of how a patient feels, functions, or survives. ⁹ COAs are classified based on the reporter and can be collected in the form of a patient‐reported outcome (PRO), a clinician‐reported outcome (ClinRO), a performance outcome (PerfO), or an assessment done by an observer (ObsRO). The main distinction between COAs and biomarkers is that COAs capture aspects of the condition that are important to the patient, that impact how they live their lives, and that may be influenced by the individual's conscious choice, judgment, or motivation. ⁹ The concept of Patient Focused Drug Development by United States health authorities has increased the emphasis of capturing assessments that are meaningful to patients and caregivers. ¹⁰ COAs, like biomarkers, are a critical part of clinical research. Proving validity, reliability, and sensitivity to change is also key for their use in drug development.

Very clear guidance and pathways have been defined by the FDA ¹¹ , ¹² and the EMA ¹³ for qualifying and using biomarkers or COAs in the drug development process. This has worked very well and resulted in the qualification of a number of drug development tools. ¹¹ , ¹²

WHAT IS CONFUSING?

DHT‐derived measures can capture multiple concepts of interests and may include data generated by active tasks performed by patients or passive data acquisition (Table 1). In some circumstances, DHT‐derived measures constitute conventional biomarkers, such as heart rate measured by electrocardiogram, or blood oxygen saturation by pulse oximetry performed remotely: the nature of these measures align with the definition of biomarkers. ¹⁸ In other circumstances, DHT‐derived measures can constitute COAs, such as the instrumented 6‐minute walk test (6MWT). However, there are scenarios when DHT‐derived measures could be classed as either biomarkers or COAs depending on whether they measure (1) a characteristic as either an indicator of the pathogenic process of a certain disease/condition, or an indicator of a response to an intervention or exposure or (2) a patient's performance or function ² linked to a concept that is meaningful to the patient. These DHT‐derived outcomes are objectively measured; can be responsive to change, such as treatment effects or disease progression; and can be used as disease‐screening features and/or to establish response to treatment, in the same way as biomarkers can be used (Figure 1). As an example, for patients living with congestive heart failure, reduction in mobility (a predictive factor for mortality) can be measured with body‐worn actigraphy devices and could be considered a biomarker ¹⁷ supporting a surrogate endpoint. The same digital measure could also be considered a COA because it assesses the patient's physical capacity (Table 1).

TABLE 1.

Examples of digital health technologies (DHT)‐derived measures and related concepts of interest.

Therapeutic area (reference)	DHT	DHT‐derived measure details (active task or passive data acquisition)	Concepts of interest
Parkinson's disease (Lipsmeier et al. 14 )	Smartphone and smartwatch with the Roche PD Mobile Application	Active task: draw a shape, dexterity, hand turning, speech, pronation postural and resting tremor, balance, cognitive test Passive data acquisition: walking activities	Bradykinesia, tremor, rigidity, postural instability, cognition, activities of daily living
Parkinson's disease (Burq et al. 15 )	Smartwatch	Active task: seated test, arm raise, arm twist, timed up and go	Upper extremity bradykinesia, rest tremor, arm swing during gait
Duchenne muscular dystrophy (Servais et al. 16 )	Suitable wearable device worn at the ankle	Passive data acquisition: ambulation measured directly in a continuous manner in a real‐world environment	Stride velocity 95th centile as the minimal velocity of the 5% most rapid strides taken by a patient in a real‐world setting
Congestive heart failure (Tan et al. 17 )	Body‐worn actigraphy devices	Passive data acquisition: activity counts, vector magnitudes walking parameters, and metabolic equivalent of task	Actigraphy‐quantified physical activity as a prognostic factor for mortality

Abbreviation: PD, Parkinson's disease.

FIGURE 1.

Characteristics of a biomarker, a clinical outcome assessment (COA), and a digital health technology (DHT)‐derived measure.

The biggest distinction from classical PROs, ClinRO's or ObsRO's (which are usually assessed using a questionnaire or a rating scale) is that DHT‐derived measures no longer represent a human perception of how patients feel or function. When concept elicitation determines that ambulation is important to patients, and it is possible to establish a meaningful change threshold for the instrument (by using anchor‐ or distribution‐based methods, for example), the measure can be considered a COA.

There are a limited number of examples where meaningful change thresholds have been established for endpoints derived from DHTs in drug development. The best‐known include the qualification of stride velocity 95th centile (SV95C) in Duchenne muscular dystrophy (DMD) as a secondary endpoint by the EMA, in which a distribution‐based approach was used ¹⁶ ; and the recent case in which agreement was reached by the FDA to use moderate to vigorous physical activity as a primary endpoint in idiopathic pulmonary fibrosis. ¹⁹ In the latter case, an anchored‐based methodology was used to determine the meaningful change threshold.

Recent examples of qualification submissions for “DHT‐Passive Monitoring” COAs to the FDA qualification programs indicate the importance of demonstrating meaningfulness to patients and relevance to their daily life (Table 2; Examples 4 and 5). These examples include measures of ambulatory ability, mobility performance, and exercise capacity.

TABLE 2.

Examples of digital health technologies‐derived measures terminology used in regulatory qualification submissions.

Qualification application and website link	Goals	Terminology used
1. EMA qualification advice: Mobilize‐D: digital mobility outcomes as monitoring biomarkers https://www.ema.europa.eu/en/documents/other/letter‐support‐mobilise‐d‐digital‐mobility‐outcomes‐monitoring‐biomarkers‐follow_en.pdf	To establish acceptable Digital Mobility Outcomes (derived from suitable mobile wearable devices and associated algorithms) as biomarkers of mobility capacity for clinical benefit (i.e., as surrogate, primary, or key secondary endpoints) in pivotal clinical trials for treatment of diseases or health conditions that impact upon mobility	Mobility outcome as a biomarker of mobility capacity
2. EMA Qualification opinion on stride velocity 95th centile as a secondary end point in Duchenne muscular dystrophy measured by a valid and suitable wearable device https://www.ema.europa.eu/en/documents/scientific‐guideline/qualification‐opinion‐stride‐velocity‐95th‐centile‐secondary‐endpoint‐duchenne‐muscular‐dystrophy_en.pdf	Stride velocity 95th centile measured at the ankle is an acceptable secondary endpoint in pivotal or exploratory drug therapeutic studies for regulatory purposes when measured by a valid and suitable wearable device to quantify a patient's ambulation ability directly and reliably in a continuous manner in a home environment and as an indicator of maximal performance	Appropriate endpoint and qualification of novel gait measurements by means of a wearable device
3. FDA Clinical Outcome Assessments (COA) Qualification Program DDT COA #000129: Advanced Gait Analysis. https://www.fda.gov/drugs/drug‐development‐tool‐ddt‐qualification‐programs/ddt‐coa‐000129‐advanced‐gait‐analysis	Spatiotemporal parameters of gait derived from DHT data (specifically, MC10 BioStamp nPoint®) measured in the home and professional healthcare settings will be used to develop an outcome measure for early detection and assessment of progressive gait abnormality of ambulatory adults with Huntington's disease	DHT – Passive Monitoring COA
4. FDA Clinical Outcome Assessments (COA) Qualification Program DDT COA #000142: Virtual Motor Exam for Parkinson's Disease, Part III Estimator (VME Part III) https://www.fda.gov/drugs/drug‐development‐tool‐ddt‐qualification‐programs/ddt‐coa‐000142‐virtual‐motor‐exam‐parkinsons‐disease‐part‐iii‐estimator‐vme‐part‐iii	Virtual Motor Exam Part III Estimator for adults with Parkinson's disease, administered on a wrist‐worn wearable device (specifically, the Verily Study watch). It provides an estimate of the motor symptom severity using a series of up to eight motor tasks that can be performed by the patients themselves at home. It is intended for use in adults with PD across the full range of disease progression to measure severity and disease progression	DHT – Passive Monitoring COA

Abbreviations: COA, clinical outcome assessment; DHT, digital health technology; EMA, European Medicines Agency; FDA, Food and Drug Administration; PD, Parkinson's disease.

At the same time, digital measures can, by their nature, be complex, generating outcomes that can be challenging for patients to relate to (e.g., spatio‐temporal characteristics of gait such as stride velocity) (Table 1). Multi‐component biomarkers suffer from the same complexity. For example, measuring the variability of finger tapping to assess bradykinesia in Parkinson's disease, ¹⁴ and tasking a study participant to complete a structured finger‐tapping exercise on the smartphone to assess hand function, ² are intended to reflect the same concept of interest and the resultant digital measure can be used to derive the same clinical endpoint to assess changes in motor symptoms.

BIOMARKERS OR COAS: CLASSIFICATION IMPLICATIONS FOR CLINICAL TRIALS

If digital measures are categorized as DHT‐derived COAs, reflect how a patient feels or functions, but it is not feasible to establish meaningfulness to patients, ¹⁰ they may not be considered valuable. This is a concern, because we may end up abandoning sensitive digital measures that could be valuable as disease biomarkers. In drug development, there are numerous examples of imaging‐ and blood‐based biomarkers (e.g., hemoglobin A1c in diabetic control, or tumor imaging and progression‐free survival in oncology ²⁰ ) which can be analyzed to support surrogate endpoints, and can be used in registration studies because of their clinical relevance and utility, although they often carry very little meaning to patients. The same rationale can and should be applied to digital measures, given the evidence that they can accurately and reliably measure clinically relevant information (Table 2).

From the point of view of drug developers, this creates a conundrum pertaining to the requirements for an evidentiary submission package, because experiments aimed at defining concepts of interest and clinical validation studies are not the same. For example, using a biomarker to support a surrogate endpoint may require one or more studies proving surrogacy. ²¹ At the same time, validation of a COA instrument requires concept‐elicitation studies to identify a burden of disease/condition that are important to patients and establishing instrument responsiveness to change ¹⁰ that are usually not required for biomarker‐related measures. ²² These differences become particularly challenging in global studies because definitions vary among regulatory authorities. For example, the EMA qualification advice for the Mobilize‐D initiative considers digital mobility outcome measures to be biomarkers, ²³ while the FDA categorizes the same measures to be COAs ¹⁰ (Table 2).

The line between using DHT‐derived measures to assess a response to a therapy and using a DHT to measure a patient's ability to perform a certain task that has implications on their quality of life can be very fine. This fine line creates uncertainty for drug developers in terms of their regulatory strategy; which, in turn, results in their hesitancy to develop and implement novel measures that could potentially reduce sample size for clinical studies ²⁴ and bring much‐needed treatments to patients faster. This is particularly true in rare or slowly progressing diseases, where the number of patients participating in clinical trials can present a significant challenge. For instance, one of the frequently used measures to support primary or secondary end point for registration studies in DMD is the 6MWT, and the change in the distance walked in 6 minutes is a derived endpoint (Table 3). At present, there is one phase III DMD study (NCT05096221) ³² using SV95C as a secondary end point, and it is likely only a matter of time before we see this DHT‐derived measure used for drug registration, particularly if qualification as a primary endpoint is pursued, as has been recommended by the EMA. ³³

TABLE 3.

Registration endpoints for drug approvals in Duchenne muscular dystrophy, including 6 minute walk test (6MWT).

Drug name	FDA accelerated approval date	Registration end point
Casimersen	February 25, 2021	Primary endpoint: change from baseline in the total distance walked during 6MWT at Week 96 Secondary end point: change from baseline in the total distance walked during 6MWT at week 144 25 Interim efficacy was assessed based on change from baseline in the dystrophin protein level 26
Viltolarsen	August 12, 2020	Primary outcome measure: change in time to stand, baseline to 48 weeks of treatment Secondary outcome measure: 6MWT, baseline to 48 weeks of treatment change in 6MWT 27 , 28
Golodirsen	December 12, 2019	Primary endpoint: change from baseline in the total distance walked 6MWT at Week 144. Change from baseline was −99.0 m for golodirsen‐treated patients vs. −181.4 m for external controls (p = 0.067), and loss of ambulation occurred in 9% vs. 26% (p = 0.21) 29
Eteplirsen	February 9, 2017	The primary endpoint was change in dystrophin production; a clinical outcome measure, the 6MWT, was also assessed. There was no significant difference in change in 6‐minute walk distance between patients treated with eteplirsen and those treated with placebo 30
Deflazacort	September 19, 2016	In Study 1, the change in average muscle strength score between baseline and Week 12 was significantly greater for the deflazacort 0.9 mg/kg/day dose group than for the placebo group The results of the analysis of the primary endpoint of average muscle strength scores in Study 2 (graded on a 0–5 scale) at 2 years were not statistically significant, possibly because of a limited number of patients remaining in the placebo arm 31

CONCLUSIONS

The unique nature of DHT‐derived measures is being increasingly recognized because of their importance in assessing patients' physiology, symptoms, and response to therapies. The scientific community also recognizes the complexity of developing DHT‐derived tools. Both patient input and empirical data‐driven approaches have value in DHT‐derived measure discovery and development as proposed by Taylor et al., ³⁴ reflecting their unique nature (Figure 1).

It is time to recognize the unique nature of DHT‐derived measures and corresponding endpoints, which combine features of biomarkers and COA instruments at the same time. A collaborative approach is required to enable progress. The biopharma industry, the technology sector, consulting businesses, academic researchers, and regulators need a dialogue to create a regulatory framework, which can be harmonized globally, on how to categorize, and validate this new type of drug development tools and deliver on the promise of DHT‐derived measures and endpoints which would create a path forward with improved predictability of results. Each stakeholder brings a different and necessary perspective into the discussion. This integration of points of view is possible only via precompetitive collaborations.

Precompetitive public–private partnerships have repeatedly played a pivotal role in shaping the drug development space by leveraging innovation and openly sharing knowledge, data, and tools. The work of the NIH Biomarker Definitions Working Group ⁸ is one example. There are multiple initiatives aimed at understanding the utility of DHT‐derived measures to develop novel medicines in rare diseases, ¹⁶ neurological disorders, ³⁵ or across multiple indications. ²³ The emerging data can be leveraged to inform future use of DHT‐derived tools and regulatory requirements. Additionally, existing regulatory pathways, such as the Innovative Science and Technology Approaches for New Drugs (iSTAND) pilot program in the United States, can enable a path for new digital measures that do not readily fall into existing biomarker and COA categories.

Precompetitive collaborations can leverage knowledge and expertise to unify the field of DHT‐derived measures, and solve the following problems:

• Recognize unique opportunities offered by DHTs in drug development and identify the differences of DHT‐derived measures (compared with measures derived from traditional biomarkers and COAs) that will require adjusting existing definitions.

• Create definitions for DHT‐derived tools and derived measures (similar to biomarkers categories or COA types) with specific examples of how these tools can be leveraged in drug development.

• Define specific requirements for evidentiary criteria to declare DHT‐based tools to be fit for purpose according to defined categories. Existing regulatory definitions can be harmonized across geographies.

FUNDING INFORMATION

No funding was received for this work.

CONFLICT OF INTEREST STATEMENT

E.S.I. is an employee of Koneksa Health and may own company stock. C.D. is an employee of Pfizer Inc. and may own company stock. M.M. is an employee of Novartis Ireland Ltd. and may own company stock.

Izmailova ES, Demanuele C, McCarthy M. Digital health technology derived measures: Biomarkers or clinical outcome assessments? Clin Transl Sci. 2023;16:1113‐1120. doi: 10.1111/cts.13529