Abstract
Current care and research pathways for amyotrophic lateral sclerosis (ALS) primarily rely on regularly scheduled visits to specialized centers. These visits provide intermittent clinical information to health care professionals and require patients to travel to the clinic. Digital health technologies enable continuous data collection directly from the patient's home, bringing new opportunities for personalized, timely care and a refined assessment of disease severity in clinical trials. In this review, we summarize the state of the art in digital health technologies for remote monitoring of patients with ALS, ranging from televisits through videoconferencing to sensor-based wearable devices. We explore how these technologies can benefit clinical care and advance treatment development. Despite significant progress, real-world adoption of these technologies remains limited. An overview is provided of the key barriers hindering their widespread implementation and the opportunities to advance the field. Significantly, there is an urgent need for harmonization across stakeholders through consensus guidelines and consortia. These efforts are essential to accelerate progress and harness the full potential of digital health technologies to better meet the needs of patients.
Introduction
Amyotrophic lateral sclerosis (ALS) is characterized by its highly heterogeneous presentation of phenotypes and progression rates.1 Monitoring the health status of patients with ALS has been challenging and mostly relies on episodic in-clinic visits, often conducted at specialized centers. These visits, however, offer a brief “snapshot” of the patient's health status, providing only limited information to the health care provider. As a result, the provision of counseling, or the timing of health care interventions, may be suboptimal. Periodic snapshots also make it difficult to accurately capture treatment responses, whether in care or research settings, and require patients to travel to the clinic. Consequently, as the disease progresses, accessibility to health care may be jeopardized, particularly when care is most needed.
Digital health technology (DHT) may overcome these challenges and improve care delivery. DHT is “a system that uses computing platforms, connectivity, software, and/or sensors, for health care and related uses,”2 encompassing a wide range of applications, including televisits through videoconferencing, self-administered online surveys, sensor-based wearables, and telehealth platforms. Unlike traditional in-clinic visits, these technologies may provide a more continuous, detailed, and real-life picture of the patient's health status, directly from the patient's home.3 DHTs bring new opportunities for clinical trials to enhance the detection of clinical treatment effects but also enable personalization of care through a decentralized, patient-focused model, potentially reducing burden for patients and their caregivers.
Navigating DHTs from inception to adoption can, however, be a complex and demanding journey, the evidence to support their use in practice being still in its infancy.4 Consequently, many of the potential benefits of DHTs have yet to be realized, and their actual implementation in practice remains limited. How far are we from an era where DHTs shape our care and research pathways to benefit patients, and what are the next steps to achieving this goal? In this review, we summarize the current state of the art of DHTs to monitor patients with ALS remotely, reflect on existing knowledge gaps and challenges, and highlight possible opportunities to advance the field in harnessing its full potential.
Methods
A systematic review was conducted to identify studies that implemented DHTs in ALS. In August 2024, we searched the PubMed database using search terms including “amyotrophic lateral sclerosis” or “motor neuron disease” combined with various terms to describe DHT. Studies were eligible if they met the following criteria: (1) enrolled at least 10 patients with ALS and (2) investigated a DHT to remotely monitor patients. Studies that solely evaluated a DHT in an in-clinic setting were excluded. We refer to eAppendix 1 (eTable 1 and eFigure 1) for a detailed description of our search strategy and screening procedure.
Results
In total, we screened 1,675 citations, of which 77 studies are identified and summarized in Figure 1; the individual study characteristics are presented in the supplemental Excel file (eTable 2). DHTs were categorized into 3 categories: televisits, self-administered online surveys, and objective digital tools.
Figure 1. Overview of Identified Digital Health Care Technologies, Tools, and Outcomes.
The number of studies that investigated digital health technologies to remotely monitor patients with ALS, categorized into televisiting, self-administered online surveys, objective tools, and telehealth platforms (A). An overview of the tools used for objective measuring (B) and their outcomes (C). For illustrative purposes, no more than 30 outcomes per category were depicted. ALS = amyotrophic lateral sclerosis; DTW = dynamic time warping; EIM = electrical impedance myography. Created in BioRender. van Unnik J (2025) BioRender.com/b57h475.
Televisits
Although televisits have long been part of routine care, the pandemic led to a tremendous global uptake, resulting in a plethora of reports on how clinical visits could be decentralized. Televisits allow the health care provider to collect and act on synchronous information through a “live” connection to assess the patient's health status. Although the phone could be used to assess the patient's health status, for example, by phone-administered ALS Functional Rating Scale–Revised (ALSFRS-R),5 here we solely consider videoconferencing. The exact method of videoconferencing varied across studies, consisting of either one-to-one meetings or group calls with various members of the multidisciplinary team. Televisits were generally feasible for discussing practical care aspects.6 Patients and their caregivers were predominantly satisfied, noting reduced travel burden and convenience as the primary benefits.7 Health care providers had a more mixed experience, however, reporting limitations in physical contact and a feeling of reduced care quality compared with in-clinic visits.7 Furthermore, reluctance to discuss psychosocial and emotional aspects has been described, highlighting that some care aspects can be best communicated in person. Technical problems, such as issues with video and audio, were also frequently reported by both patients and health care providers.
Only 1 retrospective study was identified that investigated the safety and quality of televisits8; it demonstrated that video and in-clinic visits compared well as far as provision of medication, assistive devices, and hospice services were concerned, a finding which requires confirmation by future studies. While acute care aspects were rarely discussed during televisits, others have highlighted the opportunity to provide care “on demand.” In addition, although patients generally had a positive attitude toward televisits, preferences for in-clinic visits were also reported.9 As such, we do still need to develop a better understanding of how televisits may best be positioned in and integrated with current care pathways.
Self-Administered Online Surveys
Patient-reported outcome measures (PROMs) have emerged as important remote measurement tools, offering a low-burden method to monitor symptoms, daily function, and quality of life, without the need for evaluator training. The self-administered versions of the ALSFRS-R were most frequently investigated and used either a smartphone application or web portal. The surveys were developed in 4 different languages and compared against an evaluator-administered ALSFRS-R, either in-person, by telephone, or by video call. While excellent inter-rater and intra-rater reliability were consistently reported, several studies demonstrated a small positive bias with patients reporting higher scores compared with trained evaluators.10,11 One study noted a difference when patients completed the assessment together with their caregivers compared with completing the survey alone.12 A monthly or bimonthly measurement frequency was feasible and neither physically or emotionally burdensome nor time-consuming.13 However, there are inconsistencies in administering the ALSFRS-R because of independent language translations and differences between countries in standard operating procedures, which have also affected the self-administered versions. This has recently led to a call for global harmonization.14 In addition, while much data and experience of self-administrating the ALSFRS-R are already present, equivalence to the in-clinic version has not yet been demonstrated. Efforts to provide this evidence are underway and aim to support the remote use of the ALSFRS-R in clinical trials.15
In addition, the self-reported Patient-Ranked Order of Function (PROOF) was developed to reflect that patients might perceive some ALSFRS-R domains to be more important than others. PROOF weighs the domains according to a self-reported order of importance, demonstrating excellent test-retest reliability and potential refinement of the ALSFRS-R total score.16 In addition, several Rasch-built PROMs were identified. One study investigated the Rasch-Built Overall ALS Disability Scale (ROADS), which attempts to summarize ALS disease severity into a single construct (one dimensionality).17 Others studied the ALS Impairment Multidomain Scale (AIMS), which assesses each domain separately to account implicitly for multidimensionality in ALS-related symptoms.18 When compared with the ALSFRS-R, both ROADS and AIMS subscales showed increased sensitivity to detecting changes over time.17,18 In addition, several surveys, administered through videoconferencing, were identified, including a study showing the feasibility of the Edinburgh Cognitive and Behavioral ALS Screen to assess cognitive function.19
Objective Digital Tools
Bulbar Function
Remote monitoring of bulbar symptoms typically involves the measurement of speech by eliciting specific tasks using a smartphone application or a web-based platform. Initially, these technologies only acquired voice samples, but more recent applications also allow for video recordings of the facial musculature, thereby enabling a multimodal assessment including acoustic, linguistic, and orofacial characteristics.20
Elicitation methods were largely similar between studies, although the total number of tasks varied substantially. Frequent tasks included reading passage (short story), diadochokinesis task (quick repetition of sequel of syllables), reading speech (structured sentences), and picture description. Outcomes derived from voice samples typically outperformed those based on video recordings in distinguishing abnormal from normal ALSFRS-R speech scores.20 Furthermore, voice-based outcomes measuring duration or timing of speech generally resulted in better discriminatory performance compared with variations in frequency and amplitude of voice signals, such as jitter and shimmer. Outcomes measured when reading a text aloud, such as speaking rate, speaking duration, and the percentage predicted timing of words (canonical timing agreement), were highly discriminatory for bulbar involvement,20-22 with high test-retest reliability.22 In addition, these outcomes were sensitive to detecting changes over time, where bulbar patients worsened more quickly compared with nonbulbar patients.20,21 The word count when describing a picture showed similar discriminatory and longitudinal responsiveness20 but was considerably less reliable.22 Notably, although diadochokinesis task–derived measures, such as the cycle-to-cycle temporal variation and the number of syllables, proved valuable in distinguishing bulbar from nonbulbar patients, they generally proved to be insensitive to observing changes over time.20 One study investigated the performance of multiple outcomes combined into a single composite score, which provided minimal gains as opposed to using the best performing univariable outcome when differentiating between (ab)normal ALSFRS-R speech scores.22
Significantly, no study thoroughly investigated patient acceptability or burden of remote speech function assessment. Considerable attrition rates were reported occasionally, questioning the feasibility of long-term monitoring, particularly in later stages of the disease. While many studies reported slope differences between patients with and without bulbar involvement as measured by novel outcomes, longitudinal relationships of these outcomes with current clinical standards, including functional, disability, and survival outcomes, were rarely investigated, highlighting the need for additional validation steps. In addition, the influence of cognitive decline on measured speech outcomes remains to be elucidated as it could affect outcomes derived from both structured and unstructured elicitation methods.
Motor Function
Muscle Strength
Although muscle weakness is the hallmark of ALS and has been widely used as an outcome in clinical trials, remote monitoring of muscle strength has received relatively little attention. Studies have used portable devices with integrated dynamometers to monitor hand grip23 or knee extension strength,24 enabling patients to perform the measurements independently. A pilot study showed that measuring knee extensors remotely was well accepted and provided reliable and sensitive estimates of quadriceps strength.24 This portable fixed device offers the advantage of a standardized assessment, removing the influence of the rater and their technique. A disadvantage, however, is the fact that the device is currently not readily available, although efforts within the TRICALS consortium are underway to increase its accessibility.
Muscle Integrity
Electrical impedance myography (EIM) offers a potential alternative to traditional muscle strength assessment, which may be suitable for remote monitoring because of the portability of the device and relatively low measurement complexity.23 EIM aims to identify changes in muscle integrity, by measuring voltages across a muscle region after administering an electric current to the skin. An advantage of EIM-derived metrics is that they cannot be affected by patient motivation or fatigue, preventing underestimation of muscle function that may occur with exercise-based or strength-based measurements. A downside is that these metrics are more difficult to interpret, lacking data on its use in the remote context.
Mobility
Remote monitoring of physical mobility has commonly been conducted using commercially available or research-graded wearables equipped with an accelerometer. These body-worn devices may collect data passively during everyday life or actively during tasks evoking specific movements.3 The benefits of passive monitoring include its lower dependency on the patient's capability to perform specific exercises and, potentially, more accurately reflect real-world functioning.25 In addition, studies in Parkinson disease have reported a potentially higher compliance when using passive monitoring.26 A downside is, however, that the data obtained from passive monitoring are generally noisy, posing additional statistical challenges in transforming the data into meaningful end points.
Nearly all studies derived outcomes from passively collected data. These outcomes reflected various constructs, ranging from what a patient can do (capacity) to what the patient actually does in day-to-day life (performance).25 Capacity-based outcomes were often quantified using the distribution tails, such as the 95th percentile of the number of steps per minute. By contrast, performance-based outcomes were commonly summarized using metrics of center or volume, such as the mean number of steps per minute or the total number of steps. In addition, rather than using metrics of intensity or volume, some studies have argued that metrics of variability be used to emphasize the range of actions performed throughout the day,27 potentially reducing the impact of the behavioral context, such as patient lifestyle and motivation-related factors.
An overview of the outcomes and their relations with ALS functional measures is illustrated in Figure 2. Clearly, there is currently no consensus on what construct or metric best captures disease progression as reflected by the wide range of outcomes. In addition, many outcomes relied on algorithms with limited technical validity and most were studied in small and relatively slow progressing cohorts for limited follow-up periods. Overall, most outcomes were able to detect longitudinal decline and correlated significantly with the ALSFRS-R total score. Their relationship with other accepted measures is less known, however, and the association with survival was investigated for only 1 outcome; no studies investigated the association between mobility-based outcomes and quality of life.
Figure 2. Mobility Outcomes and Their Association With Key ALS Outcomes.
Mobility outcomes were grouped into main categories (e.g., “calories” and “metabolic equivalent of task” were grouped under “energy expenditure”) and stratified by either fine or gross motor function (dark green headings) to demonstrate their associations with key ALS outcomes (gray headings). Although a wide variety of mobility outcomes, devices, and wear locations were investigated, most studies solely explored the relationship with the ALSFRS-R using limited sample sizes. The relationship with other key ALS outcomes remains to be quantified, illustrating that additional validation beyond proof-of-concept is warranted before widescale implementation is possible. ALS = amyotrophic lateral sclerosis; ALSFRS-R = ALS Functional Rating Scale–Revised; A = ankle; C = chest; H = hip; S = smartphone; W = wrist.
Respiratory Function
Remote monitoring and adjustment of ventilator oxygen flow is one of the earliest applications of DHTs in ALS.28 More recently, several parameters of respiratory function have been developed. For example, oxygen saturation can now simply be monitored through home-based pulse oximetry (SpO2). Studies have demonstrated the feasibility of instructing patients to seek contact whenever home readings surpassed a certain threshold, which could be communicated by phone or directly uploaded through modern oximetry devices or smartphone applications. Additional respiratory information could be obtained through home spirometers. In 4 studies, spirometers were used for the remote assessment of vital capacity (VC),23,29-31 with a single study also investigating maximal inspiratory pressure (MIP)31 and another study performing peak cough flow measurements.29 Instructions were provided by a single home visit, by online training sessions, or continuously through video consultation. Unsupervised VC yielded values comparable with those obtained during supervised assessments,30 or in-clinic visits,31 while unsupervised MIP resulted in lower values compared with in-clinic scores.31 One study explored the possibility of predicting VC based on remote speech measurements of sustained phonation.32 The study demonstrated moderate cross-sectional accuracy and a lower sensitivity to detecting change when comparing the predicted VC values with those measured in the clinic.
No studies have investigated the feasibility and validity of these assessments in patients with advanced bulbar involvement, which remains a significant unresolved challenge. In addition, while some studies have demonstrated that these technologies may lead to a more rapid prescription of noninvasive ventilation31 and decrease the number of unscheduled visits,28 larger prospective studies have yet to be conducted to investigate the impact of remote respiratory assessments on quality of care.
Cardiovascular Function
Autonomic dysfunction has been observed in patients with ALS, potentially leading to changes in cardiac function. Heart rate and blood pressure have been found to be predictive for future ALSFRS-R scores.33 Alongside traditional motor symptoms, measures of cardiovascular function might, therefore, provide additional prognostic insights. One study monitored heart rate during self-induced exercises aimed at preventing fatigue.34 Others investigated unsupervised monitoring of heart rate using a chest-worn sensor but reported insufficient data quality.35 The potential of remotely collected cardiovascular data to effectively characterize ALS disease progression remains, therefore, largely unexplored.
Implementation in Real-World Practice
Clinical Care
In summary of the above, patients and health care professionals may thus benefit substantially from DHTs, which could positively affect the delivery of care. The reality is, however, that the evidence of most technologies has not yet matured sufficiently and the care provision of today largely continues with traditional periodic visits.
Beyond clear evidence of clinical benefit of DHT, successful integration of these technologies in existing pathways is critical for their adoption.36 How the different digital modalities can be brought together to promote their use in clinical practice is perhaps best exemplified by telehealth platforms, which are increasingly used to supplement standard of care. While most platforms gather patient-reported and caregiver-reported data on a smartphone app to coordinate care, others have also included chat functionality to facilitate direct communication between patients and health care providers.13,37,38 These digital platforms are generally well accepted; may improve early problem identification, such as the need for noninvasive ventilation (Figure 3); and allow for proactive care, for example, by initiating televisits in case of sudden progression. Moreover, these technologies could improve the accessibility to care, particularly when traveling to the clinic is challenging, where the choice may be between digital visits or no visits at all.
Figure 3. Real-Life Example of a Remotely Monitored Patient With ALS.
Leah, a 62-year-old woman, received the ALS diagnosis 2 years ago. Since first symptom onset, she has been coping with the relentless progression of the disease. A few months ago, she experienced some shortness of breath while performing activities, which is perhaps the reason why she joined a pilot program to monitor her respiratory function at home that supplemented usual care. Since then, Leah has been self-assessing her vital capacity using home-based spirometry and monitoring for orthopnea using a self-administered survey. Data were communicated to the ALS center through a telehealth platform that is accessible on her smartphone. After 6 months, her vital capacity declined and Leah messaged through the telehealth platform that she started to have trouble breathing during the night. She was promptly referred to a pulmonologist, and NIV was initiated shortly thereafter. ALS = amyotrophic lateral sclerosis; NIV = noninvasive ventilation.
Although pilot programs have affirmed the wielding capabilities of telehealth platforms to potentially benefit current ALS care, these studies have also exposed several challenges to translating their promise into practice. Fostering interoperability to allow seamless dataflows between DHTs, electronic health care systems, and patient portals is a vital task39 but complicated because many manufactures have built their own data systems outside the existing health care infrastructure. Presenting these data to be easily interpretable by both patients and health care providers is an additional unresolved challenge, especially when collected in large volumes by tools using passive monitoring. Although visual tools and dashboards might be useful, these methods often rely on reference values that are currently not yet available for many of the outcomes discussed in this review. For outcomes that are already used in routine care, such as the ALSFRS-R and VC, warning thresholds should be carefully tested to reduce the number of (false) alerts. Moreover, it will be critical to understand the psychological aspects of frequent monitoring because these affect disease awareness and may promote self-management but could also induce anxiety and be confronting when faced with continued progression.
While DHTs have been linked to potential cost reductions,40 their impact on clinical outcomes must be quantified to fully appreciate the cost-benefit tradeoff. Whether these technologies truly improve the delivery of care remains unproven for many neurologic diseases41; this is one of the main barriers to their adoption in routine ALS care.37 Sufficient evidence on long-term cost-effectiveness is typically required for reimbursement by health insurance companies, yet conducting cost-effectiveness studies can present substantial challenges.42 Aside from the associated costs, traditional randomized clinical trials may be challenging in evaluating unblinded, rapidly evolving interventions that are typical for digital technologies such as telehealth platforms. Alternative approaches that capitalize on collecting real-world data may be more suitable, although best practices for the design of such studies have not yet been fully crystallized.43 Furthermore, additional challenges may arise when transferring technologies across regions because of cross-border differences in assessment frameworks, albeit initial harmonization efforts have recently commenced.44
As evidence of DHTs continues to mature, the uptake of these technologies is likely to spread and diversify beyond the occasional use of televisits. Telehealth platforms combined with DHT-derived outcomes offer health care providers insights into the health status of patients that extend far beyond the current model, facilitating more personalized, timely care. Moreover, the growing amount of data collected from future observational studies will bring new opportunities to further enhance clinical decision making, for example, by developing models using accelerometry data that predict the time to wheelchair dependency or loss of ambulation.
Clinical Drug Development
Despite the enormous interest for DHTs in clinical drug development, their implementation remains limited to only a few clinical trials (e.g., NCT03168711). This is not unique to ALS because DHT-derived outcomes have only been incorporated in 11% of the clinical trials for other neurologic diseases.45 This mismatch is unsurprising considering that many studies have solely focused on demonstrating proof of concept, associating outcomes derived from DHTs with the ALSFRS-R. While these exploratory studies offer valuable insights, they alone do not provide definite evidence to support the use or justify the cost of these technologies in clinical trials.
Instead, a more comprehensive approach is required.2,46 Aside from ensuring the validity and reliability of DHT-derived outcomes, data should be collected to underpin rationale for questions such as “how big a change do we have to observe in order for it to be relevant and noticeable for patients.” Comparisons with measures beyond the ALSFRS-R, such as global measures of change and quality of life, are, therefore, essential to aid the identification of meaningful changes and motivate their use for clinical research. Implementing novel outcomes in clinical trials is essential for their validation, a priority that drug regulatory agencies have acknowledged by encouraging the inclusion of DHT-derived outcomes as exploratory end points in clinical trials.
In the near future, digital outcomes could support measures such as neurofilament levels and electrophysiologic markers to capture early treatment signals and refine phase 2 to 3 decision making. At some time in the future, one might imagine DHT-derived outcomes replacing their respective ALSFRS-R subdomain counterparts, allowing for objective, detailed, and remotely collected outcome measures that can be tested in hierarchical order to demonstrate efficacy in clinical trials. Alternatively, one could combine these different measures—both clinically and statistically—into a single, composite end point to reflect the overall response to treatment. To establish how best to undertake this approach would be an interesting challenge. Such a digital surrogate—combined with definite end points such as survival—allows for data flow optimization to enhance operational efficiencies.47 In addition, it facilitates the decentralization of clinical trials, potentially improving enrollment and reducing attrition rates, diversifying populations typically enrolled, and lowering the burden for those participating.48 While the need for less burdensome research has been emphasized by expert consensus guidelines and patient advocacy groups,49,50 realizing this potential is much more ambitious, requiring extensive and coordinated research efforts, likely only possible within international consortia.51
Current Gaps, Challenges, and Potential Steps Forward
In summary, the field has grown significantly in recent years, resulting in a vast array of technologies to monitor patients, as highlighted by this review. Although some variety in technological solutions is essential to encompass all aspects affected by ALS, much of the research has focused on developing new outcomes while targeting the same disease aspect. This has yielded an abundance of small, proof-of-concept studies with limited evidence to support their use in real-world practice. At the same time, there are several disease aspects that have received limited attention despite being perceived as important by patients, such as fatigue and cognition.52 Broad consensus on a “core digital outcome set” to harmonize research within and across disease aspects could, therefore, be of significant value to the field, providing a clear direction for future research and stimulating collaboration. These harmonization efforts should decide not only on “what to measure” but also on “how to measure,” prioritizing easily scalable and device-agnostic measurements that are independent of proprietary software.4,53 It is important to note that voices of end-users must be included to develop user-centric technologies and measurement protocols,54 such as selecting outcomes important to all stakeholders.
A collaborative approach, with the development of consensus guidelines and standards, streamlines the generation of evidence required to support DHTs in real-world practices, encompassing proof of validity, reliability, clinical relevance, and clinical benefit. In addition, there is an unrealized opportunity to increase the value of DHT-generated data, which, at present, are typically isolated from the broader research community. Opt-in clauses for data sharing, without compromising patient privacy, incorporated in new and existing protocols could enable data aggregation in digital registries and large international databases, similar to current efforts aimed at unraveling the genetic basis of ALS. Several initiatives are already underway, such as the ALS Research Collaborative and PRECISION ALS,55,56 which facilitate direct comparison outcomes across studies, analysis of subgroups, insight into possible confounders, and development of prediction and disease-specific artificial intelligence models and provide ways for external validation, among other factors.
As the regulatory and reimbursement landscape continues to evolve, alternative pathways may become increasingly accessible to alleviate current challenges in generating evidence of cost-effectiveness for digital technologies, for example, by using flexible reimbursement models and collection of real-world data.43,44
Although most DHTs were deemed feasible and well tolerated by patients, follow-up durations were limited and the suitability of these technologies to monitor patients during more advanced disease stages or increasing cognitive impairment remains largely unproven. Moreover, special attention should be aimed at engaging diverse and lower digitally literate patients to increase accessibility and prevent exacerbating existing health disparities,57 for example, by integrating technologies in existing patient portals, providing metrics that promote self-management, or including a digital health counselor in the clinical team.58
Still, the uptake of DHTs may largely depend on their successful integration in existing research and care pathways. While the pharmaceutical industry could lead the effort by adopting DHTs as explanatory end points in clinical trials, implementing these technologies in routine care may be more complicated, in part because of the required reimbursement by health insurance companies and interoperability with existing infrastructure. Nevertheless, while significant challenges remain unsolved, the adoption and coverage of DHTs to remotely monitor patients have been steadily increasing, allowing us a glimpse into the future on how these technologies may better serve the needs of patients. These knowledge gaps and challenges hindering the uptake of DHTs in the clinical practice of today, and the potential solutions to facilitate their use in the care and research of tomorrow, are summarized in Table 1.
Table 1.
Current Knowledge Gaps, Challenges, and Potential Steps Toward Increased Use of Digital Health Technologies in Real-World Practice
| Topic | Gaps and challenges | Potential solutions |
| User-centric harmonization | DHT-derived outcomes aimed at capturing similar disease concepts are highly heterogeneous in used devices, software, metrics, and measurement protocols with limited involvement of end-users | Consensus on a “core digital outcome set” could provide a clear direction for future research, stimulate collaboration, and accelerate progress. These harmonization efforts should not only decide on “what to measure” but also on “how to measure,” prioritizing easily scalable, device-agnostic outcomes that are independent of proprietary algorithms. To bring everyone on board, a global task force consisting of multiple stakeholders is likely required. Beyond academic leaders and industry, engagement of patients and health care providers is critical for the design of user-centric tools, including selecting outcomes important to all stakeholders |
| Validity and reliability | DHT-derived outcomes studied beyond their proof-of-concept are rare | Collaborate with peers to build the extensive body of evidence required to support the use of DHTs in practice, unified by the “core digital outcome set.” Guide research by using structured frameworks to facilitate regulatory approval, such as the V3 model.59 Compare novel outcomes with accepted measures beyond the ALSFRS-R, including patient-reported outcomes and survival. Incorporation of novel outcomes into clinical trials is salient to establishing surrogacy |
| Clinical relevance | DHT-derived outcomes bear limited clinical relevance, are clinically inactionable, and without meaningful effect sizes | Establish meaningful change, for example, by anchoring changes in novel outcomes with clinically relevant milestones or changes reported in global measures of change. For care purposes, determine how DHTs should be integrated into routine practice, which may include defining early warning thresholds and clinically actionable tasks. Normative values, preferably sex, age, and education-matched, may be warranted to further assist clinical decision making |
| Clinical benefit | DHTs lack comparative and cost-effectiveness data against current practice | Determine added benefit by comparing DHTs against current practice, for example, by reducing in-clinic visits or visit time, or demonstrating lower measurement burden or sample size compared with traditional outcomes. Beyond the traditional clinical trial, consider real-world data approaches in assessing the cost-effectiveness of DHTs |
| Analytics | Current analytics are rudimentary with limited improvement, lacking the ability to fully appreciate the data | Explore how data are best aggregated to optimize signal-to-noise ratios. The use of different modalities may provide a more holistic disease perspective but require the simultaneous assessment of multiple devices. Be aware of extrapolating algorithms to ALS populations because many have been developed for non-neurologic populations in laboratory settings. Develop and validate new algorithms if needed, preferably strongly based on existing and conceptually meaningful metrics to ease interpretability and mitigate overoptimism.60 Continued efforts are warranted to identify and minimize the influence of possible confounders, such as artifacts due to care aids (e.g., wheelchair use) and cognitive deficits. Transparency of data processing and open-source code might be a first step to improve analytics |
| Measurement protocol | Studies investigating the optimum between protocol, signal detection, burden and patient preferences are rare | Investigate patient preferences and burden regarding device type, wear locations, monitoring schemes, and follow-up duration, as well as explore the optimal tradeoff between signal capturing and measurement burden. Unwanted temporal effects should be considered and their influence minimized, such as learning effects (particularly for task-based assessments), day-to-day variability, and seasonal influences |
| Generalizability, equality, and empowerment | Study populations are skewed toward highly motivated, digitally literate patients with limited information on advanced disease stages or from patients with cognitive impairments | Engage and support low digitally literate populations to increase generalizability and bridge the “digital divide.” Ease user complexity by integrating DHTs in existing patient portals with easy-to-understand interfaces and facilitate ongoing technical support. Provide training and involve caregivers; including a digital health counselor could be considered. Provide patients with access to their data by using user-friendly outcomes and reports that promote self-management and provide incentive for continued participation |
| Data privacy, transparency, and trust | Data derived from DHTs are privacy sensitive, complicating widespread data sharing and reuse | Patients should be empowered to decide under which conditions their data may be shared, using standard opt-in clauses combined with easily understandable information about policies and associated risks. Ongoing efforts are needed to protect patient privacy by continuously updating cybersecurity measures and data protection regulations |
| Integration and data management | Data derived from DHTs commonly flow outside the existing clinical infrastructure and are kept in silos, isolated from the broader research community | Foster digital interoperability, allowing for seamless flow of data between DHTs, electronic health records, and patient portals. Collection of data in digital registries and international databases brings new opportunities for outcome development |
Abbreviations: ALS = amyotrophic lateral sclerosis; ALSFRS-R = ALS Functional Rating Scale-Revised; DHT = digital health technology.
Conclusion
In conclusion, the use of DHTs has the potential to reshape our care and research pathways. This transition has, to date, not yet fully transpired, and only a few technologies are being used in selected settings, while the provision of care largely continues with regular, in-clinic visits. This is mainly driven by uncertain cost-benefit assessments, limiting perpetual use of DHTs beyond the initial pilot or outside a research context. For drug development, none of the outcomes derived from DHTs has been comprehensively validated and their added value in capturing clinical meaningful information is largely unknown. Given the limited evidence to date, the possibility also remains that these technologies are ineffective and add unnecessary burden, complexity, and expense. Nevertheless, seeking harmonization among all stakeholders through consensus guidelines and consortia is required to fully realize the potential of DHTs to better meet the needs of patients living with this devastating disease.
Acknowledgment
C.J. McDermott is supported by the NIHR Biomedical Research Centre and an NIHR Research Professor Award.
Glossary
- AIMS
ALS Impairment Multidomain Scale
- ALS
amyotrophic lateral sclerosis
- ALSFRS-R
ALS Functional Rating Scale–Revised
- DHT
digital health technology
- EIM
electrical impedance myography
- MIP
maximal inspiratory pressure
- PROM
patient-reported outcome measure
- PROOF
Patient-Ranked Order of Function
- ROADS
Rasch-Built Overall ALS Disability Scale
- VC
vital capacity
Author Contributions
J.W.J. van Unnik: drafting/revision of the manuscript for content, including medical writing for content; major role in the acquisition of data; study concept or design; analysis or interpretation of data. L. Ing: drafting/revision of the manuscript for content, including medical writing for content; major role in the acquisition of data. M. Oliveira Santos: drafting/revision of the manuscript for content, including medical writing for content; major role in the acquisition of data. C.J. McDermott: drafting/revision of the manuscript for content, including medical writing for content; study concept or design; analysis or interpretation of data. M. de Carvalho: drafting/revision of the manuscript for content, including medical writing for content; study concept or design; analysis or interpretation of data. R.P.A. van Eijk: drafting/revision of the manuscript for content, including medical writing for content; study concept or design; analysis or interpretation of data.
Study Funding
This study was supported by Stichting ALS Nederland (TRICALS-Reactive II).
Disclosure
The authors report no relevant disclosures. Go to Neurology.org/N for full disclosures.
References
- 1.Feldman EL, Goutman SA, Petri S, et al. Amyotrophic lateral sclerosis. Lancet. 2022;400(10360):1363-1380. doi: 10.1016/S0140-6736(22)01272-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.US Food and Drug Administration. Digital Health Technologies for Remote Data Acquisition in Clinical Investigations. Updated July 29, 2024. Accessed January 20, 2025. fda.gov/regulatory-information/search-fda-guidance-documents/digital-health-technologies-remote-data-acquisition-clinical-investigations. [Google Scholar]
- 3.Gupta AS. Digital phenotyping in clinical neurology. Semin Neurol. 2022;42(1):48-59. doi: 10.1055/S-0041-1741495 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.van Eijk RPA, Beelen A, Kruitwagen ET, et al. A road map for remote digital health technology for motor neuron disease. J Med Internet Res. 2021;23(9):e28766. doi: 10.2196/28766 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Bhushan NL, Romano CD, Gras-Najjar J, et al. Remote-use applications of the Amyotrophic Lateral Sclerosis Functional Rating Scale-Revised clinical outcome assessment tool: a scoping review. Value Health. 2024;27(10):1454-1465. doi: 10.1016/j.jval.2024.05.005 [DOI] [PubMed] [Google Scholar]
- 6.van de Rijn M, Paganoni S, Levine-Weinberg M, et al. Experience with telemedicine in a multi-disciplinary ALS clinic. Amyotroph Lateral Scler Frontotemporal Degener. 2018;19(1-2):143-148. doi: 10.1080/21678421.2017.1392577 [DOI] [PubMed] [Google Scholar]
- 7.Geronimo A, Wright C, Morris A, Walsh S, Snyder B, Simmons Z. Incorporation of telehealth into a multidisciplinary ALS Clinic: feasibility and acceptability. Amyotroph Lateral Scler Frontotemporal Degener. 2017;18(7-8):555-561. doi: 10.1080/21678421.2017.1338298 [DOI] [PubMed] [Google Scholar]
- 8.Selkirk SM, Washington MO, McClellan F, Flynn B, Seton JM, Strozewski R. Delivering tertiary centre specialty care to ALS patients via telemedicine: a retrospective cohort analysis. Amyotroph Lateral Scler Frontotemporal Degener. 2017;18(5-6):324-332. doi: 10.1080/21678421.2017.1313867 [DOI] [PubMed] [Google Scholar]
- 9.James N, Power E, Hogden A, Vucic S. Patients' perspectives of multidisciplinary home-based e-Health service delivery for motor neurone disease. Disabil Rehabil Assist Technol. 2019;14(7):737-743. doi: 10.1080/17483107.2018.1499139 [DOI] [PubMed] [Google Scholar]
- 10.Bakker LA, Schröder CD, Tan HHG, et al. Development and assessment of the inter-rater and intra-rater reproducibility of a self-administration version of the ALSFRS-R. J Neurol Neurosurg Psychiatry. 2020;91(1):75-81. doi: 10.1136/JNNP-2019-321138 [DOI] [PubMed] [Google Scholar]
- 11.Erb MK, Calcagno N, Brown R, et al. Longitudinal comparison of the self-administered ALSFRS-RSE and ALSFRS-R as functional outcome measures in ALS. Amyotroph Lateral Scler Frontotemporal Degener. 2024;25(5-6):570-580. doi: 10.1080/21678421.2024.2322549 [DOI] [PubMed] [Google Scholar]
- 12.Manera U, Cabras S, Daviddi M, et al. Validation of the Italian version of self-administered ALSFRS-R scale. Amyotroph Lateral Scler Frontotemporal Degener. 2021;22(1-2):151-153. doi: 10.1080/21678421.2020.1813307 [DOI] [PubMed] [Google Scholar]
- 13.Meyer T, Spittel S, Grehl T, et al. Remote digital assessment of amyotrophic lateral sclerosis functional rating scale: a multicenter observational study. Amyotroph Lateral Scler Frontotemporal Degener. 2023;24(3-4):175-184. doi: 10.1080/21678421.2022.2104649 [DOI] [PubMed] [Google Scholar]
- 14.Genge A, Cedarbaum JM, Shefner J, et al. The ALSFRS-R Summit: a global call to action on the use of the ALSFRS-R in ALS clinical trials. Amyotroph Lateral Scler Frontotemporal Degener. 2024;25(3-4):382-387. doi: 10.1080/21678421.2024.2320880 [DOI] [PubMed] [Google Scholar]
- 15.US Food and Drug Administration. Rare Neurodegenerative Disease Grants Awarded. Updated November 1, 2024. Accessed March 8, 2025. fda.gov/industry/fda-rare-neurodegenerative-disease-grants-program/rare-neurodegenerative-disease-grants-awarded. [Google Scholar]
- 16.van Eijk RPA, van den Berg LH, Lu Y. Cultivating patient preferences in ALS clinical trials: reliability and prognostic value of the patient-ranked order of function. Neurology. 2024;103(2):e209502. doi: 10.1212/WNL.0000000000209502 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Johnson SA, Burke KM, Scheier ZA, et al. Longitudinal comparison of the self-entry Amyotrophic Lateral Sclerosis Functional Rating Scale-Revised (ALSFRS-RSE) and Rasch-Built Overall Amyotrophic Lateral Sclerosis Disability Scale (ROADS) as outcome measures in people with amyotrophic lateral sclerosis. Muscle Nerve. 2022;66(4):495-502. doi: 10.1002/mus.27691 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.de Jongh AD, van Eijk RPA, Bakker LA, et al. Development of a Rasch-built amyotrophic lateral sclerosis impairment multidomain scale to measure disease progression in ALS. Neurology. 2023;101(6):e602-e612. doi: 10.1212/WNL.0000000000207483 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Gray D, Lesley R, Mayberry EJ, et al. Development, reliability, validity, and acceptability of the remote administration of the Edinburgh Cognitive and Behavioural ALS Screen (ECAS). Amyotroph Lateral Scler Frontotemporal Degener. 2024;25(1-2):96-103. doi: 10.1080/21678421.2023.2278512 [DOI] [PubMed] [Google Scholar]
- 20.Neumann M, Kothare H, Ramanarayanan V. Multimodal speech biomarkers for remote monitoring of ALS disease progression. Comput Biol Med. 2024;180:108949. doi: 10.1016/j.compbiomed.2024.108949 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Stegmann GM, Hahn S, Liss J, et al. Early detection and tracking of bulbar changes in ALS via frequent and remote speech analysis. NPJ Digit Med. 2020;3:132. doi: 10.1038/s41746-020-00335-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Neumann M, Kothare H, Ramanarayanan V. Combining multiple multimodal speech features into an interpretable index score for capturing disease progression in amyotrophic lateral sclerosis. Interspeech. 2023;2023:2353-2357. doi: 10.21437/interspeech.2023-2100 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Rutkove SB, Narayanaswami P, Berisha V, et al. Improved ALS clinical trials through frequent at-home self-assessment: a proof of concept study. Ann Clin Transl Neurol. 2020;7(7):1148-1157. doi: 10.1002/acn3.51096 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.van Unnik JWJ, Bakers JNE, Kokx S, et al. Portable fixed dynamometry enables home-based, reliable assessment of muscle strength in patients with amyotrophic lateral sclerosis: a pilot study. Amyotroph Lateral Scler Frontotemporal Degener. 2023;24(7-8):651-660. doi: 10.1080/21678421.2023.2231494 [DOI] [PubMed] [Google Scholar]
- 25.Warmerdam E, Hausdorff JM, Atrsaei A, et al. Long-term unsupervised mobility assessment in movement disorders. Lancet Neurol. 2020;19(5):462-470. doi: 10.1016/S1474-4422(19)30397-7 [DOI] [PubMed] [Google Scholar]
- 26.Bloem BR, Post E, Hall DA. An apple a day to keep the Parkinson's disease doctor away? Ann Neurol. 2023;93(4):681-685. doi: 10.1002/ana.26612 [DOI] [PubMed] [Google Scholar]
- 27.van Unnik JWJ, Meyjes M, Janse van Mantgem MR, van den Berg LH, van Eijk RPA. Remote monitoring of amyotrophic lateral sclerosis using wearable sensors detects differences in disease progression and survival: a prospective cohort study. eBioMedicine. 2024;103:105104. doi: 10.1016/j.ebiom.2024.105104 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Pinto A, Almeida JP, Pinto S, Pereira J, Oliveira AG, de Carvalho M. Home telemonitoring of non-invasive ventilation decreases healthcare utilisation in a prospective controlled trial of patients with amyotrophic lateral sclerosis. J Neurol Neurosurg Psychiatry. 2010;81(11):1238-1242. doi: 10.1136/jnnp.2010.206680 [DOI] [PubMed] [Google Scholar]
- 29.Tattersall R, Carty S, Meldrum D, Hardiman O, Murray D. The patient's perspective of remote respiratory assessments during the COVID-19 pandemic. Amyotroph Lateral Scler Frontotemporal Degener. 2022;23(1-2):76-80. doi: 10.1080/21678421.2021.1920982 [DOI] [PubMed] [Google Scholar]
- 30.Helleman J, Bakers JNE, Pirard E, van den Berg LH, Visser-Meily JMA, Beelen A. Home-monitoring of vital capacity in people with a motor neuron disease. J Neurol. 2022;269(7):3713-3722. doi: 10.1007/S00415-022-10996-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Geronimo A, Simmons Z. Remote pulmonary function testing allows for early identification of need for non-invasive ventilation in a subset of persons with ALS. J Neurol Sci. 2024;459:122971. doi: 10.1016/j.jns.2024.122971 [DOI] [PubMed] [Google Scholar]
- 32.Stegmann GM, Hahn S, Duncan CJ, et al. Estimation of forced vital capacity using speech acoustics in patients with ALS. Amyotroph Lateral Scler Frontotemporal Degener. 2021;22(suppl 1):14-21. doi: 10.1080/21678421.2020.1866013 [DOI] [PubMed] [Google Scholar]
- 33.Küffner R, Zach N, Norel R, et al. Crowdsourced analysis of clinical trial data to predict amyotrophic lateral sclerosis progression. Nat Biotechnol. 2015;33(1):51-57. doi: 10.1038/nbt.3051 [DOI] [PubMed] [Google Scholar]
- 34.Braga AC, Pinto A, Pinto S, de Carvalho M. Tele-monitoring of a home-based exercise program in amyotrophic lateral sclerosis: a feasibility study. Eur J Phys Rehabil Med. 2018;54(3):501-503. doi: 10.23736/S1973-9087.18.05129-8 [DOI] [PubMed] [Google Scholar]
- 35.Garcia-Gancedo L, Kelly ML, Lavrov A, et al. Objectively monitoring amyotrophic lateral sclerosis patient symptoms during clinical trials with sensors: observational study. JMIR MHealth UHealth. 2019;7(12):e13433. doi: 10.2196/13433 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Ginsburg GS, Picard RW, Friend SH. Key issues as wearable digital health technologies enter clinical care. N Engl J Med. 2024;390(12):1118-1127. doi: 10.1056/NEJMra2307160 [DOI] [PubMed] [Google Scholar]
- 37.Helleman J, Kruitwagen ET, van den Berg LH, Visser-Meily JMA, Beelen A. The current use of telehealth in ALS care and the barriers to and facilitators of implementation: a systematic review. Amyotroph Lateral Scler Frontotemporal Degener. 2020;21(3-4):167-182. doi: 10.1080/21678421.2019.1706581 [DOI] [PubMed] [Google Scholar]
- 38.Helleman J, van Eenennaam R, Kruitwagen ET, et al. Telehealth as part of specialized ALS care: feasibility and user experiences with “ALS home-monitoring and coaching.” Amyotroph Lateral Scler Frontotemporal Degener. 2020;21(3-4):183-192. doi: 10.1080/21678421.2020.1718712 [DOI] [PubMed] [Google Scholar]
- 39.Borges do Nascimento IJ, Abdulazeem H, Vasanthan LT, et al. Barriers and facilitators to utilizing digital health technologies by healthcare professionals. NPJ Digit Med. 2023;6(1):161. doi: 10.1038/s41746-023-00899-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Paganoni S, van de Rijn M, Drake K, et al. Adjusted cost analysis of video televisits for the care of people with amyotrophic lateral sclerosis. Muscle Nerve. 2019;60(2):147-154. doi: 10.1002/mus.26606 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.León-Salas B, González-Hernández Y, Infante-Ventura D, et al. Telemedicine for neurological diseases: a systematic review and meta-analysis. Eur J Neurol. 2023;30(1):241-254. doi: 10.1111/ene.15599 [DOI] [PubMed] [Google Scholar]
- 42.Hobson EV, Baird WO, Bradburn M, et al. Using telehealth in motor neuron disease to increase access to specialist multidisciplinary care: a UK-based pilot and feasibility study. BMJ Open. 2019;9(10):e028525. doi: 10.1136/bmjopen-2018-028525 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Stern AD, Brönneke J, Debatin JF, et al. Advancing digital health applications: priorities for innovation in real-world evidence generation. Lancet Digit Health. 2022;4(3):e200-e206. doi: 10.1016/S2589-7500(21)00292-2 [DOI] [PubMed] [Google Scholar]
- 44.Schmidt L, Pawlitzki M, Renard BY, Meuth SG, Masanneck L. The three-year evolution of Germany's Digital Therapeutics reimbursement program and its path forward. NPJ Digit Med. 2024;7(1):139. doi: 10.1038/s41746-024-01137-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Masanneck L, Gieseler P, Gordon WJ, Meuth SG, Stern AD. Evidence from ClinicalTrials.gov on the growth of digital health technologies in neurology trials. NPJ Digit Med. 2023;6(1):23. doi: 10.1038/s41746-023-00767-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.European Medicines Agency. Qualification of novel methodologies for medicine development. Updated June 4, 2020. Accessed January 20, 2025. ema.europa.eu/en/qualification-novel-methodologies-medicine-development.
- 47.van Unnik JWJ, Nikolakopoulos S, Eijkemans MJC, et al. Development and evaluation of a simulation-based algorithm to optimize the planning of interim analyses for clinical trials in ALS. Neurology. 2023;100(23):e2398-e2408. doi: 10.1212/WNL.0000000000207306 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Inan OT, Tenaerts P, Prindiville SA, et al. Digitizing clinical trials. NPJ Digit Med. 2020;3:101. doi: 10.1038/s41746-020-0302-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.van den Berg LH, Sorenson E, Gronseth G, et al. Revised Airlie House consensus guidelines for design and implementation of ALS clinical trials. Neurology. 2019;92(14):e1610-e1623. doi: 10.1212/WNL.0000000000007242 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.I AM ALS. Patient-Centric Trial Design (PaCTD) Rating Criteria. Accessed January 27, 2025. iamals.org/patient-centric-trial-design-pactd-rating-criteria/. [Google Scholar]
- 51.Mantua V, Arango C, Balabanov P, Butlen-Ducuing F. Digital health technologies in clinical trials for central nervous system drugs: an EU regulatory perspective. Nat Rev Drug Discov. 2021;20(2):83-84. doi: 10.1038/d41573-020-00168-z [DOI] [PubMed] [Google Scholar]
- 52.Zizzi C, Seabury J, Rosero S, et al. Patient reported impact of symptoms in amyotrophic lateral sclerosis (PRISM-ALS): a national, cross-sectional study. eClinicalMedicine. 2023;55:101768. doi: 10.1016/j.eclinm.2022.101768 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Musson LS, Mitic N, Leigh-Valero V, et al. The use of digital devices to monitor physical behaviour in motor neuron disease: a systematic review. J Med Internet Res. 2025;27:e68479. doi: 10.2196/68479 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Helleman J, Johnson B, Holdom C, et al. Patient perspectives on digital healthcare technology in care and clinical trials for motor neuron disease: an international survey. J Neurol. 2022;269(11):6003-6013. doi: 10.1007/s00415-022-11273-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.ALS Therapy Development Institute. ALS Research Collaborative (ARC). Accessed January 17, 2025. als.net/arc/. [Google Scholar]
- 56.McFarlane R, Galvin M, Heverin M, et al. PRECISION ALS-an integrated pan European patient data platform for ALS. Amyotroph Lateral Scler Frontotemporal Degener. 2023;24(5-6):389-393. doi: 10.1080/21678421.2023.2215838 [DOI] [PubMed] [Google Scholar]
- 57.Ibrahim H, Liu X, Zariffa N, Morris AD, Denniston AK. Health data poverty: an assailable barrier to equitable digital health care. Lancet Digit Health. 2021;3(4):e260-e265. doi: 10.1016/S2589-7500(20)30317-4 [DOI] [PubMed] [Google Scholar]
- 58.Perret S, Alon N, Carpenter-Song E, et al. Standardising the role of a digital navigator in behavioural health: a systematic review. Lancet Digit Health. 2023;5(12):e925-e932. doi: 10.1016/S2589-7500(23)00152-8 [DOI] [PubMed] [Google Scholar]
- 59.Bertha A, Alaj R, Bousnina I, et al. Incorporating digitally derived endpoints within clinical development programs by leveraging prior work. NPJ Digit Med. 2023;6(1):139. doi: 10.1038/s41746-023-00886-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Berisha V, Liss JM. Responsible development of clinical speech AI: bridging the gap between clinical research and technology. NPJ Digit Med. 2024;7(1):208. doi: 10.1038/s41746-024-01199-1 [DOI] [PMC free article] [PubMed] [Google Scholar]



