Digital and precision clinical trials: innovations for testing mental health medications, devices, and psychosocial treatments

Abstract

Mental health treatment advances - including neuropsychiatric medications and devices, psychotherapies, and cognitive treatments - lag behind other fields of clinical medicine such as cardiovascular care. One reason for this gap is the traditional techniques used in mental health clinical trials, which slow the pace of progress, produce inequities in care, and undermine precision medicine goals. Newer techniques and methodologies, which we term digital and precision trials, offer solutions. These techniques consist of (1) decentralized (i.e., fully-remote) trials which improve the speed and quality of clinical trials and increase equity of access to research, (2) precision measurement which improves success rate and is essential for precision medicine, and (3) digital interventions, which offer increased reach of, and equity of access to, evidence-based treatments. These techniques and their rationales are described in detail, along with challenges and solutions for their utilization. We conclude with a vignette of a depression clinical trial using these techniques.

Background: Why do we need to change our clinical trial techniques?

A grand challenge in mental health is to provide highly effective drug, device, or psychosocial treatment to those in most need and with equity [1]. We are coming up short in addressing this challenge. Treatments for major mental disorders such as schizophrenia and depression often provide incomplete benefits and sometimes significant side effects. A considerable proportion of those seeking treatment either receive little to no benefit in the example of treatment-refractory depression [2] or have residual disability due to illness domains (e.g., cognitive) that are untouched by existing medications in the example of schizophrenia [3]. Further, provision of treatment is often inequitable, especially for minoritized racial and ethnic groups and those in rural areas.

Yet, it often seems that treatment development and testing are getting harder, more expensive, and more failure-prone [4]. Developing entirely new medications beyond the monoaminergic drugs has been limited since the original psychopharmacology revolution of the 1950 s. Treatment optimization has been minimal: with some notable exceptions (e.g., augmentation in treatment-resistant depression), we still have little evidence base for the optimal use of our medications and psychotherapies to bring about the best possible benefits. Precision medicine – highly-effective treatments tailored to individual patients – has remained elusive [5]. Finally, the inequity of mental health treatment begins with clinical trials suffering a lack of representation of minoritized racial and ethnic groups and rural patients.

These challenges can be overcome, but only if we change how we conduct clinical trials. A great deal of progress is being made in clinical trial methodologies referred to as “digital and precision trials”. However, this progress is not widely known or implemented by clinical trialists. Therefore, the purpose of this article is to make the field of neuropsychiatry research aware of the methods of digital and precision trials, why they are necessary, and where they are in the developmental cycle.

How has our field been limited by the existing pace and methods of clinical trials?

Slow and inadequate recruitment

Trials in mental health (including neuropsychiatric, behavioral, substance use, and cognitive disorders) are slow because recruitment is slow, and many fail because of the inability to recruit adequate numbers of participants. This slow pace is a major hindrance to progress, which depends on large sample sizes in trials for definitive tests and subgroup analyses. For comparison, the largest NIH-funded study of antihypertensives enrolled 42,000 individuals [6], while the largest NIH-funded study of antidepressants, STAR*D, enrolled 4,000 individuals [7]. Even STAR*D was an extreme outlier, as most mental health trials enroll a few hundred individuals or fewer. Despite these small sample sizes, the trials tend to be expensive and lengthy, taking many years to conduct. If this continues – if our trials will always be slow and expensive to study only a few hundred individuals – it will be a serious impediment to progress in mental health, which is already limited in contrast to other fields of clinical medicine [8].

High failure rate

Treatments often find no separation from a placebo or other control, and initially intriguing findings of treatment-relevant subgroups or treatment optimization often fail to replicate. Again, this is in part because of the recruitment challenge leading to small sample sizes. It is also due to the slow pace of our trials, meaning that it is difficult to improve their quality and success rate through repetition and iterative learning [9]. But it is also due to measurement imprecision. Our traditional outcome measures, such as the Hamilton Depression Rating Scale, have fairly low precision, including test-retest reliability [10, 11]. Moreover, our measurements of target engagement (e.g., neurocircuitry or behavioral tasks that are proposed to assess the mechanism by which treatment works) sometimes have reliability that is low or unreported [12–14]. This low precision is a main reason why our field has not made progress towards treatment optimization and precision medicine [15].

Poor fit

There is great concern that many of our interventions that have been developed are highly labor-intensive, overly specialized, have poor system fit and are either unavailable or suboptimal for large swaths of the population [16–18]. In part this is because many single site or even multi-site trials are not broadly generalizable, and that methods to enhance recruitment of participants with busy lives, racial and ethnic minoritized communities and those who live in rural areas are needed. This has been a driver of the appetite for digital interventions.

In short, these three concerns – the slow pace of trials mainly due to recruitment challenges; high measurement error limiting both trial success and the ability to get to precision medicine; and subsequent challenges in implementing interventions – are all addressable by the digital and precision trial techniques shown in Fig. 1 and described below.

Fig. 1.

Digital and precision trials offer solutions for big problems in mental health.

What are the methods of digital and precision trials?

Decentralized trials improve recruitment, generalizability, and potentially success rate of trials

A traditional clinical trial recruits only participants who are in close proximity to the trial site and are able to come in person for often-frequent visits. As a result, recruitment rate and generalizability are highly limited by who is willing and able to come to the trial site. To get around this problem trials use multiple, often numerous sites each recruiting a small number in order to get to a large overall sum. The cost, complexity, and quality concerns of multi-site clinical trials have been described in detail [19–21].

In contrast, in a decentralized or fully-remote trial, patients participate entirely through digital and/or telemedicine interfaces, with no (or very little) face-to-face contact with the trial site [22]. Decentralized trials use a small number, or just one, site to recruit from a wide geographic area – even an entire country. Screening is done over the internet and via phone/videoconference, along with e-consent; treatments are shipped or provided digitally or via telemedicine; and outcome measures are also conducted remotely, via digital survey, telephone/videoconference, or having assessments (e.g., at-home EEG) shipped to the participant’s home. For more intensive interventions (e.g., injections or infusions), local providers such as visiting nurses administer treatments and assess safety.

Decentralized trials have existed for many years, with one significant example being the BRIGHTEN study, which randomized a nationally representative sample of 1,098 participants in five months in order to test apps for depression [23]. One might have thought that the prospect of rapidly recruiting a large, broadly generalizable sample would have resulted in an immediate and widespread uptake of this technique by trialists in mental health. However, this innovation was largely ignored until the COVID-19 pandemic forced a widespread shift to remote measures and intervention; as a result, clinicians and policy-makers suddenly realized what patients had known for a long time: remote intervention and assessment is feasible, often desirable because of reduced burden and increased convenience; e-consent is simple and ethical; and intervention at a distance can be done without loss of (or even with improvement in) safety and fidelity.

An early example during the pandemic was the STOP COVID trial, a fully-remote trial [24] in which SARS-CoV-2 positive patients were screened over the phone, e-consented, had study medication and assessment supplies delivered to them, and had follow-up and clinical management done via at-home testing, surveys, and telephone calls [25]. The study managed to randomize 176 patients, was completed in 5 months despite the difficulties of carrying out such work early in the pandemic, and successfully demonstrated not only the efficacy of fluvoxamine in reducing COVID-19 related clinical deterioration (this preliminary demonstration was replicated in a larger study published the following year), but also the safety and tolerability of the drug with rapid titration. Other COVID-19 trials used similar techniques to recruit throughout the US and other countries [26–28].

The speed at which fully-remote studies can be conducted for conditions of high concern is impressive. As an example: in response to concerns about potential suicide rates during the early phases of COVID-19, the UW ALACRITY Center and the Center for Suicide Prevention, Assessment and Research conducted a rapid, large scale national trial of people at risk for suicide and who identified as either an essential worker or unemployed due to COVID-19 restrictions. In 4 weeks, 1,356 participants across the US were randomized to one of four mobile applications to test their effects on risk for suicide, and data collection was completed during that time frame [29]. This timely completion helped clarify the role that these apps can play in mitigating adverse mental health effects during a public health crisis.

Well-known advantages of decentralized trials are their reach and recruitment: because participants do not have to be physically proximate to, and able to repeatedly visit, the trial site, recruitment can be from any part of the country or even the world, using wide-capture techniques such as digital ads, social media, health care systems’ electronic medical records, and patient groups interested in the illness. Along with this advantage is the ability to enrich sample demographics for those participants who have historically been under-represented in research [23, 30–33].

Relative costs may be smaller, depending on the type of intervention to be studied. The main cost saving is from reducing delays: recruitment can be accelerated simply by allotting more money to the above techniques [34]. In addition, fewer sites means fewer contracts and less bureaucracy (e.g., one IRB rather than many). Then, the only real limitation to the speed of a fully-remote clinical trial is the amount of human infrastructure available to conduct it.

Another significant benefit of decentralized trials is high quality due to experience and expertise. This benefit ensues from the low number of sites (even one) required to recruit a large and generalizable sample. Each new study requires the development of experience working with a new intervention, measurements, and other techniques. This is true even when working with expert trialists. This gaining of experience through repetition and learning is one of the major predictors of the success of large projects [9]. Trials are both complex and unique, so both expertise and experience are needed. Quality through experience is far more likely if one committed expert study group is randomizing 1,000 participants rather than 100 sites each randomizing 10 participants.

Decentralized trials may be a particularly fitting setting for the use of artificial intelligence (AI) in clinical trials. It has been proposed that AI could speed and enhance clinical trials in numerous ways, including optimizing design, accelerating initiation and recruitment (e.g., by matching potential trial participants with appropriate trials), improving patient adherence and preventing patient dropouts, improving safety oversight, and improving data collection and analysis [35, 36]. This is in its infancy, but we can expect to see a great increase in AI throughout clinical trials in the future [37].

Although these methods are attractive, they do come with their own set of concerns. These include the potential for attracting fraudulent participants (often referred to as gaming) and the challenges in engaging participants in continuing data collection.

Fraudulent participants are a growing problem in research [38]. Fraud takes a number of forms, including computer programs that run scripts to complete surveys (bots), one person signing up multiple times on different devices (gamers), multiple individuals who work for an organization to extract money from studies (cyber cafes), and individuals who sign up for research for the financial benefit and are not honest in their responses (fraudsters). Each of these types of fraudulent participation can be managed with the use of individualized study links sent to pre-verified participants. In this instance, each potential participant is verified by collecting IP address (a unique identifier that computers use to communicate with each other), age, zip code, and email, then asking for one’s birth year and if using a virtual private network (VPN) on a separate page. Once this is collected, an algorithm can verify IP address matches to zip code and age matches birth year. Additional safeguards include providing a check for nonsensical responses and text in open ended questions, the use of captcha, and having false question directions in an image format. Once participants are verified they are then sent a one-time, individualized study link that cannot be shared with others [39–41]. Survey programs such as Qualtrix and SurveyMonkey also have new methods for screening out potentially fraudulent participants [42]. Strategies include IP address logging, coordinating self-reported data with geolocation, conducting video interviews to verify identity, and built in attentional and consistency checks. However, few studies today report on which, if any, strategies they utilize and the impact of these [43]. Even with these protections in place, data must be reviewed regularly to ensure that no one slipped through the system. Data review strategies include the use of “non-reactive indicators” [44] checking for time to complete surveys (and excluding participants who answer surveys much more rapid than normal speeds), straightlining (when participants provide their answers all in the same place, such as all “1 s”), and algorithmic patterning (the use of algorithms to create repetitive patterns). A final set of recommendations is to consider advertising on research platforms such as Prolific and advocacy websites such as Mental Health America as sites like Facebook may be more prone to fraud; [45] and to not advertise any financial incentive.

Engagement and retention challenges can be managed through the use of various incentives, such as graduated incentive values that increase in value overtime, the use of effort recognition to encourage continued engagement such as gif experiences, point or badge assignments, fun awards, return of information from surveys, and useful tips [46, 47].

Regarding return of information, some data suggests that patients value feedback as much as monetary rewards [46]. However, methods to share novel types of digital data like smartphone based digital phenotyping remain nascent, which is likely why there are few examples today [48]. Return of digital marker results has been incorporated into the ongoing large-scale AMP Schizophrenia study as an example of the newer research in this space addressing the need for feedback to participants [49].

These solutions show that the unique challenges of decentralized trials are addressable. As more investigators and sponsors accept decentralized trials, these methods will continue to evolve, potentially overcoming some of the major obstacles to progress in the field of mental health treatment testing.

Precise measurement improves trial success and is essential for precision medicine

Clinical trials are experiments, and the success of any experiment depends critically on assay sensitivity – the ability to precisely measure the outputs of the experiment [50]. This requires outcome measures with high reliability, including test-retest reliability. Many articles have been published about the poor signal to noise ratio problems caused by low test-retest reliability of outcome measures, including greater sample size requirements and reduced treatment effect size as measured in the trial [51, 52]. This article will not reiterate those arguments but will instead highlight why precise measurement – of both outcomes and mechanisms – is not only desirable for clinical trials in general but essential for getting to precision medicine.

There has been a strong push for precision medicine in mental health interventions. This is understandable given the overall modest effect size of treatments for mental disorders. It would still be ideal if some very high-effect size treatments were developed, akin to the discovery of game-changing medications such as penicillin, highly life-saving techniques such as modern obstetric techniques, or highly-effective procedures such as coronary artery bypass. But with few exceptions (such as electroconvulsive therapy for catatonia), our treatments only help a small proportion of patients. Additionally, in clinical trials the low effect size of treatments stems partly from a high rate of improvement in the placebo (or other control) condition [53].

The complexity of the brain, the heterogeneity of underlying syndromes such as schizophrenia and major depression, and the experience of clinicians and patients that different people respond to different treatments, all point to precision medicine as a hope for improving the outcomes of patients. The underlying basis of precision medicine is that, within the construct of one clinical syndrome such as major depression, there exist many subgroups of patients who have either essentially different illnesses or different biological (or psychosocial) features which cause them to require different treatments.

The challenge is not only discovering that heterogeneity but measuring it in a clinical trial. One established method for doing so, favored by NIMH, is called target engagement. Namely, the mechanism by which an intervention works (e.g., changing a specific feature of psychopathology or neurobiology) needs to be measured in the trial alongside the clinical outcome, and one must demonstrate (1) a change in that measure with treatment, or “engagement” of the target, plus (2) a change in the outcome measure, plus (3) correlation of change in the target measure with improvement in the outcome measure. NIMH’s approach to target engagement is nuanced depending on whether a clinical trial studies a pharmacologic or psychosocial intervention and the phase of development (as outlined in NIMH PARs 21-133, 21-135, and 21-137).

A second method is to identify at baseline a subgroup that is more or less likely to respond to treatment. In this case, the subgroup might be measured by a genetic factor (e.g., genotype affecting a medication’s pharmacokinetics or pharmacodynamics), neuroimaging finding, clinical feature, etc. The critical test is whether that factor is a moderator of the treatment: namely, whether the effect size (e.g., drug-placebo difference in the outcome measure change) is greater in the presence of that factor vs. the absence.

Baseline factors alone are unlikely to be sufficient for getting to precision medicine: because treatment response reflects a dynamic relationship between the treatment and the patient, both baseline and dynamic measures are needed [54]. Nevertheless, many precision medicine efforts utilize only this baseline moderator approach. A classic example is the short allele of the SLC6A4 gene promoter, which influences transcription of, and therefore the amount of, serotonin transporter in the synapse. Because the serotonin transporter is thought to be the target of SSRIs, this functional genetic variation in the SLC6A4 promoter was a strong candidate for influencing the likelihood that a depressed patient will improve with an SSRI. Notably, despite numerous studies totaling several thousand individuals testing this hypothesis, there is not a scientific consensus that this relationship is true (although some commercial pharmacogenetic tests utilize it) [55, 56].

A third method, the predominant type of precision medicine in clinical use, is individualized treatment optimization. This means that once a treatment is started, it is individually adjusted according to the patient’s response, tolerability, and acceptability. This may mean iterative dose and timing adjustments, as well as combining the treatment with other interventions that are additive, synergistic, or a therapeutic permissive. This is seen in routine psychiatric care, although there is very little evidence base underlying it. In clinical trials, this is often seen in flexible-dosing studies, augmentation trials, Micro-Randomized Trials (MRT), Sequential Multiple Assessment Randomized Trials (SMART) and Just-In-Time-Adaptive-Intervention (JITAI) trials [57, 58].

But for any of these techniques – target engagement, baseline moderation, or optimization - to work in clinical trials, high measurement precision is needed. In the target engagement example, it is essential that the target can be measured precisely enough so that its change is reliably measured (daunting because change scores are per force less reliable than baseline scores). The necessary test-retest reliability for this type of mediational analysis is estimated to be a correlation of 0.9 or greater [59]. There is increasing recognition that precision medicine (i.e., understanding individual differences) requires very high reliability [60] but this has not yet led to a sea-change in how clinical trials are conducted. In general, there is no requirement to demonstrate high test-retest reliability for measures in clinical trials. This must change; to advance precision medicine, trialists will need to focus on reducing measurement error.

There are three major sources of error that explain why measures in mental health may have low test-retest reliability. First, they often measure subjective states which can be hard to rate (e.g., “how is your mood?”); second, the variable being measured may be a transient state, so it varies day-to-day or within-day (e.g., mood, energy, cognitive function/performance) or is intermittent (e.g., a measurement of hallucinations); third, the measurement has some inherent noise (e.g., most behavioral or cognitive performance measures).

All of these sources of measurement error can be improved by frequent measurement which is possible via the techniques of ecological momentary assessment (EMA) and digital phenotyping. EMA studies subjective thoughts and feelings (e.g., depressed mood) by repeatedly collecting data in an individual’s environment (e.g., “what is your level of depression right now?”) [61]. Digital phenotyping uses multi-modal, usually passively-collected, data from smart devices to create a digital picture of an individual’s behavior based on, e.g., keystroke speed [62], activity, and internet use [63].

EMA is more reliable than retrospective measurement (asking them this same assessment over the past week) [64], and some research has demonstrated that EMA improves the ability to detect treatment effects [65, 66]. A similar example is the more recent proliferation of brief self-administered cognitive assessments designed to be conducted multiple times daily for each time point (e.g., the baseline pre-treatment assessment) [67, 68]. Therefore, trials using digital data collection can precisely measure and test mechanisms and outcomes through the technique of repeatedly sending surveys or assessments to the participant via their smartphone or other device.

In addition to reducing error through repetition, these measures may provide greater validity through their ecological nature: assessing patients in their lived environment rather than the artificial environment of the research clinic. Further, EMA may also improve measurement validity by posing to patients a question that is easier for them to answer: most of us could answer without difficulty how our energy is at this exact moment but would have difficulty summarizing how our energy has been on average over the past week [10].

Beyond reducing measurement noise, temporally dense data (such as highly repeated surveys or continuous passive measurement) can provide additional opportunities for examining dynamic relationships, such as how some symptoms or behaviors drive others and how interventions may ameliorate these relationships. An example outside of neuropsychiatry is continuous glucose monitoring, which has been found both superior to individual measures and capable of measuring blood sugar control dynamically [69]. Similarly, a recent innovation in mental health is the study of dynamic relationships, e.g., between motivation and behavior in schizophrenia [70]. The opportunity for innovative assessment seems particularly salient to mental health, where there has been little change in outcome measures since the 1950s (when the Hamilton scales were created for assessing depressive and anxiety symptoms) [71].

Trials using multimodal digital data collection to test mechanisms and outcomes offer a second use case. For example, the Accelerating Medicine Partnership® (AMP®) - Schizophrenia study seeks to predict the trajectory of first-episode psychosis and is employing longitudinal digital phenotyping alongside more traditional assessments like neuroimaging, genetics, EEG etc [72]. This multimodal data collection may prove to be a risk biomarker or as early indicators of clinical benefit in trials, but also can help inform the context and often unmeasured covariates impossible to capture with traditional assessment approaches. Given the central role of sleep, physical activity, and social connections as both a trigger and symptom of many mental illnesses, the scalable nature of digital phenotyping to directly capture related objective data offers unique potential for mechanistic research.

One final source of unwanted noise in mental health clinical trials is the placebo response, which is both high and rising [53]. An intriguing question is whether we could reduce placebo response in neuropsychiatric clinical trials through the use of more precise and innovative measurement techniques. To the extent that changes in outcome measures in the placebo group could result from noise – an unreliable baseline score, endpoint score, or both – improved reliability and validity through frequent ecological measures would reduce this noise. As an example from another field (hypertension), the move from single in-office measurements of blood pressure endpoints to ambulatory assessment results in a near-complete reduction in placebo response [73]. Moreover, ecological and otherwise more-meaningful measurements could reduce placebo response by directly assessing the core of a neuropsychiatric illness-related suffering and impaired quality of life that most traditional assessments only indirectly assay [74, 75]. However, what we call “placebo response” is more complex than measurement error and includes expectancy and naturalistic improvement of the illness [76]. The recent treatment studies in psychedelics highlight that managing expectancy may be a necessary part of many treatments [77]. Efforts to manage placebo response, then, may not desire to completely eradicate it but rather to make it homogeneous across patients and treatment arms. Here again, digital techniques could help by tightly controlling treatment expectancy, or “set and setting”, using (for example) the same instructions and expectations for each patient.

So what is the downside of using digital data collection for EMA, other repeated measures, and digital phenotyping? Unfortunately, since most of these measures are “novel” they require validation, and the mental health field has largely avoided innovations in measurement in our clinical trials. The argument is that if it’s not a widely-validated consensus measure, it should not be used. This barrier to innovation must be overcome. Fortunately, there are efforts underway to do so in depression; the Symptoms of Major Depressive Disorder Scale is a new FDA approved patient-reported outcome for assessing effects of rapid-acting antidepressant, and an EMA version is under development [78]. This means that studies testing drugs and devices for FDA approval could then use these scales. Meanwhile, behavioral/psychotherapy studies and other treatment research not requiring FDA approval can perform their own validation tests alongside the clinical trials. Trials also need to understand practice and reactive effects of high-density testing or questionnaires (less of an issue with passive sensor monitoring). Another concern is participant burden – it may be burdensome to answer multiple daily surveys over many weeks (although it also is burdensome to drive to a clinical trial site and undergo a lengthy in-person assessment). A third concern is expense and clinical site burden related to device and app maintenance and failure. There is a proliferation of apps that provide EMA or digital phenotyping, and with each passing year these become more widely available, less “buggy”, and less expensive to purchase and maintain.

Digital interventions are scalable

The 2023 White House report on mental health research priorities highlights the need for intervention strategies that increase access to care while maintaining (or hopefully improving) the quality of mental health care [79]. There is good reason for this emphasis: it is estimated that 40% of individuals with mental illness do not receive treatment [80], a scathing statistic that stems from the low availability of psychiatrists and therapists. A low supply of psychiatrists and therapists is expected to persist [81], but even if supply could be improved, there are many settings in which mental health professionals tend not to be available, including rural settings, settings serving underinsured individuals, and scenarios (such as urgent problems that need rapid help, and problems arising in the context of acute medical illness and medical settings such as perioperative) where mental illness is common yet treaters are rare to nonexistent. Even where mental health professionals are available, issues such as convenience, stigma, cultural fit, and personal preference may lead many patients to want alternatives to going to the office for care. These issues – scalability, reach, and preference – provide strong rationales for digital interventions in areas where broadband is available.

Digital intervention is a broad term that includes everything from traditional treatment delivered using digital technology, such as psychotherapy delivered using on-demand secure messaging, to psychoeducation using an app, to novel interventions that are designed to target a neural mechanism as is done in therapeutic video games.

One important issue in digital interventions is the need for a proper control condition [82]. Depending on the stage and context of the research [83], the control condition could be no treatment waitlist control, or, if the developer is interested in efficacy testing (e.g., to obtain FDA approval), the comparator must be an active control that does not have the therapeutic component as part of the intervention (for instance, a standard video game compared to one informed by a neurobiological construct).

One additional advantage of both digital interventions and digital surveys in decentralized trials is that they allow us to study engagement (either via querying participants about their experience or by directly measuring their engagement, i.e., time spent using the intervention) as well as clinical outcomes in a single study - and also assess interventions to improve engagement which could facilitate better reach and cost effectiveness.

A challenge for trials: equity/racial and ethnic representation [60]

The importance of equity in clinical trials is highlighted by the 2022 White House report, which underscores the need to advance equity as a cross-cutting research priority [79].

There is an assumption that digital tools increase equity for those living in rural areas, people of color, and historically under-represented ethnic groups [84]. However, this assumption ignores some key things. First, many rural states and low and middle income countries have either no broadband or unstable broadband, making the delivery and assessment of interventions challenging and data collection spotty to non-existent [85]. Although U.S. policy toward eradicating digital disparities is being discussed, research with rural communities and those living in low and middle income countries will remain a challenge for the time being. The significance of poverty and rurality on access to technology was highlighted during COVID-19 when many schools had to urgently transition to remote learning. For poor and rural dwelling students and families, education became inaccessible because of lack of internet access and limited technology [86].

A second equity challenge concerns the costs of devices and data plans. People of color, historically under-represented ethnic groups and many of those in rural areas are living with low incomes. Data plans in the US are expensive, and many families who live in poverty have older generation phones, limited data, and share their devices with other family members. Additionally, participants may change their phone number regularly to capitalize on data plan promotions by switching service providers, which can make keeping track of these participants very challenging. In some cultures, such as indigenous populations [87], the definition of family extends beyond the household and into an entire community. These communities will continue to be under-represented unless there is a concerted effort to overcome these disparities. Luckily many apps are tied to accounts with specific logins which means that the same person can participate even when they have a different device or phone number; these apps can also capture data on the phone and make the research team aware of these changes. Ideally, many apps should work in offline modes so they are accessible regardless of internet connection.

A third consideration is the limited availability of digital intervention and assessment to disabled people and older adults. App developers often do not consider American Disabilities Act (ADA) [88] regulations when creating digital solutions. To be accessible to those with poor literacy or visual impairments, surveys and apps must include voice-over tools that can read text, images, and digital movement. Many standard voice-over tools do not adequately read text on screens with graphics, unlabeled buttons, or movement. Additionally, voice-over tools perform better when there is minimal distraction and color on the screen, with the contrast between text and background needing to be clear [89]. While many older adults now are experienced in the use of computers, tablets and phones, particularly since COVID-19 [90], many devices, such as sports watches and smartphones are more challenging to use because of age-related changes in dexterity and movement control [90]. There are few digital interventions for anxiety or depression that have been tested in older adults [91]. Similarly, apps that provide cognitive assessment will need to be validated in older adults.

A final equity consideration is education and digital literacy in the use of technology to conduct research [92]. Investigators using digital technology to conduct research may need to offer training to participants or indicate how many people were not able to participate due to digital literacy. Digital literacy scales may also help to screen people for eligibility. Several recent scales have been published, with adequate validity and reliability [93], although most focus on older adult populations [94] and people in low and middle-income countries [95, 96].

What stands in the way of progress?

In addition to the challenges of equity described above, several challenges around approval/validation, engagement, privacy, and steep development costs remain.

According to a February 2023 FDA guidance, they refer to digital interventions and measures as Digital Health Technologies (DHT) and define these as systems that use computing platforms, connectivity, software, and/or sensors for health care and related uses [97]. They note that while some DHTs will meet the definition of a device under the Federal Food, Drug, and Cosmetic Act (FD&C Act), others do not and will not need to. Thus the use case will determine the level and degree of regulation. The FDA had in past years proposed the Pre-Certification program to streamline approval and use in trials, but in September 2022 ended the pilot with no plans to pursue it further [98].

As discussed above, digital endpoints related to mental health are not yet recognized by the FDA. While there is expanding research on these digital markers, recent reviews on depression [99], bipolar disorder [100], and schizophrenia [101] all conclude that the current research is not suitable to draw any firm conclusions. These reviews note the high degree of clinic heterogeneity, varied measurements, and unique methods that preclude advanced analytical methods or meta-analyses. However, digital measures are now being embedded into larger studies such as the global Accelerating Medicine Partnership Schizophrenia study [72]. This should open up a path towards validation and acceptance by regulatory bodies.

Embedding digital metrics into studies needs to strive for a low degree of missing data. A large European study offered smartphone digital phenotyping to offer 600 participants for up to 14 months but concluded that a more thorough investigation into the quality of the data was warranted before more complex analyses could be conducted [102]. The challenges around digital data missingness are often related to engagement for two reasons. First EMA data requires active engagement from participants as does charging a wearable sensor or smartphone. Second, even for digital phenotyping or smartphone sensor data, emerging research suggests that active engagement is still necessary as if the participant does not open the app after several days, the operating system on the device may simply throttle any sensor data capture [103]. While various solutions ranging from gamification to simply paying participants to engagement have been attempted, there has actually been less effort around sharing digital data streams back with participants [104]. Given the nature of digital data, future efforts to make digital data more interactive and accessible to those collecting it appears a promising area to enhance longitudinal data capture.

Privacy has remained another barrier to progress in digital markers. A 2020 study published in Lancet Digital Health reported that only 25% of American trust an academic or medical research institution with their electronic health data and only 12% trust a health technology company with their data [105]. With the March 2023 announcement that the Federal Trade Commission fined a mental health technology company nearly 8 million dollars for breaching patient privacy, there is a signal that the government may now begin to fully enforce digital health privacy. Already in 2023 numerous states are creating their own digital mental health privacy protection registration (e.g. Montana HB 446, Kentucky HB 196) with the hope that public faith in the privacy of digital mental health data may be restored.

A final barrier for digital data capture is the steep development, and often lengthy time scale, necessary to create a viable digital platform. While costs are rarely published, they certainly cost well over $200,000 considering the need for an Android app, Apple app, middleware, and database all designed to meet strict privacy regulations and high user interface/experience demands concomitant with mental health needs. This has led many academics to partner with industry although this also has risks given industry partners can abruptly cease to be research partners when they face financial challenges (e.g., Mindstrong in February 2023) or lose interest in research (e.g., Calm in Fall 2022; Meru Health 2023) [106]. Indeed, recent calls by congress to explain data-sharing practices in these companies have led many such companies to stop partnering with academics mid-study, despite having IRB approvals, informed consent processes, and funding from the NIH. Yet building platforms presents additional challenges as many are not maintained or updated once the research is completed. For example, a 2022 review of apps used in schizophrenia research found that 51% of those apps were not accessible and that those studies using non-psychosis-specific app platforms were more likely to be available than custom-created app solutions [107]. This suggests that for initial studies, leveraging existing app platforms through modifications and adaptions is likely the best path to ensure funds are used for science vs technology development. If there is a need to build an app, rather than repurpose an existing one, building in small incremental steps with frequent assessments will be more productive and waiting for the perfect app to be ready in the future.

In conclusion, here is a scenario of digital and precision trial techniques to test a new depression treatment

Imagine a clinical trial testing a new drug (or device or psychotherapy) for depression. This new treatment is highly targeted: it corrects a feature of depression pathophysiology not helped by traditional treatments and therefore holds promise to benefit a segment of individuals with currently treatment-refractory depression.

The clinical trial team decides to conduct a large, decentralized trial recruiting throughout the United States, Canada, and Mexico. Because the trial is fully remote, only three study teams are needed (one for each country). Because there are only three teams needed for all of North America, highly expert and committed clinical trial groups can be identified that focus on this study for its duration.

Additionally, because each team is responsible for recruiting a large sample, the teams develop increasing experience and expertise as the trial continues. The intimidating sample size of n = 2000 is feasible with only three teams because they use digital advertisements and patient support groups throughout North America to recruit a sample that is not only large but representative.

The pace of recruitment is determined based on infrastructure capabilities and an adequate advertising budget to achieve milestones in a timely manner. The study uses EMA combined with a novel digital phenotyping based upon participants’ own phone usage to collect temporally dense, innovative, and mechanistic data to understand not only whether the treatment works, but how well, in whom, and by which mechanisms it works. For example, the dynamic digital phenotyping data reveals that patients with early changes in activity levels after just a few days on the medications are highly likely to eventually be treatment responders at the end of six weeks. This means that future use of the medication can focus its use only on patients with this early dynamic change.

These ambitious aims are aided by the large sample size improving power but also by the high test-retest reliability of the measures improving validity. The small number of sites also speeds the trial in terms of limiting the amount of site-specific contracts, IRB-related bureaucracy, and data management complexity.

At the end of the trial, the research team has demonstrated the efficacy of a novel treatment that could help millions of individuals with hard-to-treat depression and gained an understanding of its optimal use. Further, both the fully remote recruitment and the engagement surveys conducted along with the intervention testing ensure that most racial and ethnic groups are well-represented and the needs of these and other traditionally underserved populations are assessed, leading to an intervention that can be implemented in an equitable manner.

Remaining and emerging scientific issues can then be immediately tested in a second decentralized trial by the same study groups who, now that they have gained significant experience during the first trial, can conduct the second with even greater speed and quality.

This scenario is fictional but achievable. Digital and precision techniques, described in detail in this paper, have considerable advantages for advancing neuropsychiatric treatment and are expected to see increasing use in both academia and the private sector.

Author contributions

EL, JT, and PA all participated in developing the concept for the manuscript, writing the manuscript, and critically reviewing and editing it.

Competing interests

EL: Consultant for Prodeo, Pritikin ICR, IngenioRx, Boehringer-Ingelheim, and Merck. Research funding from Janssen. Patent application pending for sigma-1 receptor agonists for COVID-19. JT: scientific advisory board of Precision Mental Wellness. PA: scientific advisory board of Headspace Health, Koa Health, and Chorus Sleep.