Abstract
Health care technologies have the ability to bridge or hinder equitable care. Advocates of digital mental health interventions (DMHIs) report that such technologies are poised to reduce the documented gross health care inequities that have plagued generations of people seeking care in the United States. This is due to a multitude of factors such as their potential to revolutionize access; mitigate logistical barriers to in-person mental health care; and leverage patient inputs to formulate tailored, responsive, and personalized experiences. Although we agree with the potential of DMHIs to advance health equity, we articulate several steps essential to mobilize and sustain meaningful forward progression in this endeavor, reflecting on decades of research and learnings drawn from multiple fields of expertise and real-world experience. First, DMHI manufacturers must build diversity, equity, inclusion, and belonging (DEIB) processes into the full spectrum of product evolution itself (eg, product design, evidence generation) as well as into the fabric of internal company practices (eg, talent recruitment, communication principles, and advisory boards). Second, awareness of the DEIB efforts—or lack thereof—in DMHI research trials is needed to refine and optimize future study design for inclusivity as well as proactively address potential barriers to doing so. Trials should incorporate thoughtful, inclusive, and creative approaches to recruitment, enrollment, and measurement of social determinants of health and self-identity, as well as a prioritization of planned and exploratory analyses examining outcomes across various groups of people. Third, mental health care advocacy, research funding policies, and local and federal legislation can advance these pursuits, with directives from the US Preventive Services Taskforce, National Institutes of Health, and Food and Drug Administration applied as poignant examples. For products with artificial intelligence/machine learning, maintaining a “human in the loop” as well as prespecified and adaptive analytic frameworks to monitor and remediate potential algorithmic bias can reduce the risk of increasing inequity. Last, but certainly not least, is a call for partnership and transparency within and across ecosystems (academic, industry, payer, provider, regulatory agencies, and value-based care organizations) to reliably build health equity into real-world DMHI product deployments and evidence-generation strategies. All these considerations should also extend into the context of an equity-informed commercial strategy for DMHI manufacturers and health care organizations alike. The potential to advance health equity in innovation with DMHI is apparent. We advocate the field’s thoughtful and evergreen advancement in inclusivity, thereby redefining the mental health care experience for this generation and those to come.
Keywords: Digital Mental Health Interventions, mental health, health equity, access to health care, health plan implementations
Introduction
The past few decades have marked significant momentum in the digital mental health field. More than 350,000 health-related mobile apps are available and a significant portion of these are specifically related to mental health support [1]. Enthusiasts postulate that digital mental health interventions (DMHIs) may eliminate commonplace access barriers [2,3] while also providing evidence-based mental health care that yields meaningful outcomes [4,5]. Moreover, there is a tenable undercurrent of hope, combined with appropriate and poignant questioning, on whether or not DMHI can genuinely and reliably reduce health care disparities. “Techquity” is a recently coined colloquial term that refers to this potential to either bridge or hinder equitable health care [6,7].
Problems with access to mental health care, both historical and present, are well documented across the United States and are especially pronounced within minoritized and underserved populations [8-13]. A key and increasingly common approach to understanding these inequities is through the study of social determinants of health (SDOH), which focuses on how the various circumstances in which people live affect their health and well-being [14,15]. Groups of individuals affected by systemic inequities in mental health care, in terms of both prevalence and care access, have been identified via various SDOH factors including, but not limited to, people who experience stigma or discrimination because of race, ethnicity, class, gender, sexual orientation (eg, LGBTQIA+ [Lesbian, Gay, Bisexual, Transgender, Queer, Intersex, Asexual+]); language or another aspect of identity; those with low household income or educational attainment; those who are underemployed or unemployed; older adults; those living in rural communities; members of Medicare or Medicaid; uninsured or underinsured people; veterans; and people living with disabilities [16-20]. When considering DMHIs, we must also acknowledge the “digital divide,” as many individuals may lack access, consistent connectivity, skills, or trust to engage with digital mental health technologies, further contributing to and even widening existing inequities, as was the case during the COVID-19 pandemic [21-23].
The arc of how to progress toward bridging equitable health care is multifaceted, complex, dynamic, and of course, evergreen. Scholars have contributed important theoretical and applied work toward addressing inequities in digital health and DMHI, focusing on a range of topics from development to dissemination [24-28]. This viewpoint, in the spirit of the overarching special issue theme “Reflecting on Transformative Technologies, Interventions, Methods, and Policy Issues,” aims to contribute to the conversation on health equity in US-based DMHIs in 2 ways. First, by referencing extant frameworks and best practices, and second, by articulating perspectives across key stages of product and organizational maturation, from initial product development to commercialization and health care ecosystem adoption (Figure 1; see also [14,25,28-41]). The present authorship collaboration will do this by drawing upon its collective expertise in several relevant domains, including scientific research in academia and industry, DMHI commercialization and deployment strategies from both industry and health care system perspectives, DMHI product development, and organizational leadership. We also include salient illustrative examples from real-world development at Woebot Health, a US-based company that manufactures DMHIs, with which a number of the authors are affiliated.
To begin to articulate the arc, we first focus on key elements in product and intervention development. Available development frameworks call for requisite community involvement, partnership, and trust throughout as well as embedded technological support for varied digital literacy levels [25,28]. Beyond development, multiple DMHI app evaluation frameworks have been published [42-44], some with specific callouts for the import of diversity equity and inclusion therein [45]. Moreover, industry manufacturers and academicians alike are encouraged to consider their company’s or lab’s integration of diversity, equity, inclusion, and belonging (DEIB) principles into talent recruitment, workstreams, and advisory boards. Next, we discuss considerations in evidence generation. Several recent systematic reviews and meta-analyses suggest that DMHI is effective for addressing symptoms of depression and anxiety [46-48], including across the age range from adolescents [49] to older adults [50]. However, simultaneous calls to action for enhanced diversity in clinical trials underscored the ongoing need for inclusive research [29,51], calling into question whether these conclusions may be biased due to unrepresentative samples [52]. Methodological strategies to facilitate inclusive clinical trial designs will be considered. The adoption of evidence-based DMHIs has been encouraged by the American Medical Association [53] and American Psychiatric Association [54], perhaps augmenting existing options for mental health care [55]. We also highlight the need for various ecosystems (academic, industry, payer, provider, and value-based care organizations) to articulate their requirements for health equity to be a part of a manufacturer’s DMHI product and evidence-generation portfolio before adoption or deployment within a system. Specification of considerations of an equity-informed commercialization strategy for evidence-based DHMI will be discussed.
Given the promising yet relatively nascent status of the DMHI field, we encourage transparent and open conversation and a collective articulation to help specify and manifest an evergreen process of monitoring for and reducing mental health care inequities.
Product Development
As noted in the World Health Organization’s Global Strategy on Digital Health 2020-2025, “When planning and prioritizing digital health interventions...the specific potential of digital technologies to promote health equity should be leveraged. Designed properly, digital solutions can propel inclusiveness as digital connectivity can transcend physical barriers” [56]. As any creation usually bears some imprint of the creator, a key starting principle for building an inclusive service is ensuring that the team that builds it represents a diverse set of perspectives and backgrounds, which include those of the intended users. Equity must also be baked into the product development life cycle. Similar to the European Union’s General Data Protection Regulation (GDPR) principle of “Privacy by Design,” we may think about this as “Equity by Design.” This entails explicit exploration of whether equity is likely to be achieved early on in design sessions, identifying any potentially problematic pitfalls and addressing them systematically. Human-centered design roots itself in the philosophy that thorough discovery and understanding of human needs optimize product design to meet those needs, as well as making the experience meaningful and satisfying [57]. Some human-centered design frameworks have been tailored to address DMHIs broadly [58], while others specifically target improving the health equity of DMHI [30,59]. For example, human-centered design approaches, which organically focus on inviting and hearing needs at hand, should be purposefully extended to include voices from marginalized communities [30,60], including communities with diverse digital literacy levels [61]. The PIDAR (Partner, Identify, Demonstrate, Access, Report) framework, developed by the Society of Behavioral Medicine’s Health Equity Special Interest Group, highlights partnerships as both integral to avoiding increased health inequities in DMHI and a fundamental first step in an equity-centric intervention development approach [25]. Methodologies for community-based participatory research have been articulated and incorporate strategies such as rapid prototyping and iteration for efficient colearning, as well as key human-centric and empathy-first principles of commitment, collaboration, and listening [62-64].
As more DMHIs begin to integrate aspects of artificial intelligence (AI), machine learning (ML), and large language models (LLMs), issues surrounding health equity in the rapidly evolving world of AI/ML are particularly salient to this conversation. These technologies hold great potential to, for example, enhance a conversational agent’s ability to understand natural language inputs as well as generate nuanced and tailored responses. However, there are a number of risks associated with the use of LLMs in mental health, both general and specific to health equity concerns. General concerns are LLM’s potential to “hallucinate” or generate false or misleading information as well as the limitations imposed by training data, which can be biased, inaccurate, or even actively harmful [65]. Equity-related LLM risks include their potential to exacerbate or amplify disparities if algorithms are constructed using data that reflect historical or societal biases and inequities; additionally, they increase skepticism and alienation of minoritized individuals due to lack of transparency and perception of bias [66,67]. Furthermore, many LLMs are trained primarily on a corpus of English sources, and may only be available in certain languages (eg, English), creating potential biases and risks for nonprimary language learners in understanding and interpreting these interventions, or directly excluding large swaths of people who do not speak certain languages [31,68,69]. Health equity–centered AI development frameworks are emerging to proactively address such risks [70-72]. For example, frameworks focused on the integration of LLMs into health care products recommend that LLMs are (1) thoroughly evaluated for equity-related risks, (2) rigorously monitored during deployment, (3) integrated into systems with humans-in-the-loop (eg, providers or users) to guard against biased algorithmic drift, and (4) trained in collaboration with impacted communities or populations they are intended for, all of which can help mitigate the exacerbation of potential biases [32,61,72]. Certainly, more research as well as the development and refinement of regulatory frameworks for responsible AI utilization in DMHI is called for overall [73]. President Biden’s Executive Order highlights the need to protect the “rights and safety of the public” through (1) strengthening AI governance, (2) advancing responsible AI innovation, and (3) managing risk from the use of AI [74].
Organizationwide Practices
Embedding DEIB practices into the fabric of organizations, from corporate objectives to hiring practices and more, has been advocated for openly, and is another key element of the larger techquity conversation with implications for DMHIs [33,75]. There is strong support for the benefits that a diverse workforce can bring to an organization in terms of key business outcomes (eg, innovation and revenue generation), as well as aligning with what talent may seek within a company’s culture [76,77]. For mental health–focused organizations, the inclusion of people with lived experience of mental health challenges is especially critical, as their involvement in all aspects of DMHI research, development, policy, and practice is fundamental to promoting equity [34,78]. Diversity at both the inception of digital health innovation (eg, within design teams at DMHI manufacturing organizations) and in health care systems deploying such innovations (eg, accountable care organizations or integrated delivery networks) is part of the holistic picture [33]. Organizational commitment to various endeavors and working groups (eg, podcasts, blogs, hiring practices, educational seminars, advisory boards, health benefits, and employee well-being programs, task forces) that host and elevate conversations on equity promote recognition of its fundamental importance as well as set a regular cadence of expected updates or outputs [79-82]. Examples are the DMHI industry commitment to supporting their own employees’ mental health via a variety of DEIB efforts (ie, Headspace [83]), the establishment of a clinical diversity advisory board (ie, Woebot Health [80]), and podcast interviews with industry leaders to discuss key DEIB topics (ie, Meeting of the Minds Podcast; Health Nonprofit Digital Marketing [82,84]). Powerful work led by the University of California San Francisco’s Taskforce on Equity and Anti-Racism in Research underscored several core learnings from their endeavors, among them a clear call for a diverse workforce across the full membership of the research team, as well as ongoing education and support for DEIB in research [35]. The National Institutes of Health (NIH) offers awards dedicated to engaging undergraduate students from diverse backgrounds in biomedical research [85,86] and Rush Education and Career Center Hub (REACH) offers support in STEM (Science, Technology, Engineering, Mathematics) research to students from preschool age onward [87]. Both of these highlight direct efforts to diversify researchers of the future.
Evidence Generation
Enhancing Health Equity in Evidence Generation for DMHIs: Frameworks and Methodological Considerations
The generation of evidence of the feasibility, acceptability, efficacy, effectiveness, and safety of DMHIs provides another critical opportunity for the consideration of health equity. Existing frameworks that have been used to explore the centering of health equity in research are PIDAR, RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance), and others, many of which highlight partnerships with diverse stakeholders and potential users, consideration of how access to technology and digital literacy affect eligibility, and how data are collected and disseminated [25,26,31]. In addition to these frameworks, we offer further methodological considerations to center evidence generation in health equity, drawing on our experience in implementing these practices.
Meta-analytic research has generally supported DMHI’s efficacy, feasibility, and acceptability [46-48]. However, despite the anticipated promise of DMHIs in addressing the health equity gap, little is known about outcomes across various sociodemographic subgroups and whether the trials recruited diverse samples; additionally, only a few studies have specifically focused on interventions intended for minoritized populations [31,88]. For example, a recent systematic literature review spearheaded by Woebot Health highlighted the underreporting of sociodemographic characteristics in clinical trials evaluating DMHIs and pointed to even fewer reporting any results by sociodemographic characteristics [36]. To examine the real impact of DMHIs on health equity, evidence-generation methodology needs to be thoughtfully designed and implemented.
Recruitment and Trial Design
First, the recruitment of diverse samples is the foundation upon which evidence generation rests. A variety of tactics can be used to recruit diverse samples, including the use of culturally sensitive recruitment materials; community outreach; thoughtful partnerships with socially, economically, geographically, or racially diverse health care systems; resources to support those with limited access to the internet; and diverse research and recruitment teams [25,28]. For example, in a recent research collaboration between Scripps Translational Science Institute and Woebot Health, utilization of such methods facilitated successful recruitment and enrollment of the a priori target of approximately 50% of a sample self-identifying as individuals from categories noted as historically underrepresented in biomedical research, including racial/ethnic minorities and rural/nonmetropolitan area residents, among others [89]. Real-world partnerships are particularly important for inviting collaboration with marginalized groups and exploring the potential of the DMHI to address health equity, with the necessity of real-world evidence outlined as an important component in several frameworks on health equity within DMHIs [26,28,31]. In these cases, clinician-referred recruitment should be managed with care and not utilized as a stand-alone recruitment strategy. Medical research has shown that clinicians and researchers may selectively exclude certain minority participants due to biases or assumptions about their interest in research, ability to follow study protocols, and likelihood of dropping out, among others [90]. Indeed, a large review [91] of minority participation in health research demonstrated that while minorities are willing to participate, they are less likely to be recruited. Where possible, health systems, patient registries, or other common recruitment sources should consider an “opt-out” method for study recruitment invitations. Specifically, upon entry to the system/registry, patients consent by default to be contacted for research opportunities unless they choose to “opt out” of such communications. This might allow for a greater diversity of individuals to be invited to participate in studies in the first place, without compromising their agency to decide which particular studies to apply for. To guide recruitment, specific targets for race, ethnicity, sexual orientation, and gender, among others, should be determined a priori based on the desired end users and relevant prevalence rates. While attempting to create targets based on census data or population prevalence is one viable option, we suggest aligning targets more closely with the prevalence rates of mental health problems in specific sociodemographic groups to more accurately capture real-world needs [92]. Additionally, to have a sufficient sample size to examine group differences, recruiting large enough samples or overrecruiting for a particular group(s) of interest may be necessary.
Sociodemographic Surveys and Data Collection
Second, sociodemographic surveys need to be thoughtfully designed and consistently implemented. Sociodemographic reporting should include culturally sensitive and inclusive response options to questions on race, ethnicity, sexual orientation, and gender identity, as well as questions related to SDOH such as food and housing insecurity [93]. To promote honest disclosure, participants should be given the opportunity to respond electronically and be assured of the privacy and protection of their sensitive data. Of note, it is often more challenging to collect comprehensive sociodemographic and SDOH data in the context of deployment outside of research settings. Thoughtful approaches to the collection and use of these data should be agreed upon by stakeholders and tested with end users to ensure that factors such as assessment burden, privacy concerns, and potential cultural differences in the appropriateness of questionnaire content are appropriately considered.
Analysis and Outcomes
Third, analyzing and reporting on sociodemographic characteristics and outcomes across subgroups are integral to examining the impact of DMHIs on health equity. At a minimum, DMHI research should provide the sociodemographic characteristics of their sample. Additionally, consumer-based DMHI data (outside the research context) should be presented in dashboards and reports that include a breakdown of sociodemographic characteristics, where possible and sample size permitting. Ideally, outcomes are also reported by sociodemographic groups of interest, or included as covariates in models to better understand their potential impact on or association with outcomes (see, eg, [94]). However, given the small sample sizes in many sociodemographic subgroups of interest, traditional hypothesis-confirming statistical methods of significance testing and modeling may not be possible. For such cases, we suggest focusing on exploratory, descriptive, and hypothesis-generating approaches such as within-group effect sizes (eg, the effect size of a symptom change score in non–Hispanic Black participants) and between-group effect sizes (eg, the effect size of the difference between symptom change scores in non–Hispanic Black and non–Hispanic White participants) where possible. Consolidating certain subgroups into another broader group may also be necessary in certain cases (eg, collapsing “genderqueer,” “nonbinary,” “agender,” and “different identity” into “other” for gender identity; see [95]), ideally to reach a sample size of at least 20-25 per subgroup [96], although greater care must be taken in the interpretation of these less precise groupings. For this exploratory approach, replication is key to identifying consistent and reliable patterns, and these analyses should be prespecified and conducted for every study even if only for internal purposes. Data from these exploratory analyses can be used to identify potential questions and hypotheses for future research, provide preliminary data for power analyses and study planning, and inform intervention improvements and possible precision targets. Researchers should also go beyond the assessment of differences in outcomes by sociodemographic characteristics, and consider how they can identify, capture, and analyze the structural factors (eg, discrimination, exclusionary policies, digital redlining) that may be responsible for creating or exacerbating inequities [97,98].
The outcomes assessed should be comprehensive and include efficacy (eg, symptom change), engagement (eg, app use metrics, therapeutic alliance), satisfaction and feasibility (including trust and perception of bias), and safety. Moreover, the assessed outcomes must reflect not only the priorities and values of the researchers, but also those of the participants. Doing so allows for a more nuanced understanding of the impact of DMHIs on various groups of people, and is necessary for a precision-based research and development approach that can be leveraged to address health equity and create better products. Qualitative data (eg, free-text questions about app experience, participant interviews) should be solicited, as they illuminate the nuance of experience across different groups of people and can reveal previously unknown issues or concerns among users.
Safety, in particular, is an important outcome to examine in the context of health equity [99]. Despite this, not all DMHI trials provide safety data and there is little standardization in how adverse events are captured and reported [100]. This may be, to a certain extent, because different types of DMHIs require varying degrees of safety capturing and oversight. Trials also vary considerably in the level of clinical support to mitigate and manage potential adverse events. Safety measurement must align with each intervention’s risk and capabilities to address safety concerns, while being as consistent as possible across DMHIs. Lastly, when generating internal safety reports for each trial, safety events and any other related data should be provided alongside sociodemographic data to allow exploration of prevalence rates within different groups of people.
All of these equity-centering research practices, from recruitment and data collection to analysis and safety considerations, should also be integrated into iterative internal product development processes and user experience (UX) research as well as external evidence-generation efforts.
Advocacy, Policy, Regulation, and Funding
Shifting our view “up” to another level, we turn to considerations of advocacy, policy, regulation, and funding, which may unlock paths to address inequities unavailable at the individual or even company level [27]. For example, the translation of mental health evidence–generation findings into new policies intended to improve health equity in mental health care has been bolstered by the implementation of advocacy efforts and new policy, regulation, and funding efforts at both the federal and local levels [52,101,102]. Advocacy efforts such as lobbying and grassroots campaigns focused on improving the equitable distribution of mental health services and awareness of mental health disparities have been critical in advancing policy change and allocation of resources as well [103,104].
In part due to these advocacy efforts, the past several decades have brought about various legislative initiatives focused on parity, community mental health initiatives, workforce diversity, and trauma-informed care that have served to improve mental health equity. Perhaps the most groundbreaking example of federal parity legislation in the past 25 years was the implementation of the Mental Health Parity and Addiction Equity Act of 2008 [105] which required insurance companies to offer mental health and substance use disorder care coverage similar to that of medical care, paving the way for equitable access to the support and services for those in need. Broader efforts to expand internet access have also evolved in recent years. These initiatives include broadband expansion that uses public or private funding to build the infrastructure needed to deliver internet to areas without current capabilities [106], net neutrality regulations established in 2003 [107], public Wi-Fi funded by governments or municipalities, and digital literacy programs that teach the fundamentals of using the internet [108]. The ability of users to connect online to DMHIs has been greatly enhanced by these collective efforts.
The regulatory oversight of medical devices by the Food and Drug Administration (FDA) can also impact digital mental health equity. Clinical trial diversity initiatives, regulatory science research and policies, public health education initiatives, postmarket surveillance activities, and drug/device presubmission interactions and approval processes have the ability to support or leave vulnerable populations in need of additional digital mental health care [109-111]. For example, following the onset of the COVID-19 pandemic and the nationwide public health emergency, the FDA announced in April 2020 that it would temporarily waive certain requirements for digital therapeutics targeting psychiatric disorders. This measure aimed to provide urgently needed mental health support tools to a struggling population, although it expired in November 2023 [112].
In addition, various organizations such as the American Psychological Association (APA) [101], the National Institute of Mental Health (NIMH) [52], the US Preventive Services Taskforce (USPSTF) [37], the Substance Abuse and Mental Health Services Administration (SAMHSA) [102], and others have provided support for recommendations and funding that have advanced mental health equity. For example, the USPSTF recently developed a framework to incorporate health equity into each step of their recommendation-making process. The NIMH’s ongoing Laboratories to Optimize Digital Health program seeks to fund innovative research projects that study ways to increase access, efficacy and effectiveness, and quality of DMHIs, particularly among those who experience health disparities [113]. Beyond the critical research focused on interventions to help more vulnerable populations, researchers should also focus on understanding individual-level differences in the prevalence, root causes, and mechanisms of expression of different mental health challenges [114-116]. Additional funding opportunities should be made available to train underrepresented minorities, develop culturally competent interventions, and determine the impact of the implementation of interventions or policies within underserved communities and health care settings. Trends in research that require the involvement of patients with lived experiences, interventions integrating peer support, and community-based participatory research [63] highlight the importance of including multiple stakeholders to identify priorities, develop and test interventions, and inform policy. These efforts reflect a commitment to advocating for and enacting policies that guide, regulate, and fund work that ensures a wide variety of users can access and benefit from DMHIs. Progress in these areas will be crucial in advancing equity across the field of DMHI in the future.
Commercial and Market Engagement Strategy
For-profit companies and commercial endeavors represent an opportunity to accelerate the traction of equity-related efforts in mental health. Company success in terms of market penetration and financial growth are well aligned and in many cases dependent on the factors described earlier (DEIB maturity in the workforce, more inclusive practices in product design and evidence generation, more representative and true to real-world evidence gathering, and reinforcement of equity-related policy). Today’s market rewards and reinforces equity-centered practices as customers are seeking products that meet sophisticated requirements in this regard. Deloitte estimated that their life sciences clients could garner a 10× return on their investment by bolstering equitable care access in underserved markets [38] and entrepreneurs focused on mental health equity are attracting early support and funding [117]. That said, challenges exist for for-profit ventures as well. For example, the cost of providing DMHI access to “hard to reach” populations (eg, those without a smartphone or internet access, or who are members of groups too small to amass commercial attention) may be greater, impacting profitability and sustainability of outreach efforts (see Galea [118] for an analogous case related to the greater cost of vaccinating certain vulnerable populations). Moreover, in cases where end users, and not an intermediary organization (eg, an employer, health plan, or health care provider), directly bear some or all of the cost for access to a DMHI, existing financial inequities may perpetuate or exacerbate inequities in mental health access and outcomes [119]. In both these cases, commercial stakeholders may consider steps such as advocating for ring-fenced funding, or allocation of protected funds that are specifically and exclusively intended for the purpose of reaching historically minoritized and underserved populations. Importantly, the purpose of these protected funds would not be to optimize customer acquisition costs, but rather to optimize representation.
Commercial and medical affairs functions in health care organizations such as DMHI and pharmaceutical manufacturers play a vital role in translating and reinforcing progress from science/policy out to the market and market demands back to internal groups [120]. Customer listening allows firms to collect very clear requirements on how products need to function and be deployed to address real-world equity-related challenges, helping product and evidence teams prioritize their road maps [39]. In the other direction, commercial and medical affairs teams showcase emerging work by science and product, building confidence and momentum in the market to go after equity-related unmet needs with more urgency.
Strong and creative commercial and medical affairs functions can also help build differentiated paths to market by developing tailored marketing, enrollment, and payment models to meet the specific needs of underserved populations. Customers who struggle to meet quality standards or equity requirements may be more inclined to pursue digital solutions aggressively. Manufacturers can foster more effective deployments by offering configurable solutions and flexibility in implementation and marketing. Additionally, not all DMHI manufacturers operate as for-profit ventures; exploring other models for addressing health inequities, such as the US State Department’s “Public Health 3.0” initiative [121] and nonprofit business models [122], is crucial for scaling new products.
Equity-Informed Implementation Science in Real-World Deployments
Although it appears as the final section of this paper, the importance of thoughtful consideration of health equity within and across DMHI ecosystem deployments cannot be underscored enough. This view is critical because it spans the individual, familial, and community levels, as well as the larger encompassing “ecosystem” organizations, such as health care systems or provider networks, which may be in unique positions to operate across multiple levels of SDOH to address inequities [27]. Deployment is also where “the rubber meets the road,” and the actual impact of DMHIs is realized.
Implementation science frameworks offer considerations into how evidence-supported interventions are “implemented” in routine practice in naturalistic, real-world contexts; and may, for example, recommend evaluation of factors such as immediate and sustained adoption of the intervention, cost, and barriers therein [123]. Such real-world data are of paramount importance in determining not only the real-world effectiveness of DMHIs, but also integral for informing and refining approaches to deployment processes (ie, provider training and patient-facing information). Moreover, such data create an opportunity for monitoring if the DHMI or its deployment characteristics are closing or exacerbating equity gaps, paving the way for deployment adjustments accordingly.
Ecosystems purchasing or deploying DMHIs (eg, payer, provider, value-based care organizations) are encouraged to transparently and reliably emphasize the need for health equity to be included in a manufacturer’s DMHI product and evidence-generation portfolio before adoption or deployment within their system. To begin, existing implementation science frameworks have been adapted to focus on health equity and can provide thoughtful questions for ecosystems and deployment partners to consider. These may include an early focus on “reach” (ie, who is included and excluded in different deployment strategies), proactive design and selection of interventions for vulnerable populations, and measuring and monitoring outcomes through the lens of addressing inequities [40]. Ecosystems may also consider adopting a health equity–informed implementation science framework [33], such as the Evidence- and Consensus-Based Digital Healthcare Equity Framework [41], or may choose to create and implement their own, as demonstrated by health care organizations such as Elevance [124], Kaiser Permanente [125], Reliant Medical Group [126], and the Centers for Medicare and Medicaid Services [127].
One example of how health equity might be considered in the deployment of a DMHI within a health care ecosystem is through the collection, monitoring, and evaluation of SDOH data. DMHIs (or their underlying platform infrastructure) could be designed to thoughtfully gather information about specific SDOH factors from patients. Such data could inform further risk assessment or clinical follow-ups, and facilitate the offering of available resources provided by the system to the patient. Processes like this may enable earlier intervention, more precise tailoring of interventions, and greater utilization of available support and resources for those in greatest need. This approach has the potential to benefit underserved individuals, DMHI manufacturers, and health care systems as the shift toward whole-person care evolves.
New models of care that support DMHI implementation are also emerging in postpandemic health care delivery: one such role is the digital navigator. This role facilitated the rapid transition to telehealth in the early days of the pandemic as a means to support patients accessing platforms to access care. The digital navigator can access which phone and data plan a patient has to optimally match them with a DMHI that takes the whole-person’s life context into account. This role is one method for addressing the friction from referral to activation that is often experienced in real-world deployments [128]. Other promising examples of alternative care models are stepped care [129,130] and collaborative care [131,132], both of which have accrued growing bodies of support in the literature, and which offer opportunities for DMHI integration and implementation. As these new models of care mature, data exchange to support the optimal levels of human and digital touchpoints can support the broader use of DMHIs in populations that may not traditionally seek care for mental health.
Conclusions
The potential to advance health equity through innovation in DMHIs is apparent, but by no means guaranteed. Here, utilizing decades of learning across multiple domains of expertise, we explored the progress and opportunities within the DMHI field to address mental health care inequities and articulated several steps essential to mobilize and sustain meaningful forward progression in this endeavor. While notable progress has been made over the past 25 years, we advocate for the field’s thoughtful and evergreen advancement in inclusivity and equity, thereby continuing to redefine the mental health care experience for this generation and those to come.
Acknowledgments
We thank Adam Platt, MSc, for coordinating scientific communications and facilitating the organization of the manuscript preparation and submission, as well as Katherine McGuckin, MS, for conceptual and drafting contributions to the “Commercial and Market Engagement Strategy” section. We also thank Julie Ross and Robbert Zusterzeel for their review and comments, and Stephanie Rapoport for helping to design the summary figure. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. Funding to publish this article as open access was provided by Woebot Health.
Abbreviations
- AI
artificial intelligence
- APA
American Psychological Association
- DEIB
diversity, equity, inclusion, and belonging
- DMHI
digital mental health intervention
- FDA
Food and Drug Administration
- GDPR
General Data Protection Regulation
- LGBTQIA+
Lesbian, Gay, Bisexual, Transgender, Queer, Intersex, Asexual+
- LLM
large language model
- ML
machine learning
- NIH
National Institutes of Health
- NIMH
National Institute of Mental Health
- PIDAR
Partner, Identify, Demonstrate, Access, Report
- REACH
Rush Education and Career Center Hub
- RE-AIM
Reach, Effectiveness, Adoption, Implementation, Maintenance
- SAMHSA
Substance Abuse and Mental Health Services Administration
- SDOH
social determinants of health
- STEM
Science, Technology, Engineering, Mathematics
- USPSTF
US Preventive Services Taskforce
Footnotes
Authors' Contributions: AR, in collaboration with RMM, MF, and AD, conceptualized the piece and outlined it. The introduction was drafted by AR and RMM. The “Product Development” section was drafted by AR with support from RMM and AD. The “Organizationwide Practices” section was drafted by AR. The “Evidence Generation” section was drafted by MF, with support from RMM and VLFH. The “Advocacy, Policy, Regulation, and Funding” section was drafted by VLFH with support from RMM. The “Commercial and Market Engagement Strategy” section was drafted by ML, AR, and RMM. The “Equity-Informed Implementation Science in Real-World Deployments” section was drafted by AR, RMM, and TH. TA, DCM, PW, and CG reviewed and provided contributions and feedback throughout. All authors contributed to multiple draft revisions, with ownership for incorporation of all author edits and flow of the overall manuscript championed by RMM and AR. All authors read and approved the submitted version.
Conflicts of Interest: AR, MF, VLFH, AD, TH, ML, and RMM are all employees of Woebot Health. DCM has accepted honoraria and consulting fees from Boehringer-Ingelheim, Optum Behavioral Health, Centerstone Research Institute, and the One Mind Foundation; royalties from Oxford Press; and has an ownership interest in Adaptive Health, Inc. TA has nothing to declare. PW is an associate editor at the Journal of Medical Internet Research and is on the editorial advisory boards of BMC Medicine, The Patient, and Digital Biomarkers; was a former consultant to Woebot Health; and is currently employed by Sano Genetics and holds stock/options in Sano Genetics. CG is a consultant to Woebot Health. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
References
- 1.IQVIA Institute for Human Data Science Digital health trends 2021: innovation, evidence, regulation, and adoption. IQVIA. 2021. [2024-04-23]. https://www.iqvia.com/insights/the-iqvia-institute/reports-and-publications/reports/digital-health-trends-2021 .
- 2.Radez J, Reardon T, Creswell C, Lawrence PJ, Evdoka-Burton G, Waite P. Why do children and adolescents (not) seek and access professional help for their mental health problems? A systematic review of quantitative and qualitative studies. Eur Child Adolesc Psychiatry. 2021 Feb;30(2):183–211. doi: 10.1007/s00787-019-01469-4. https://europepmc.org/abstract/MED/31965309 .10.1007/s00787-019-01469-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.National Institutes of Mental Health (NIMH) Technology and the future of mental health treatment. NIMH. 2023. [2024-04-23]. https://www.nimh.nih.gov/health/topics/technology-and-the-future-of-mental-health-treatment .
- 4.Philippe TJ, Sikder N, Jackson A, Koblanski ME, Liow E, Pilarinos A, Vasarhelyi K. Digital health interventions for delivery of mental health care: systematic and comprehensive meta-review. JMIR Ment Health. 2022 May 12;9(5):e35159. doi: 10.2196/35159. https://mental.jmir.org/2022/5/e35159/ v9i5e35159 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Faria Manuel, Zin Stella Tan Pei, Chestnov Roman, Novak Anne Marie, Lev-Ari Shahar, Snyder Michael. Mental health for all: the case for investing in digital mental health to improve global outcomes, access, and innovation in low-resource settings. J Clin Med. 2023 Oct 25;12(21):6735. doi: 10.3390/jcm12216735. https://www.mdpi.com/resolver?pii=jcm12216735 .jcm12216735 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Clark CR, Akdas Y, Wilkins CH, Rhee K, Johnson KB, Bates DW, Dankwa-Mullan I. TechQuity is an imperative for health and technology business: let's work together to achieve it. J Am Med Inform Assoc. 2021 Aug 13;28(9):2013–2016. doi: 10.1093/jamia/ocab103. https://europepmc.org/abstract/MED/34157112 .6307886 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Ipsos. The HLTH Foundation Ipsos. 2022. [2024-04-23]. https://www.ipsos.com/sites/default/files/ct/publication/documents/2022-03/Ipsos-HLTH-Path-to-techquity_0.pdf .
- 8.Cook BL, Trinh N, Li Z, Hou SS, Progovac AM. Trends in racial-ethnic disparities in access to mental health care, 2004-2012. Psychiatr Serv. 2017 Jan 01;68(1):9–16. doi: 10.1176/appi.ps.201500453. https://europepmc.org/abstract/MED/27476805 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Cook BL, Hou SS, Lee-Tauler SY, Progovac AM, Samson F, Sanchez MJ. A review of mental health and mental health care disparities research: 2011-2014. Med Care Res Rev. 2019 Dec;76(6):683–710. doi: 10.1177/1077558718780592. [DOI] [PubMed] [Google Scholar]
- 10.Erikson CE, Dent RB, Park YH, Luo Q. Historic redlining and contemporary behavioral health workforce disparities. JAMA Netw Open. 2022 Apr 01;5(4):e229494. doi: 10.1001/jamanetworkopen.2022.9494.2791603 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Institute of Medicine (US) How Far Have We Come in Reducing Health Disparities? Progress Since 2000: Workshop Summary. Washington, DC: National Academies Press; 2012. Introduction; p. 1. [PubMed] [Google Scholar]
- 12.Salmond S, Dorsen C. Time to reflect and take action on health disparities and health inequities. Orthop Nurs. 2022;41(2):64–85. doi: 10.1097/NOR.0000000000000828. https://europepmc.org/abstract/MED/35358124 .00006416-202203000-00003 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Schueller SM, Hunter JF, Figueroa C, Aguilera A. Use of digital mental health for marginalized and underserved populations. Curr Treat Options Psych. 2019 Jul 5;6(3):243–255. doi: 10.1007/s40501-019-00181-z. [DOI] [Google Scholar]
- 14.Allen J, Balfour R, Bell R, Marmot M. Social determinants of mental health. Int Rev Psychiatry. 2014 Aug;26(4):392–407. doi: 10.3109/09540261.2014.928270. [DOI] [PubMed] [Google Scholar]
- 15.Marmot M. Social determinants of health inequalities. Lancet. 2005;365(9464):1099–104. doi: 10.1016/S0140-6736(05)71146-6. https://core.ac.uk/reader/1669331?utm_source=linkout .S0140-6736(05)71146-6 [DOI] [PubMed] [Google Scholar]
- 16.National Institute of Mental Health (NIMH) Medicaid data show wide differences in mental health care in the United States. National Institute of Mental Health (NIMH) 2023. [2024-04-23]. https://www.nimh.nih.gov/news/science-news/2023/medicaid-data-show-wide-differences-in-mental-health-care-in-the-united-states .
- 17.Williams ND, Turpin RE, Akré Ellesse-Roselee L, Boekeloo BO, Fish JN. Disparities in mental health care access among persons differing in sexual identity: nationally representative findings. Psychiatr Serv. 2022 Apr 01;73(4):456–459. doi: 10.1176/appi.ps.202100045. https://europepmc.org/abstract/MED/34407630 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Kirkbride JB, Anglin DM, Colman I, Dykxhoorn J, Jones PB, Patalay P, Pitman A, Soneson E, Steare T, Wright T, Griffiths SL. The social determinants of mental health and disorder: evidence, prevention and recommendations. World Psychiatry. 2024 Feb;23(1):58–90. doi: 10.1002/wps.21160. https://europepmc.org/abstract/MED/38214615 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Compton MT, Shim RS. The social determinants of mental health. FOC. 2015 Oct;13(4):419–425. doi: 10.1176/appi.focus.20150017. [DOI] [Google Scholar]
- 20.Alegría Margarita, NeMoyer A, Falgàs Bagué Irene, Wang Y, Alvarez K. Social determinants of mental health: where we are and where we need to go. Curr Psychiatry Rep. 2018 Sep 17;20(11):95. doi: 10.1007/s11920-018-0969-9. https://europepmc.org/abstract/MED/30221308 .10.1007/s11920-018-0969-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Digital access: a super determinant of health. Substance Abuse and Mental Health Services (SAMHSA) 2023. [2024-04-23]. https://www.samhsa.gov/blog/digital-access-super-determinant-health .
- 22.Beaunoyer E, Dupéré Sophie, Guitton MJ. COVID-19 and digital inequalities: reciprocal impacts and mitigation strategies. Comput Human Behav. 2020 Oct;111:106424. doi: 10.1016/j.chb.2020.106424. https://europepmc.org/abstract/MED/32398890 .S0747-5632(20)30177-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Spanakis P, Peckham E, Mathers A, Shiers D, Gilbody S. The digital divide: amplifying health inequalities for people with severe mental illness in the time of COVID-19. Br J Psychiatry. 2021 Oct;219(4):529–531. doi: 10.1192/bjp.2021.56. https://europepmc.org/abstract/MED/35048887 .S0007125021000568 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Simmons D, Kaganoff S, Drasser K. Building toward equity: a working model for digital health. Rock Health. 2021. Aug, [2024-04-23]. https://rockhealth.com/insights/building-toward-equity-a-working-model-for-digital-health/
- 25.Miller SJ, Sly JR, Alcaraz KI, Ashing K, Christy SM, Gonzalez B, Lu Q, Newton RL, Redmond M, Shen M, Thomas-Purcell K, Yi J, Veinot T, Meade CD. Equity and behavioral digital health interventions: strategies to improve benefit and reach. Transl Behav Med. 2023 Jun 09;13(6):400–405. doi: 10.1093/tbm/ibad010. https://europepmc.org/abstract/MED/36940409 .7081265 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Jaworski BK, Webb Hooper M, Aklin WM, Jean-Francois B, Elwood WN, Belis D, Riley WT, Hunter CM. Advancing digital health equity: directions for behavioral and social science research. Transl Behav Med. 2023 Apr 03;13(3):132–139. doi: 10.1093/tbm/ibac088. https://europepmc.org/abstract/MED/36318232 .6786016 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Richardson S, Lawrence K, Schoenthaler AM, Mann D. A framework for digital health equity. NPJ Digit Med. 2022 Aug 18;5(1):119. doi: 10.1038/s41746-022-00663-0. doi: 10.1038/s41746-022-00663-0.10.1038/s41746-022-00663-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Friis-Healy EA, Nagy GA, Kollins SH. It is time to REACT: opportunities for digital mental health apps to reduce mental health disparities in racially and ethnically minoritized groups. JMIR Ment Health. 2021 Jan 26;8(1):e25456. doi: 10.2196/25456. https://mental.jmir.org/2021/1/e25456/ v8i1e25456 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.The White House. Washington, DC: The White House; [2024-04-23]. Advancing equity and racial justice through the federal government. https://www.whitehouse.gov/equity/ [Google Scholar]
- 30.Stiles-Shields C, Cummings C, Montague E, Plevinsky JM, Psihogios AM, Williams KDA. A call to action: using and extending human-centered design methodologies to improve mental and behavioral health equity. Front Digit Health. 2022;4:848052. doi: 10.3389/fdgth.2022.848052. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Ramos G, Hernandez-Ramos R, Taylor M, Schueller SM. State of the science: using digital mental health interventions to extend the impact of psychological services. Behavior Therapy. 2024 Apr;:e1. doi: 10.1016/j.beth.2024.04.004. (forthcoming) [DOI] [Google Scholar]
- 32.Singh N, Lawrence K, Richardson S, Mann DM. Centering health equity in large language model deployment. PLOS Digit Health. 2023 Oct;2(10):e0000367. doi: 10.1371/journal.pdig.0000367. https://europepmc.org/abstract/MED/37874780 .PDIG-D-23-00181 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Lawrence K. Digital health equity. In: Lawrence K, Linwood SL, editors. Digital Health. Brisbane, Queensland, Australia: Exon Publications; 2022. [PubMed] [Google Scholar]
- 34.Muchamore I, Karanikolas P, Gooding P. How lived experience expertise shapes research and development in digital mental health: a review of literature and insights. Wellcome Open Research. 2024. [2024-07-19]. https://wellcomeopenresearch.org/documents/9-373 .
- 35.Folk JB, Carter SP, Jagsi R, Mangurian C. Office of Diversity and Outreach UCSF. San Francisco, CA: Office of Diversity and Outreach UCSF; 2023. [2024-04-24]. Title IX offices at top United States universities with medical centers: staffing, investigations and transparency. https://research.ucsf.edu/sites/g/files/tkssra2931/f/UCSF_Task_Force_Report_Final_Anti-Racism_in_Research_IDEA.pdf . [Google Scholar]
- 36.Kirvin-Quamme A, Kissinger J, Quinlan L, Montgomery R, Chernenok M, Pirner MC, Pajarito S, Rapoport S, Wicks P, Darcy A, Greene CJ, Robinson A. Common practices for sociodemographic data reporting in digital mental health intervention research: a scoping review. BMJ Open. 2024 Feb 12;14(2):e078029. doi: 10.1136/bmjopen-2023-078029. https://bmjopen.bmj.com/lookup/pmidlookup?view=long&pmid=38346876 .bmjopen-2023-078029 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Lin JS, Webber EM, Bean SI, Evans CV. Development of a health equity framework for the US Preventive Services Task Force. JAMA Netw Open. 2024 Mar 04;7(3):e241875. doi: 10.1001/jamanetworkopen.2024.1875. https://jamanetwork.com/journals/jamanetworkopen/fullarticle/10.1001/jamanetworkopen.2024.1875 .2815866 [DOI] [PubMed] [Google Scholar]
- 38.Wilkins D. The business case for health equity in life sciences. Deloitte. 2022. [2024-04-24]. https://www2.deloitte.com/us/en/blog/health-care-blog/2022/the-business-case-for-health-equity-in-life-sciences.html .
- 39.Equitable healthcare starts with medical affairs. H1. 2022. [2024-04-24]. https://h1.co/blog/a-different-approach-and-a-wider-scope-equitable-healthcare-starts-with-medical-affairs/
- 40.Baumann AA, Cabassa LJ. Reframing implementation science to address inequities in healthcare delivery. BMC Health Serv Res. 2020 Mar 12;20(1):190. doi: 10.1186/s12913-020-4975-3. https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-020-4975-3 .10.1186/s12913-020-4975-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Hatef E, Austin M, Scholle S, Buckley B. Agency for Healthcare Research and Quality. Rockville, MD: Agency for Healthcare Research and Quality (US); 2024. Feb, [2024-04-24]. Evidence- and consensus-based digital healthcare equity framework: a practical guide for implementation. https://digital.ahrq.gov/ahrq-funded-projects/creating-digital-healthcare-equity-framework-accompanying-guide-its-use#nav-publications . [Google Scholar]
- 42.Lagan S, Sandler L, Torous J. Evaluating evaluation frameworks: a scoping review of frameworks for assessing health apps. BMJ Open. 2021 Mar 19;11(3):e047001. doi: 10.1136/bmjopen-2020-047001. https://bmjopen.bmj.com/lookup/pmidlookup?view=long&pmid=33741674 .bmjopen-2020-047001 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Neary M, Schueller SM. State of the field of mental health apps. Cogn Behav Pract. 2018 Nov;25(4):531–537. doi: 10.1016/j.cbpra.2018.01.002. https://europepmc.org/abstract/MED/33100810 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Silberman J, Wicks P, Patel S, Sarlati S, Park S, Korolev IO, Carl JR, Owusu JT, Mishra V, Kaur M, Willey VJ, Sucala ML, Campellone TR, Geoghegan C, Rodriguez-Chavez IR, Vandendriessche B, Evidence DEFINED Workgroup. Goldsack JC. Rigorous and rapid evidence assessment in digital health with the evidence DEFINED framework. NPJ Digit Med. 2023 May 31;6(1):101. doi: 10.1038/s41746-023-00836-5. doi: 10.1038/s41746-023-00836-5.10.1038/s41746-023-00836-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Ramos G, Ponting C, Labao JP, Sobowale K. Considerations of diversity, equity, and inclusion in mental health apps: a scoping review of evaluation frameworks. Behav Res Ther. 2021 Dec;147:103990. doi: 10.1016/j.brat.2021.103990. https://linkinghub.elsevier.com/retrieve/pii/S0005-7967(21)00189-3 .S0005-7967(21)00189-3 [DOI] [PubMed] [Google Scholar]
- 46.Linardon J, Cuijpers P, Carlbring P, Messer M, Fuller-Tyszkiewicz Matthew. The efficacy of app-supported smartphone interventions for mental health problems: a meta-analysis of randomized controlled trials. World Psychiatry. 2019 Oct;18(3):325–336. doi: 10.1002/wps.20673. https://europepmc.org/abstract/MED/31496095 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Karyotaki E, Riper H, Twisk J, Hoogendoorn A, Kleiboer A, Mira A, Mackinnon A, Meyer B, Botella C, Littlewood E, Andersson G, Christensen H, Klein JP, Schröder Johanna, Bretón-López Juana, Scheider J, Griffiths K, Farrer L, Huibers MJH, Phillips R, Gilbody S, Moritz S, Berger T, Pop V, Spek V, Cuijpers P. Efficacy of self-guided internet-based cognitive behavioral therapy in the treatment of depressive symptoms: a meta-analysis of individual participant data. JAMA Psychiatry. 2017 Apr 01;74(4):351–359. doi: 10.1001/jamapsychiatry.2017.0044. https://eprints.whiterose.ac.uk/113096/ 2604310 [DOI] [PubMed] [Google Scholar]
- 48.Pauley D, Cuijpers P, Papola D, Miguel C, Karyotaki E. Two decades of digital interventions for anxiety disorders: a systematic review and meta-analysis of treatment effectiveness. Psychol Med. 2023 Jan;53(2):567–579. doi: 10.1017/S0033291721001999.S0033291721001999 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Lehtimaki S, Martic J, Wahl B, Foster KT, Schwalbe N. Evidence on digital mental health interventions for adolescents and young people: systematic overview. JMIR Ment Health. 2021 Apr 29;8(4):e25847. doi: 10.2196/25847. https://mental.jmir.org/2021/4/e25847/ v8i4e25847 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Riadi I, Kervin L, Dhillon S, Teo K, Churchill R, Card KG, Sixsmith A, Moreno S, Fortuna KL, Torous J, Cosco TD. Digital interventions for depression and anxiety in older adults: a systematic review of randomised controlled trials. Lancet Healthy Longev. 2022 Aug;3(8):e558–e571. doi: 10.1016/S2666-7568(22)00121-0. https://linkinghub.elsevier.com/retrieve/pii/S2666-7568(22)00121-0 .S2666-7568(22)00121-0 [DOI] [PubMed] [Google Scholar]
- 51.Food and Drug Administration (FDA) FDA. Silver Spring, MD: Food and Drug Administration; [2024-04-23]. Diversity action plans to improve enrollment of participants from underrepresented populations in clinical studies guidance for industry. https://www.fda.gov/media/157635/download . [Google Scholar]
- 52.National Institute of Mental Health (NIMH) NIMH. Bethesda, MD: NIMH; [2024-04-23]. NIMH’s approach to mental health disparities research. https://www.nimh.nih.gov/about/organization/od/odwd/nimhs-approach-to-mental-health-disparities-research . [Google Scholar]
- 53.Berg S. AMA unveils playbook to speed digital health adoption. American Medical Association. 2018. [2024-04-23]. https://www.ama-assn.org/practice-management/digital/ama-unveils-playbook-speed-digital-health-adoption .
- 54.The app evaluation model. American Psychiatric Association. [2024-04-23]. https://www.psychiatry.org:443/psychiatrists/practice/mental-health-apps/the-app-evaluation-model .
- 55.Bond RR, Mulvenna MD, Potts C, O'Neill S, Ennis E, Torous J. Digital transformation of mental health services. Npj Ment Health Res. 2023 Aug 22;2(1):13. doi: 10.1038/s44184-023-00033-y. https://europepmc.org/abstract/MED/38609479 .10.1038/s44184-023-00033-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.World Health Organization (WHO) WHO. Geneva, Switzerland: WHO; 2021. [2024-04-23]. Global strategy on digital health 2020-2025. https://iris.who.int/bitstream/handle/10665/344249/9789240020924-eng.pdf . [Google Scholar]
- 57.What is human-centered design (HCD)? The Interaction Design Foundation. [2024-04-23]. https://www.interaction-design.org/literature/topics/human-centered-design .
- 58.Vial S, Boudhraâ Sana, Dumont M. Human-centered design approaches in digital mental health interventions: exploratory mapping review. JMIR Ment Health. 2022 Jun 07;9(6):e35591. doi: 10.2196/35591. https://mental.jmir.org/2022/6/e35591/ v9i6e35591 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Brewer LC, Fortuna KL, Jones C, Walker R, Hayes SN, Patten CA, Cooper LA. Back to the future: achieving health equity through health informatics and digital health. JMIR Mhealth Uhealth. 2020 Jan 14;8(1):e14512. doi: 10.2196/14512. https://mhealth.jmir.org/2020/1/e14512/ v8i1e14512 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Porche MV, Folk JB, Tolou-Shams M, Fortuna LR. Researchers' perspectives on digital mental health intervention co-design with marginalized community stakeholder youth and families. Front Psychiatry. 2022;13:867460. doi: 10.3389/fpsyt.2022.867460. https://europepmc.org/abstract/MED/35530032 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Figueroa CA, Murayama H, Amorim PC, White A, Quiterio A, Luo T, Aguilera A, Smith ADR, Lyles CR, Robinson V, von Vacano C. Applying the digital health social justice guide. Front Digit Health. 2022;4:807886. doi: 10.3389/fdgth.2022.807886. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Unertl KM, Schaefbauer CL, Campbell TR, Senteio C, Siek KA, Bakken S, Veinot TC. Integrating community-based participatory research and informatics approaches to improve the engagement and health of underserved populations. J Am Med Inform Assoc. 2016 Jan;23(1):60–73. doi: 10.1093/jamia/ocv094. https://europepmc.org/abstract/MED/26228766 .ocv094 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Israel BA, Coombe CM, Cheezum RR, Schulz AJ, McGranaghan RJ, Lichtenstein R, Reyes AG, Clement J, Burris A. Community-based participatory research: a capacity-building approach for policy advocacy aimed at eliminating health disparities. Am J Public Health. 2010 Nov;100(11):2094–102. doi: 10.2105/AJPH.2009.170506.AJPH.2009.170506 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64.Chen E, Leos C, Kowitt SD, Moracco KE. Enhancing community-based participatory research through human-centered design strategies. Health Promot Pract. 2020 Jan;21(1):37–48. doi: 10.1177/1524839919850557. [DOI] [PubMed] [Google Scholar]
- 65.Blease C, Torous J. ChatGPT and mental healthcare: balancing benefits with risks of harms. BMJ Ment Health. 2023 Nov;26(1):e300884. doi: 10.1136/bmjment-2023-300884. http://mentalhealth.bmj.com/lookup/pmidlookup?view=long&pmid=37949485 .bmjment-2023-300884 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.Omiye JA, Lester JC, Spichak S, Rotemberg V, Daneshjou R. Large language models propagate race-based medicine. NPJ Digit Med. 2023 Oct 20;6(1):195. doi: 10.1038/s41746-023-00939-z. doi: 10.1038/s41746-023-00939-z.10.1038/s41746-023-00939-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67.Timmons AC, Duong JB, Simo Fiallo N, Lee T, Vo HPQ, Ahle MW, Comer JS, Brewer LC, Frazier SL, Chaspari T. A call to action on assessing and mitigating bias in artificial intelligence applications for mental health. Perspect Psychol Sci. 2023 Sep;18(5):1062–1096. doi: 10.1177/17456916221134490. https://europepmc.org/abstract/MED/36490369 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.Chua L, Ghazi B, Huang Y, Kamath P, Kumar R, Manurangsi P, Sinha A, Xie C, Zhang C. Crosslingual capabilities and knowledge barriers in multilingual large language models. arXiv. :1. doi: 10.48550/arXiv.2406.16135. Preprint posted online on June 23, 2024. [DOI] [Google Scholar]
- 69.Shen L, Tan W, Chen S, Chen Y, Zhang J, Xu H, Zheng B, Koehn P, Khashabi D. The language barrier: dissecting safety challenges of LLMs in multilingual contexts. arXiv. :1. doi: 10.48550/arXiv.2401.13136. Preprint posted online on January 23, 2024. [DOI] [Google Scholar]
- 70.Chen I, Szolovits P, Ghassemi M. Can AI help reduce disparities in general medical and mental health care? AMA J Ethics. 2019 Feb 01;21(2):E167–179. doi: 10.1001/amajethics.2019.167. https://journalofethics.ama-assn.org/article/can-ai-help-reduce-disparities-general-medical-and-mental-health-care/2019-02 .amajethics.2019.167 [DOI] [PubMed] [Google Scholar]
- 71.Green BL, Murphy A, Robinson E. Accelerating health disparities research with artificial intelligence. Front Digit Health. 2024;6:1330160. doi: 10.3389/fdgth.2024.1330160. https://europepmc.org/abstract/MED/38322109 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 72.Abràmoff Michael D, Tarver ME, Loyo-Berrios N, Trujillo S, Char D, Obermeyer Z, Eydelman MB, Foundational Principles of Ophthalmic Imaging/Algorithmic Interpretation Working Group of the Collaborative Community for Ophthalmic Imaging Foundation‚ Washington‚ D.C. Maisel William H. Considerations for addressing bias in artificial intelligence for health equity. NPJ Digit Med. 2023 Sep 12;6(1):170. doi: 10.1038/s41746-023-00913-9. doi: 10.1038/s41746-023-00913-9.10.1038/s41746-023-00913-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 73.Meskó Bertalan, Topol EJ. The imperative for regulatory oversight of large language models (or generative AI) in healthcare. NPJ Digit Med. 2023 Jul 06;6(1):120. doi: 10.1038/s41746-023-00873-0. doi: 10.1038/s41746-023-00873-0.10.1038/s41746-023-00873-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 74.Young S. Advancing governance, innovation, and risk management for agency use of artificial intelligence. Executive Office of the President Office of Management and Budget. 2024. [2024-04-23]. https://www.whitehouse.gov/wp-content/uploads/2024/03/M-24-10-Advancing-Governance-Innovation-and-Risk-Management-for-Agency-Use-of-Artificial-Intelligence.pdf .
- 75.Nundy S, Cooper LA, Kelsay E. Employers can do more to advance health equity. Harvard Business Review. 2023. [2024-04-23]. https://hbr.org/2023/01/employers-can-do-more-to-advance-health-equity .
- 76.The Deloitte Millennial Survey 2018. Deloitte. 2018. [2024-04-23]. https://www2.deloitte.com/tr/en/pages/about-deloitte/articles/millennialsurvey-2018.html .
- 77.Eswaran V. The business case for diversity in the workplace is now overwhelming. World Economic Forum. 2019. [2024-04-23]. https://www.weforum.org/agenda/2019/04/business-case-for-diversity-in-the-workplace/
- 78.Hawke LD, Sheikhan NY, Jones N, Slade M, Soklaridis S, Wells S, Castle D. Embedding lived experience into mental health academic research organizations: critical reflections. Health Expect. 2022 Oct;25(5):2299–2305. doi: 10.1111/hex.13586. https://europepmc.org/abstract/MED/35999670 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 79.Michalak C, Jackson M. Supporting the well-being of your underrepresented employees. Harvard Business Review. 2022. [2024-04-23]. https://hbr.org/2022/03/supporting-the-well-being-of-your-underrepresented-employees .
- 80.Woebot Health Forms Clinical Diversity Advisory Board. Woebot Health. 2021. [2024-04-23]. https://woebothealth.com/woebot-health-forms-clinical-diversity-advisory-board/
- 81.Delogne MC. Woebot Health to present abstracts on diversity in digital mental health interventions at ABCT 2022. Woebot Health. 2024. [2024-04-23]. https://woebothealth.com/woebot-health-to-present-abstracts-on-diversity-in-digital-mental-health-interventions-at-abct-2022/
- 82.Hemphill C. 8 building blocks of health equity strategy. Woebot Health. [2024-04-23]. https://woebothealth.com/8-building-blocks-of-health-equity-strategy/
- 83.Making mental health a part of your DEIB strategy. Headspace. 2024. [2024-04-23]. https://get.headspace.com/guide-making-mental-health-a-part-of-your-deib-strategy .
- 84.Hemphill C. What does racism cost our healthcare system? Woebot Health. [2024-04-24]. https://woebothealth.com/what-does-racism-cost-our-healthcare-system/
- 85.NIMHD Research Diversity Supplement Application Guidelines. National Institute on Minority Health and Health Disparities. [2024-04-24]. https://www.nimhd.nih.gov/programs/extramural/training-career-dev/research-supplements/diversity-supplements.html .
- 86.Welcome to NIGMS. National Institute of General Medical Sciences. [2024-04-24]. https://nigms.nih.gov/
- 87.Empower minds. Create opportunities. REACH. [2024-04-24]. https://reachatrush.org/
- 88.Ramos G, Chavira DA. Use of technology to provide mental health care for racial and ethnic minorities: evidence, promise, and challenges. Cognitive and Behavioral Practice. 2022 Feb;29(1):15–40. doi: 10.1016/j.cbpra.2019.10.004. [DOI] [Google Scholar]
- 89.Bullard L, Rapoport S, Ariniello L, Delgado F, Waalen J, Williams A, Robinson A, Ajayi T. Utilization of community-based participatory research strategies in a postpartum digital health trial. Annual Postpartum Support International Conference; July 26-28, 2024; Washington, DC. 2024. [Google Scholar]
- 90.Joseph G, Dohan D. Diversity of participants in clinical trials in an academic medical center: the role of the 'Good Study Patient?'. Cancer. 2009 Feb 01;115(3):608–15. doi: 10.1002/cncr.24028. https://onlinelibrary.wiley.com/doi/10.1002/cncr.24028 . [DOI] [PubMed] [Google Scholar]
- 91.Wendler D, Kington R, Madans J, Van Wye Gretchen, Christ-Schmidt H, Pratt LA, Brawley OW, Gross CP, Emanuel E. Are racial and ethnic minorities less willing to participate in health research? PLoS Med. 2006 Feb;3(2):e19. doi: 10.1371/journal.pmed.0030019. https://dx.plos.org/10.1371/journal.pmed.0030019 .05-PLME-RA-0281R2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 92.Cullen MR, Lemeshow AR, Amaro S, Bandera EV, Cooper LA, Kawachi I, Lunyera J, McKinley L, Poss CS, Rottas MM, Schachterle SE, Thadeio PF, Russo LJ. A framework for setting enrollment goals to ensure participant diversity in sponsored clinical trials in the United States. Contemp Clin Trials. 2023 Jun;129:107184. doi: 10.1016/j.cct.2023.107184. https://linkinghub.elsevier.com/retrieve/pii/S1551-7144(23)00107-6 .S1551-7144(23)00107-6 [DOI] [PubMed] [Google Scholar]
- 93.Call CC, Eckstrand KL, Kasparek SW, Boness CL, Blatt L, Jamal-Orozco N, Novacek DM, Foti D, Scholars for Elevating EquityDiversity (SEED) An ethics and social-justice approach to collecting and using demographic data for psychological researchers. Perspect Psychol Sci. 2023 Sep;18(5):979–995. doi: 10.1177/17456916221137350. https://europepmc.org/abstract/MED/36459692 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 94.Boucher E, Honomichl R, Ward H, Powell T, Stoeckl SE, Parks A. The effects of a digital well-being intervention on older adults: retrospective analysis of real-world user data. JMIR Aging. 2022 Sep 02;5(3):e39851. doi: 10.2196/39851. https://aging.jmir.org/2022/3/e39851/ v5i3e39851 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 95.Pratap A, Neto EC, Snyder P, Stepnowsky C, Elhadad N, Grant D, Mohebbi MH, Mooney S, Suver C, Wilbanks J, Mangravite L, Heagerty PJ, Areán Pat, Omberg L. Indicators of retention in remote digital health studies: a cross-study evaluation of 100,000 participants. NPJ Digit Med. 2020;3:21. doi: 10.1038/s41746-020-0224-8. doi: 10.1038/s41746-020-0224-8.224 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 96.Placzek M, Friede T. Clinical trials with nested subgroups: analysis, sample size determination and internal pilot studies. Stat Methods Med Res. 2018 Nov;27(11):3286–3303. doi: 10.1177/0962280217696116. [DOI] [PubMed] [Google Scholar]
- 97.McCall T, Asuzu K, Oladele CR, Leung TI, Wang KH. A socio-ecological approach to addressing digital redlining in the United States: a call to action for health equity. Front Digit Health. 2022;4:897250. doi: 10.3389/fdgth.2022.897250. https://europepmc.org/abstract/MED/35924138 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 98.Egede LE, Ruiz R, Mosley-Johnson E, Aguilar-Gaxiola SA, Corbie GM, Wilkins CH, Vitale A, Boulware LE. Laying the groundwork to make diversity, equity, and inclusion front and center in clinical and translational research. J Clin Transl Sci. 2023;7(1):e95. doi: 10.1017/cts.2023.524. https://europepmc.org/abstract/MED/37125065 .S2059866123005241 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 99.Martinez-Martin N. Envisioning a path toward equitable and effective digital mental health. AJOB Neurosci. 2022;13(3):196–198. doi: 10.1080/21507740.2022.2082597. https://europepmc.org/abstract/MED/35797130 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 100.Taher R, Hsu C, Hampshire C, Fialho C, Heaysman C, Stahl D, Shergill S, Yiend J. The safety of digital mental health interventions: systematic review and recommendations. JMIR Ment Health. 2023 Oct 09;10:e47433. doi: 10.2196/47433. https://mental.jmir.org/2023//e47433/ v10i1e47433 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 101.Health equity. American Psychological Association. [2024-04-24]. https://www.apaservices.org/advocacy/issues/health-equity .
- 102.Behavioral health equity. SAMHSA - Substance Abuse and Mental Health Services. [2024-04-24]. https://www.samhsa.gov/behavioral-health-equity .
- 103.Advocate for change. NAMI. [2024-04-24]. https://www.nami.org/advocacy/advocate-for-change/
- 104.Advocacy network. Mental Health America. [2024-04-24]. https://mhanational.org/issues/advocacy-network .
- 105.The mental health parity and addiction equity act (MHPAEA) Centers for Medicare & Medicaid Services. 2013. [2024-04-24]. https://www.cms.gov/marketplace/private-health-insurance/mental-health-parity-addiction-equity .
- 106.Cohen ES, editor. Broadband internet: access, regulation, and policy. New York, NY: Nova Science Publishers; 2008. [Google Scholar]
- 107.Wu T. Network neutrality, broadband discrimination. SSRN Journal. 2003;2:141. doi: 10.2139/ssrn.388863. [DOI] [Google Scholar]
- 108.Rolan G, Denison T, Mackenzie C. "Broadband, digital literacy and public libraries: the Mill Park story". Library Hi Tech News. 2015;6:15–18. doi: 10.1108/lhtn-03-2015-0020. [DOI] [Google Scholar]
- 109.Clinical trial diversity. FDA. [2024-04-24]. https://www.fda.gov/consumers/minority-health-and-health-equity/clinical-trial-diversity .
- 110.Spinner J, Araojo RR. FDA's strategies to close the health equity gap among diverse populations. J Prim Care Community Health. 2021;12:21501327211000232. doi: 10.1177/21501327211000232. https://journals.sagepub.com/doi/abs/10.1177/21501327211000232?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub0pubmed . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 111.Grabman G, Tenenbaum C. FDA regulation must uphold women's health. Food & Drug Law Institute (FDLI) 2022. [2024-08-30]. https://www.fdli.org/wp-content/uploads/2022/12/Grabman-Tenenbaum-FDLJ-77-3.pdf .
- 112.Food and Drug Administration (FDA) Enforcement policy for digital health devices for treating psychiatric disorders during the Coronavirus Disease 2019 (COVID-19): public health emergency guidance for industry and Food and Drug Administration staff. FDA. 2020. [2024-04-24]. https://www.regulations.gov/document/FDA-2020-D-1138-0068 .
- 113.PAR-22-154: Laboratories to optimize digital health. National Institutes of Mental Health. 2023. [2024-04-24]. https://grants.nih.gov/grants/guide/pa-files/PAR-22-154.html .
- 114.Kessler RC, Wang PS. The descriptive epidemiology of commonly occurring mental disorders in the United States. Annu Rev Public Health. 2008;29:115–29. doi: 10.1146/annurev.publhealth.29.020907.090847. [DOI] [PubMed] [Google Scholar]
- 115.Hossain MM, Tasnim S, Sultana A, Faizah F, Mazumder H, Zou L, McKyer ELJ, Ahmed HU, Ma P. Epidemiology of mental health problems in COVID-19: a review. F1000Res. 2020;9:636. doi: 10.12688/f1000research.24457.1. https://europepmc.org/abstract/MED/33093946 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 116.Ohlsson H, Kendler KS. Applying causal inference methods in psychiatric epidemiology: a review. JAMA Psychiatry. 2020 Jun 01;77(6):637–644. doi: 10.1001/jamapsychiatry.2019.3758. https://europepmc.org/abstract/MED/31825494 .2757020 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 117.Meet New Profit’s 2023 mental health equity catalyze cohort. New Profit. 2023. May 22, [2024-04-24]. https://www.newprofit.org/go/meet-new-profits-2023-mental-health-equity-catalyze-cohort/
- 118.Galea S. The price of health equity. JAMA Health Forum. 2021 Apr 01;2(4):e210720. doi: 10.1001/jamahealthforum.2021.0720. https://jamanetwork.com/article.aspx?doi=10.1001/jamahealthforum.2021.0720 .2778583 [DOI] [PubMed] [Google Scholar]
- 119.Knifton L, Inglis G. Poverty and mental health: policy, practice and research implications. BJPsych Bull. 2020 Oct;44(5):193–196. doi: 10.1192/bjb.2020.78. https://europepmc.org/abstract/MED/32744210 .S2056469420000789 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 120.Soliman W. ACMA. 2021. [2024-04-24]. https://medicalaffairsspecialist.org/blog/diversity-inclusion-clinical-trials-medical-affairs .
- 121.DeSalvo KB, Wang YC, Harris A, Auerbach J, Koo D, O'Carroll Patrick. Public Health 3.0: a call to action for public health to meet the challenges of the 21st century. Prev Chronic Dis. 2017 Sep 07;14:E78. doi: 10.5888/pcd14.170017. https://europepmc.org/abstract/MED/28880837 .E78 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 122.Dwyer M. Nonprofit leaders see potential to improve health equity through cross-sector partnerships. Pew. 2022. [2024-04-24]. https://pew.org/3zMdVT3 .
- 123.Bauer MS, Damschroder L, Hagedorn H, Smith J, Kilbourne AM. An introduction to implementation science for the non-specialist. BMC Psychol. 2015 Sep 16;3(1):32. doi: 10.1186/s40359-015-0089-9. https://bmcpsychology.biomedcentral.com/articles/10.1186/s40359-015-0089-9 .10.1186/s40359-015-0089-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 124.Efforts to close the digital health equity divide may improve access to healthcare. ElevanceHealth. 2023. [2024-04-24]. https://www.elevancehealth.com/our-approach-to-health/health-equity/efforts-to-close-the-digital-health-equity-divide .
- 125.Mordecai D, Histon T, Neuwirth E, Heisler WS, Kraft A, Bang Y, Franchino K, Taillac C, Nixon JP. How Kaiser Permanente Created a Mental Health and Wellness Digital Ecosystem. NEJM Catalyst. 2021 Jan;2(1) doi: 10.1056/cat.20.0295. https://catalyst.nejm.org/doi/full/10.1056/CAT.20.0295 . [DOI] [Google Scholar]
- 126.Youn SJ, Jaso B, Eyllon M, Sah P, Hoyler G, Barnes JB, Jarama K, Murillo L, O'Dea H, Orth L, Pennine M, Rogers E, Welch G, Nordberg SS. Leveraging implementation science to integrate digital mental health interventions as part of routine care in a practice research network. Adm Policy Ment Health. 2024 May;51(3):348–357. doi: 10.1007/s10488-023-01292-9.10.1007/s10488-023-01292-9 [DOI] [PubMed] [Google Scholar]
- 127.CMS framework for health equity. Centers for Medicare & Medicaid Services. [2024-04-24]. https://www.cms.gov/priorities/health-equity/minority-health/equity-programs/framework .
- 128.Chen K, Lane E, Burns J, Macrynikola N, Chang S, Torous J. The digital navigator: standardizing human technology support in app-integrated clinical care. Telemed J E Health. 2024 Jun;30(7):e1963–e1970. doi: 10.1089/tmj.2024.0023. [DOI] [PubMed] [Google Scholar]
- 129.Nicholas J, Ringland KE, Graham AK, Knapp AA, Lattie EG, Kwasny MJ, Mohr DC. Stepping up: predictors of 'stepping' within an iCBT stepped-care intervention for depression. Int J Environ Res Public Health. 2019 Nov 25;16(23):4689. doi: 10.3390/ijerph16234689. https://www.mdpi.com/resolver?pii=ijerph16234689 .ijerph16234689 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 130.Nordgreen T, Haug T, Öst Lars-Göran, Andersson G, Carlbring P, Kvale G, Tangen T, Heiervang E, Havik OE. Stepped care versus direct face-to-face cognitive behavior therapy for social anxiety disorder and panic disorder: a randomized effectiveness trial. Behav Ther. 2016 Mar;47(2):166–83. doi: 10.1016/j.beth.2015.10.004.S0005-7894(15)00114-8 [DOI] [PubMed] [Google Scholar]
- 131.Adewuya AO, Momodu O, Olibamoyo O, Adegbaju A, Adesoji O, Adegbokun A. The effectiveness and acceptability of mobile telephone adherence support for management of depression in the Mental Health in Primary Care (MeHPriC) project, Lagos, Nigeria: A pilot cluster randomised controlled trial. J Affect Disord. 2019 Jun 15;253:118–125. doi: 10.1016/j.jad.2019.04.025.S0165-0327(18)32942-2 [DOI] [PubMed] [Google Scholar]
- 132.Jonassaint CR, Belnap BH, Huang Y, Karp JF, Abebe KZ, Rollman BL. Racial differences in the effectiveness of internet-delivered mental health care. J Gen Intern Med. 2020 Feb;35(2):490–497. doi: 10.1007/s11606-019-05542-1. https://europepmc.org/abstract/MED/31745855 .10.1007/s11606-019-05542-1 [DOI] [PMC free article] [PubMed] [Google Scholar]