Table.

Example studies addressing digital health equity

Study	Study setting	Primary objective of digital platform/ intervention	Study design / outcome (s)	Key findings/lessons for digital health equity
1. Interventions/Programs that employ digital health co-design to advance equity in usability, uptake, and/or effectiveness of digital health platforms
Papoutsi et al., 2021	Nationwide in the U.K.	Compare co-design in 3 case studies	Workshops with patients as well as providers to obtain feedback on tools. Pilot testing.	If co-design focuses narrowly on the technology, opportunities will be missed to coevolve technologies alongside clinical practices and organization routines.
Brewer et al., 2020	African American churches, Rochester and Minneapolis-St. Paul, MN, U.S.	Engage the community to develop a general health app	CBPR: Community members involved in the mixed methods study design to incorporate community members in intervention development. FAITH! Partners designated to refine recruitment, implementation, and results dissemination	Leveraged established stakeholders and trusted social networks Focused on understanding the social context of potential end users Integrated community engagement through user-centered design or participatory design Gain an understanding of community partner technology infrastructure
Buman et al., 2013	Three senior, low-income housing sites, South San Francisco, Menlo Park, San Mateo, CA, U.S.	Develop and evaluate the utility of a computerized, tablet-based participatory tool designed to engage older residents in identifying neighborhood elements that affect active living opportunities	Formative testing. Participants used tool to record common walking routes and geocoded audio narratives and photographs of the local neighborhood environment while navigating their usual walking route	Tool was found to complement other assessments and can assist decision makers in consensus-building processes for environmental change
Jackson et al., 2022	Prince George’s and Montgomery Counties, Maryland, U.S.	Design a prevention-focused, personalized mHealth, information-seeking smartphone app that is culturally appropriate and acceptable	1-year, multi-method participatory research process that engaged English-speaking African American and bilingual or Spanish-speaking Hispanic adults	Community partnerships provided the chain of trust that help Black, Indigenous, and people of color (BIPOC) participants feel comfortable participating in app research. Community-based participatory research principles yielded promising results to engage these populations in digital health research Interactive design sessions uncovered participants’ needs and development opportunities for digital health tools. Multiple design sessions with different methods provided an in-depth understanding of participants’ preferences and needs.
Avila et al., 2019	Safety-net hospital, San Francisco, CA, U.S.	Inform development of text-messaging intervention to encourage physical activity	Focus group and individual interviews with English- and Spanish-speaking patients to integrate user feedback into intervention design	Key barriers to use (pain and depression) were identified and addressed in intervention, alongside technical requirements
Nouri et al., 2019	Public sector urban primary care clinics, San Francisco, CA, U.S.	Assess relevance of user-centered methods for diverse patient pool	Semi-structured interviews, coding, and card sorting	Engagement in design methods varied by digital and health literacy Augmentation of card sorting with direct observation and audiovisual cues may be more productive in eliciting feedback for those with communication barriers
Unertl et al., 2016	US and Canada. Projects using CBPR	Case analysis of 5 studies implementing community-based participatory research (CBPR) in health informatics work	Examined each case individually for success factors and barriers, and identified common patterns across cases	CBPR projects resulted in more relevant products that match community need. Challenges exist including longer time frame and mismatch in style and culture.
2. Example studies that provide individual-level digital literacy support or training as a core program component
Hoffman et al., 2020	Boston, MA, U.S.	Identify digital resources for hospitalized patients with serious mental illness to increase wellness, make informed decisions about apps, and use apps and data for behavior change	Exploratory group evaluation of apps culminating into the development of two training manuals	Wide range of starting digital skills. Group training requires flexibility to meet participants at current digital literacy level Training can increase perception on importance of using digital tools to access health information and confidence in finding health information online
Lyles et al., 2019	Safety-net clinics, San Francisco, CA, U.S.	Increase patient portal enrollment	Pre- and post-evaluation of effectiveness of in-person training vs web-based videos about patient portal navigation	Both in-person and web-based videos were better than no training However nearly 80% did not log-in after training, suggesting need for very intense training or significant improvement in usability of patient portals
Watkins and Xie, 2014	N/A (review article)	Studies aiming to increase eHealth literacy	Systematic review. Collaborative learning and tailored content developed based on NIH materials (both in-person and web-based)	Few evaluations of health outcomes Few theory-based interventions Few experimental study designs
Lee et al., 2014	N/A (review article)	Studies aiming to increase ability to find reliable health information (workshops most common approach)	Review of in-person and web-based trainings, both group and individual sessions	Overall increases in self-reported knowledge and/or skills
Stein et al., 2018	King County, WA, U.S.	Evaluate an intervention that teaches hospitalized patients at a safety net hospital how to access and use their EHR online portal	RCT of in-person patient portal education during admission involving registration, login, navigating website, and reviewing discharge summary	Education/training was effective at increasing portal use
Fields et al., 2020	San Francisco, CA, U.S.	To assess barriers and facilitators to technology training implementation	Pilot involving community-based organization to evaluate the impact of technology training on older adults’ loneliness, social support, and technology use in real-world settings.	Embedding training within existing community-based programs holds promise as a potentially sustainable mechanism to provide digital training to isolated older adults.
National Health Services, Widening Digital Participation, 2017	England, U.K.	3-year program provided seed funding to (a) establish 20+ projects focused on specific patient populations in each community; (b) create portal to promote digital skills (Learn My Way) and use of digital tools to promote health (Staying Healthy); (c) support “pre-digital” skills	Help those without digital skills to access health information and support online. The program aimed to put the individual in charge of their health, with a long-term aim to also relieve pressure on frontline health services.	Projects reached 285,000+ people and specifically 53,000+ improved their digital literacy 83% that used the digital skills portal reported more confident about using online tools to manage their health; 33% who completed training reported fewer primary care visits Governments can support widescale dissemination and innovation through establishing standard curriculum and collaborating with community-based organizations to reduce overall costs
3. Example digital health studies leveraging community/social relationships to support use
Nguyen et al., 2021	Community-based organizations, San Francisco, CA, U.S.	Development of a digital health search/referral platform to connect community members with resources in their neighborhoods	Qualitative study focused on user needs and requirements for a digital health platform	Community organizations essential for tangible resource connections as well as overall social support and trust Digital platforms must enhance existing human knowledge of community assets/needs
Heisler et al., 2019	Urban VA clinic, Detroit, MI, U.S.	Peer coaching intervention for diabetes management, with vs without digital tool	RCT with glycemic control as primary outcome measure Assessments at baseline, 6 months, 12 months	Peer coaching critical for improving outcomes, including detailed documentation of coaching implementation Digitally-enabled coaching may need longer-term use in future studies
Roddy et al., 2022	Urban academic medical center and community clinics, Nashville, TN, U.S.	Text messaging intervention to support diabetes management, with specific sub-intervention that engaged family/adult supporters	Secondary analysis of RCT, with glycemic control as the primary outcome measure	Family/adult support mediated the improvements in diabetes control post-intervention Future studies needed to tease apart these influences
Holt et al., 2018	African American churches in Metro DC area, U.S.	Evaluate and compare web-based vs. in-person peer coaching for preventive health behaviors	RCT evaluating cancer-related knowledge and screening behaviors between groups	No significant differences in cancer knowledge or screening rates at 24 months by group Web-based coaching as effective as in-person, but process measures by arm not reported
Handley et al., 2021	Safety-net settings, San Francisco Bay area	Examine fidelity and acceptability of coaching intervention by language of participants (English vs Spanish speakers)	Secondary analysis of RCT data to determine how in-person and digital support might have varied by participant demographics	High overall engagement and acceptability of coaching in the study, with no differences in modality by participant language. Critical to have clear analytic plan to examine implementation outcomes based on modality of coaching (digital vs. in-person) and key participant demographics
4. Example studies focusing on systems-level implementation of digital interventions or programs, specifically within safety net settings
Watkinson et al., 2021	U.K., England	Measure system-level acceptance of Health Information Exchange (HIE) and understand barriers/facilitators to adoption	Mixed-methods study to examine differences in acceptance between user groups and care settings	Social care users had lower acceptance and adoption. They also lacked resources to properly use HIE system
Peynetti Velázquez et al., 2020	Boston, MA, U.S. safety net system	Rapid implementation of telepsychiatry to meet care needs for diverse patient population	Implementation study outlining change management processes used to implement services	Multiple departments engaged to create patient-focused implementation Core domains focused on during the intervention included people, process, technology, monitoring, environment, and equity
California Healthcare Foundation innovation fund, 2011	California, U.S.	Philanthropic investment program to specifically fund private sector digital health companies with potential to improve care quality for Medicaid patients	By 2021, the fund’s portfolio served over 5M Medi-Cal enrollees at over 250 hospitals & 100 clinics in California. Portfolio companies experienced an average 115% annual growth the market opportunity in California	Vital and feasible to use capital investment for supporting private companies working in the Medicaid market
Barnett et al., 2017	Los Angeles safety-net hospital	Decrease wait time to see a specialist	Examine growth, usage, and outcomes of eConsult system implementation	Rapid growth in eConsult use. Decreased wait times to see a specialist Health systems and plans partnered to solve a high-priority problem, achieving implementation prior to more well-resourced settings
Aulakh and Maguire, 2021	Safety-net systems nationwide	Provide guidance for safety-net leaders and providers to improve digital healthcare services via innovation approaches	Collaboratives and peer learning for innovation is successful strategy for innovation in safety net settings	Shared learning models across sites and settings (healthcare, community-based organizations) can increase impact Phased implementation and support (technical assistance, training, networking) are critical
Lyles et al., 2014	Safety-net systems in California	Qualitative study of safety net leaders, focusing on drivers of innovation implementation	Examples of successful innovations alongside unique contexts for implementation	Safety net leaders emphasize their approaches to centering equity and addressing highest priority topics, rather than supporting too many programs/pilots
5. Example studies of programs that addressed structural barriers to digital health interventions, such as broadband access or devices
Gujral et al., 2022	Rural U.S.	Assess association between increased distribution of tablets during COVID-19 pandemic and mental health service use and related outcomes	Retrospective cohort study Loaned iPads to rural U.S. Veterans from March 2020-April 2021. Compared outcomes 10 months before tablet receipt and 10 months after tablets to controls	Increased mental health care use, reduced suicidal behavior and ED visits For rural veterans already engaged in mental health care, tablets can help increase access to mental health services
Zulman et al., 2019 Jacobs et al., 2019 Slightam et al., 2020	U.S., nationwide in the VA	Evaluate implementation of tablet distribution to high-need veterans with health care access barriers	Retrospective cohort study 2016 VA pilot distributed video-enabled tablets with 4G wireless or Wi-Fi to veterans with access barriers. Evaluated outcomes of tablet adoption and reach, and barriers and facilitators of tablet use for telehealth.	Zulman et al, 2019: 80% of patients who received tablets used them; those who were older and who had fewer chronic conditions were less likely to use. Facility-level barriers to implementing tablet program included staffing shortages and lack of staff training Slightam et al, 2020: lack of digital skills and poor Internet connection associated with lower preference for video visits Jacobs et al, 2019: time and money savings for those who live far away from VA, have travel barriers, do not have mental health diagnosis
Whealin et al., 2017	Rural areas of the Pacific Islands	Evaluate veterans’ perceptions of home therapy for PTSD through video-enabled tablets	VA pilot of tablets and secure WiFi for home treatment of PTSD Pre- and post- engagement questionnaire	Feasible to use tablets to deliver treatment to rural veterans of racial/ethnic minority ancestry; Some patients still had privacy and connectivity concerns
Davis et al., 2016	U.S.	Understand technical needs to support veterans after distribution of tablets for telemental health services	Assess workload and productivity of Peer Technical Consultant (PTC) in providing technical support to veterans Survey veterans and providers during and after telemental health program treatment about role of PTC	For veterans with diverse digital literacy skills and mental health care needs, robust technical support is needed for successful use of devices and telehealth technologies. PTC should be a full-time contract-employee to increase availability of technical support
Schueller et al., 2019	Homeless shelter network located in Chicago, IL, U.S.	Evaluate feasibility and acceptability of remotely delivered mental health intervention with brief emotional support and coping skills among young adults experiencing homelessness	Single-arm feasibility pilot trial, pre-post intervention evaluation Participants received mobile phone, service/ data plan, 1 month of coaching Assess session and program completion, and acceptability based on satisfaction ratings	High rates of program completion and satisfaction among participants Little change on pre-post measures of depression, PTSD, emotion regulation Feasible and acceptable to provide technology-based mental health services to young adults experiencing homelessness
Kazevman et al., 2021	Ontario, Canada	Improve access to primary care, adherence to public health directives and adherence to self-isolation guidelines during COVID-19 pandemic	Provide free donated and prepaid cell phones to patients without a listed phone number who presented for care at emergency department during COVID-19 pandemic Protocol paper for pilot mixed-methods study, no data reported	Examination of program on health outcomes underway Approach attempts to improve patient access to health care, information, and social services
Moczygemba et al., 2021	Austin, TX, U.S.	Assess the accuracy, acceptability, and outcomes of a GPS-mHealth intervention to alert community health paramedics when people experiencing homelessness were in emergency department or hospital	Pre-post design with assessments at baseline, 1 month, 2 months, 3 months, and 4 months post-enrollment	Limited accuracy for ED/hospital alerts Decrease in depression symptoms, improved medication adherence Cell phone provision can help individuals with complex medical needs and experiencing homelessness improve medication adherence and maintain contact with social support networks
LAUNCH program (FCC-NCI), 2020	Appalachia (Rural U.S.)	Better understand landscape of connected cancer care management in rural America	Framework and proposed program to improve telehealth-enabled cancer care for rural America	Need for community-based participatory approach for human-centered design of digital cancer care