Abstract
Background
Behavioral digital health interventions (e.g., mobile apps, websites, wearables) have been applied widely to improve health outcomes. However, many groups (e.g., people with low income levels, people who are geographically isolated, older adults) may face obstacles to technology access and use. In addition, research has found that biases and stereotypes can be embedded within digital health interventions. As such, behavioral digital health interventions that intend to improve overall population health may unintentionally widen health-related inequities.
Purpose
This commentary offers guidance and strategies to mitigate these risks when using technology as a means for delivering a behavioral health intervention.
Methods
A collaborative working group from Society of Behavioral Medicine’s Health Equity Special Interest Group developed a framework to center equity in the development, testing and dissemination of behavioral digital health interventions.
Results
We introduce Partner, Identify, Demonstrate, Access, Report (PIDAR), a 5-point framework to avoid the creation, perpetuation, and/or widening of health inequities in behavioral digital health work.
Conclusions
It is critically important to prioritize equity when conducting digital health research. The PIDAR framework can serve as a guide for behavioral scientists, clinicians and developers.
Keywords: Digital health, Behavioral health, Health equity, Framework
Implications.
Practice: Healthcare professional should prioritize equity when disseminating digital health interventions
Policy: Research and empirical evidence should be used to support policies that promote digital equity and inclusion.
Research: Behavioral researchers should center equity when designing and testing digital health interventions.
Introduction
Behavioral digital health interventions—that is, behavioral interventions delivered electronically (e.g., apps, websites, wearables) - have been applied widely to improve health outcomes [1–7]. The COVID-19 pandemic has underscored the critical role of technology in healthcare [8]. In the context of the pandemic, behavioral digital health interventions offer individuals the opportunity to access interventions remotely, while minimizing the risk of COVID-19 spread. Throughout the pandemic, technology has been used to improve access to health-enhancing resources such as healthy food, education, and social connections [9, 10]. Although the pandemic has cast a spotlight on the importance of behavioral digital health interventions, the potential impact and proliferation of these interventions extend far beyond the context of the pandemic.
The executive summary of the WHO Guideline Recommendations on Digital Interventions stated that “Amid the heightened interest, digital health has also been characterized by implementations rolled out in the absence of a careful examination of the evidence base on benefits and harms” [11]. Regarding potential harms, it is critical to evaluate whether certain groups are excluded from the potential benefits of behavioral digital health interventions.
A substantial body of literature has documented inequities in Interent access and device ownership [12–16]. The first level “digital divide,” is defined as “the economic, education, and social inequalities between those who have computers and online access and those who do not” [17]. Over time, the digital divide has lessened, with more individuals owning a Smartphone or having access to the Internet [14], however inequities remain. For example, approximately 96% of people with an income at or above $75,000 per year and 93% people with a college education or more own smartphones, while only 76% of people earning less than $30,000 per year and 75% of people with less than a high school education own smartphones [18]. In addition, a longitudinal study found that 65% of participants who did not have Internet connection reported that cost was a barrier [19]. Research also suggests that individuals aged 65 and older are consistently the least likely to have home broadband connectivity [14]. In addition, a recent study found that approximately one-quarter of individuals living in a rural area reported that accessing high-speed Internet is a major problem [20]. There are also considerations around the quality of digital access. For example, data on smartphone owners from 2014 suggests that Non-Hispanic Black people (42%) and Hispanic people (36%) are more likely than Non-Hispanic white people (17%) to have temporarily canceled or cut-off cell phone service due to financial constraints [21]. Furthermore, the device used (e.g., smartphone vs. desktop computer) can impact the quality of Internet access. There are notable inequities in who has access or uses a desktop computer or laptop vs. those who are “smartphone dependent” and are only able to access to the Internet through their smartphones [15].
Having access to devices and broadband connectivity is often necessary yet insufficient. There is also a substantial 2nd-level digital divide in the usage, and skills needed to use the Internet [22–25]. Low digital skills, limited health literacy, privacy concerns, lack of digital readiness and mistrust of information can all serve as significant digital barriers [23, 26–28]. Due to the 1st and 2nd level digital divide, behavioral digital health interventions may be more likely to be adopted by those with high levels of digital access, readiness and skills and may exclude those who have digital barriers [29, 30].
Beyond the digital divide, behavioral health interventions may also negatively impact health inequities through biases and stereotypes incorporated into technical design and implementation of these interventions. Behavioral digital health interventions, like any other technology, are subject to the implicit and explicit biases of their human designers and developers [31, 32]. Biased interventions have the potential for serious consequences on health outcomes. For example, the potential harmful impact of biases in digital health was highlighted in a recent study that found that a widely used healthcare algorithm underestimated Black patients’ health status, which consequently impacted Black patients’ receipt of healthcare services [32]. In addition, some digital health interventions may unintentionally create or perpetuate harmful stereotypes and biased norms. For example, a qualitative study reported that women’s health apps often portrayed biased standards for health (e.g., centered around whiteness, fertility, thinness) [33].
Biases may also impact the dissemination of digital health interventions. For instance, negative attitudes about age and technology that health care providers have [34] could impair their provider’s ability to recommend digital interventions to older patients. These biases may directly and indirectly worsen health inequities.
In sum, despite the promise of behavioral digital health interventions, and their widespread dissemination, there is the potential for these interventions to unintentionally create and exacerbate health inequities, particularly in the context of the digital divide and technology bias. A working subgroup of behavioral scientists from Society of Behavioral Medicine’s Health Equity Special Interest Group created an actionable framework that outlines guidance and strategies to mitigate and avoid these risks. We offer the following framework: PARTNER, IDENTIFY, DEMONSTRATE, ACCESS, REPORT (PIDAR) which is intended to guide researchers, clinicians, and developers who design, test, and/or disseminate behavioral digital health interventions.
Recommendations
PIDAR: Five-point framework with recommendations for behavioral digital health research
(PIDAR) See Table 1.
Table 1.
Partner | Partner with a diverse group of key stakeholders and potential users |
Identify | Identify the potential impact of the digital health intervention on health inequities |
Demonstrate | Demonstrate how to use technology |
Access | Improve access to technology |
Report | Report the impact of the research on health equity |
Point 1: PARTNER with key stakeholders who are potential end-users.
Partnering with groups of patients, community members, key stakeholders, and potential end-users will better ensure that the digital health interventions are culturally, linguistically, and literacy appropriate. Diverse perspectives and partnerships can help ensure that the interventions are unbiased and inclusive. These partnerships should span across all stages of the research process, including identifying/refining the research question, designing the intervention, developing intervention and evaluation materials, testing the efficacy, and disseminating/scaling the intervention. We recommend drawing techniques from both Human-Centered Design (HCD) [35] and Community Based Participatory Research (CBPR) [36, 37]. Chen and colleagues highlight that HCD and CBPR bring together overlapping learner-centric approaches, and suggest integrating the two complementary approaches (e.g., center on empathy, establish transdisciplinary research teams, conduct rapid prototyping) [38].
When implementing CBPR and HCD, participatory approaches (e.g., low-fidelity prototyping) [39] can be used to gain feedback on interventions as they are conceptualized and designed [40]. Once a functional prototype has been developed, iterative user testing is recommended with a sample of end-users. Both qualitative and quantitative approaches can help ensure that the intervention is acceptable, understandable, feasible, user-friendly, and accessible to the intended range of end-users. It is also important that design processes take into account people’s cognitive and physical abilities [41–43]. Importantly, the inclusion of experts, community partners, end-users, and key stakeholders in mixed methods research helps enrich the science and promote equity.
Point 2: IDENTIFY the potential impact of digital health interventions on health inequities.
At the outset of conducting digital health research, we recommend research teams brainstorm potential positive as well as negative impacts the digital health interventions could have on impacted groups, particularly those traditionally disadvantaged by the digital divide and impacted by biases/discrimination. The investigative team (including community partners and/or end users) should evaluate whether access, adoption, and impact of the intervention will be equitable across all possible end-users. Examples of questions that can facilitate this process are provided in Table 2. To avoid unintentionally widening health inequities, researchers should design interventions that have maximum reach and inclusion.
Table 2.
1. Does the intervention address the real needs of all end-users? |
2. Do all end-users have equitable access to the technology needed to use the intervention? If not, what national and regional resources are available to help the end-user gain access? |
3. Does the intervention require a high level of general, health, or digital literacy? |
4. Can end-users with varying levels of English proficiency use the intervention? Is the intervention available in different languages to accommodate the needs of non-English speaking end-users? |
5. Does accessing or using the intervention place a financial burden on any end-user (e.g., cost of the intervention, broadband connectivity, data plans, technology device)? |
6. Do the intended dissemination partners (e.g., clinics, hospitals, healthcare providers) have the infrastructure and funding needed to implement, run, and maintain the intervention? |
7. Can individuals with varying physical and cognitive limitations access and/or use the intervention? |
8. Does the intervention content (e.g., text, video, graphics) represent or depict a diverse group of people or the intended end-users? Does the intervention content contain biases or stereotypes? |
9. Does any subgroup of the intended end-users stand to benefit less/more from the intervention? |
While it is important to consider the potential risks of the intervention at the outset of the research, these questions should be asked, and potential consequences should be evaluated throughout the research process. As such, we recommend conducting periodic evaluations of the potential harms/unintended consequences of the behavioral digital health intervention. In particular, we suggest that researchers continually examine whether outcomes of interest (e.g., intervention’s use, adoption and impact) are equitable across various end-users. To that end, researchers and funders should ensure that sample sizes of impacted communities are large enough to make subgroup analyses possible. In addition, we recommend identifying potential biases and stereotypes that may be incorporated within the technology. These reviews should be conducted by individuals within the research team (including end-user stakeholders) as well as external reviewers (e.g., data safety monitoring boards, institutional review boards, end-users) in order to minimize biases.
Point 3: DEMONSTRATE how to use digital health interventions.
It is critical to consider end-users’ technological abilities, skills, and comfort. We recommend that researchers provide end-users with resources to help support and improve patients’ digital literacy, as defined as “the ability to use information and communication technologies to find, evaluate, create, and communicate information, requiring both cognitive and technical skills” [44]. According to a recent survey, most adults believe public libraries should help teach community members how to use digital technologies including apps, computers, and smartphones [45]. Where such services and resources are offered, researchers can take advantage of well-established programs (e.g., citizen science programs by the National Library of Medicine) [46] designed to improve people’s ability to use technology. It is important to note that basic literacy and digital literacy are interdependent constructs, and that both require careful consideration and attention [47].
Of note, digital literacy training programs may not always be available or feasible. As such we also recommend that researchers provide training and/or demonstrations for how to use the digital health interventions that are under investigation. These trainings can be in the form of one-on-one instruction, group/family instruction, peer-based learning, demonstration videos, embedded help in a system, and/or clear graphic or written instructions. In addition, it can be helpful to provide as-needed technical assistance to help users navigate the digital health intervention. Researchers should ensure that trainings occur in the modality and days of the week/time of day preferred by the groups they are aiming to represent and serve.
Point 4: Improve ACCESS to technology.
More focused efforts are needed to ensure that groups who would benefit from digital health interventions will have equitable access to them. Many studies with digital interventions specify having access to device or Internet as a study eligibility criterion. However, this should not be the default option and solutions should be sought to maximize inclusion. In the short term, researchers could provide the participants with the devices (i.e., smartphones, tablets, wearables) needed to engage in the digital health intervention. Importantly, what happens after the end of the project should also be considered. In some cases, providing devices to all participants may not be feasible or scalable. One approach is to work with existing institutions that provide free Internet access and/or free/low-cost devices, such as public libraries, nonprofit organizations, and government programs. Digital navigators can be help connect patients to available local and national resources. An alternative option might be to provide a physical point of access in the form of free-standing technology devices or kiosks, which have the advantage of being able to be placed in targeted venues for optimal reach to underserved populations [48]. To increase access, when possible interventions should be designed to be agonistic to device and operating system to maximize accessibility.
It is also important to ensure that the healthcare systems and hospitals have equitable access to the technology, infrastructure and resources needed to successfully disseminate and maintain the intervention. When digital health interventions are integrated into a healthcare setting (e.g., hospital, clinic, federally qualified health center), researchers should help develop infrastructure (e.g., electronic medical record system integration) for clinics to easily integrate the interventions into existing workflows.
In addition, it is important to consider the costs that may be associated with adopting the behavioral digital health interventions. For example, there may be costs associated with using the interventions (e.g., data usage/plans). Researchers should consider features that would minimize costs to the end-users. In addition, when designing a digital health intervention, researchers should budget for features that can improve access and accessibility for a broad range of end users (e.g., translation services, text readers, video content).
It is necessary for researchers to prioritize ensuring equitable access of the digital health intervention for all potential end-users, including patients, community members, providers, and health systems/clinics. Policy changes that would expand access to broadband internet are vital to ensuring access to digital resources that may improve health outcomes, expand access to education, and allow community building [9], including behavioral interventions. Researchers can play a critical role in applying empirical evidence to advocate for policy change to promote equity.
Point 5: REPORT the impact of the research on health equity.
It is important that researchers report any unintended consequences to both scientific and public audiences. Without proper and transparent reporting, these unintended consequences are bound to be repeated. Funders and journal editors should explicitly solicit information about these unintended consequences in progress reports and publications, respectively. In addition, national societies and associations could host panels and workshops focused on this topic to allow further discussion about unintended consequences and how to overcome them. Reporting these unintended consequences, both within and across studies, allows the opportunity to support exchanges between researchers where lessons learned and potential solutions can be shared. This reporting can also be used to identify areas/topics of future research and funding priorities.
Discussion
The recommendations outlined in this commentary should serve as an impetus for continued dialogue, awareness, and refinement of strategies to ensure that behavioral digital health interventions can meet the shared and unique needs of impacted communities. While this commentary offers broad recommendations, it should not be viewed as an exhaustive or comprehensive guide, but rather as an essential step toward greater inclusivity and equity as an imperative in the field of behavioral medicine. The five-point framework is not intended to be simply linear recommendations, but rather to serve as a foundational set of recommendations to be integrated throughout all phases of the research and dissemination process. It is intended to be a flexible and fluid blueprint that is responsive to the needs of the intended audience(s).
The commentary offers guidance on prioritizing equity when developing and testing digital health interventions. Of note, even as the ubiquity of digital health interventions spreads, it is important to recognize that some populations will not adopt, or may be slower to adopt, these technologies. In addition, for some, digital health interventions are not appropriate or feasible. When research teams and community partners are designing, testing, and disseminating behavioral interventions, they should carefully consider whether digital health is an appropriate or optimal modality and not have a subgroup of population left behind.
Conclusion
However, the development, testing, and dissemination of behavioral digital health interventions can have significant impacts on the perpetuation of health inequities. If inequities in digital health outcomes are not anticipated and intentionally avoided, there is a risk of further compounding health inequities. The PIDAR framework (Partner, Identify, Demonstrate, Access, Report) can be applied to help behavioral medicine researchers develop, implement and test digital health interventions to ensure health equity is consistently prioritized.
Acknowledgment
Dr. Brian Gonzalez reports fees unrelated to this work from SureMed Compliance and Elly Health, Inc.
Contributor Information
Sarah J Miller, Department of Population Health Science and Policy, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
Jamilia R Sly, Department of Population Health Science and Policy, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
Kassandra I Alcaraz, Sidney Kimmel Comprehensive Cancer Center, Johns Hopkins University, Baltimore, MD, USA.
Kimlin Ashing, City of Hope, CA, USA.
Shannon M Christy, Department of Health Outcomes and Behavior, Division of Population Science, H. Lee Moffitt Cancer Center and Research Institute, Tampa, FL, USA; Department of Oncologic Sciences, Morsani College of Medicine, University of South Florida, Tampa, FL, USA.
Brian Gonzalez, Department of Health Outcomes and Behavior, Division of Population Science, H. Lee Moffitt Cancer Center and Research Institute, Tampa, FL, USA; Department of Oncologic Sciences, Morsani College of Medicine, University of South Florida, Tampa, FL, USA.
Qian Lu, University of Texas MD Anderson Cancer Center, Houston, TX, USA.
Robert L Newton, Jr., Population and Public Health, Pennington Biomedical Research Center, Baton Rouge, LA, USA
Michelle Redmond, University of Kansas School of Medicine-Wichita, Wichita, KS, USA.
Megan Shen, Fred Hutchinson Cancer Center, Seattle, WA, USA.
Kamilah Thomas-Purcell, Nova Southeastern University, College of Healthcare Sciences, Department of Health Science, Ft. Lauderdale, FL, USA.
Jean Yi, Fred Hutchinson Cancer Center, Seattle, WA, USA.
Tiffany Veinot, School of Information and School of Public Health, University of Michigan, Ann Arbor, MI, USA.
Cathy D Meade, Department of Health Outcomes and Behavior, Division of Population Science, H. Lee Moffitt Cancer Center and Research Institute, Tampa, FL, USA; Department of Oncologic Sciences, Morsani College of Medicine, University of South Florida, Tampa, FL, USA.
Funding
Research reported in this publication was supported by the National Institutes of Health under award number K01HL135472, R01CA242742, K07CA190726, K01CA204456, K07CA207580. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Compliance with Ethical Standards
Conflicts of interest: Dr. Brian Gonzalez reports fees unrelated to this work from SureMed Compliance and Elly Health, Inc. All other authors declare that they have no conflicts of interest.
Primary Data: Findings reported have not been previously published and this manuscript is not being simultaneously submitted elsewhere. Data have not been previously reported elsewhere. The authors have full control of all primary data and agree to allow the Journal to review data if requested.
Ethical Approval: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This article does not contain any studies with animals performed by any of the authors.
Informed Consent: Informed consent was obtained from all individual participants included in the study.
Transparency Statements:
a)Study registration: This study was not formally registered.
b)Analytic plan pre-registration: The analysis plan was not formally pre-registered.
c)Data availability: There is not de-identified data associated with this study.
d)Analytic code availability: There is not analytic code associated with this study.
e)Materials availability: Materials used to conduct the study are not publicly available.
References
- 1. Bailey JV, Murray E, Rait G, et al. Interactive computer-based interventions for sexual health promotion. Cochrane Database of Systematic Reviews. 2010; (9). doi: 10.1002/14651858.CD006483.pub2. [DOI] [PubMed] [Google Scholar]
- 2. Harris J, Felix L, Miners A, et al. Adaptive e-learning to improve dietary behaviour: a systematic review and cost-effectiveness analysis. Health Technol Assess. 2011; 15(37):1-160. doi: 10.3310/hta15370. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Khadjesari Z, Murray E, Hewitt C, Hartley S, Godfrey C. Can stand-alone computer-based interventions reduce alcohol consumption? A systematic review. Addiction. 2011; 106(2):267–282. doi: 10.1111/j.1360-0443.2010.03214.x. [DOI] [PubMed] [Google Scholar]
- 4. Fedele DA, Cushing CC, Fritz A, Amaro CM, Ortega A. Mobile Health Interventions for Improving Health Outcomes in Youth: A Meta-analysis. JAMA Pediatr 2017; 171(5):461–469. doi: 10.1001/jamapediatrics.2017.0042. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Pal K, Eastwood SV, Michie S, et al. Computer-based diabetes self-management interventions for adults with type 2 diabetes mellitus. Cochrane Database of Systematic Reviews. 2013(3). doi: 10.1002/14651858.CD008776.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. van Beugen S, Ferwerda M, Hoeve D, et al. Internet-Based Cognitive Behavioral Therapy for Patients With Chronic Somatic Conditions: A Meta-Analytic Review. J Med Internet Res. 2014; 16(3):e88. doi: 10.2196/jmir.2777. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Widmer RJ, Collins NM, Collins CS, West CP, Lerman LO, Lerman A. Digital Health Interventions for the Prevention of Cardiovascular Disease: A Systematic Review and Meta-analysis. Mayo Clin Proc. 2015; 90(4):469–480. doi: 10.1016/j.mayocp.2014.12.026. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Keesara S, Jonas A, Schulman K. Covid-19 and Health Care’s Digital Revolution. N Engl J Med. 2020; 382(23):e82. doi: 10.1056/NEJMp2005835. [DOI] [PubMed] [Google Scholar]
- 9. Benda NC, Veinot TC, Sieck CJ, Ancker JS. Broadband internet access is a social determinant of health!. Am J Public Health. 2020; 110(8):1123–1125. doi: 10.2105/ajph.2020.305784. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Vogels, E. A., Perrin, A., Rainie, L., & Anderson, M. (2020). 53% of Americans Say the Internet Has Been Essential During the COVID-19 Outbreak. Pew Research Center. https://www.pewresearch.org/internet/2020/04/30/53-of-americans-say-the-internet-has-been-essential-during-the-covid-19-outbreak/ [Google Scholar]
- 11. WHO guideline. (2019). Recommendations on Digital Interventions for Health System Strengthening - Executive Summary. World Health Organization. https://www.ncbi.nlm.nih.gov/books/NBK541888/ [Google Scholar]
- 12. Hunsaker A, Hargittai E. A review of Internet use among older adults. New Media & Society 2018; 20(10):3937–3954. doi: 10.1177/1461444818787348. [DOI] [Google Scholar]
- 13. Perrin, A., & Atske, S. (2021). 7% of Americans don’t use the internet. Who are they? Retrieved 10/22/2021, from https://pewrsr.ch/2GrhLUj
- 14. Pew Research Center. (2021). Internet Broadband Fact Sheet. Washington, D.C.: Pew Research Center. https://www.pewresearch.org/internet/fact-sheet/internet-broadband/
- 15. Pew Research Center. (2021). Mobile Fact Sheet. Washington, D.C.: Pew Research Center. https://www.pewresearch.org/internet/fact-sheet/mobile/ [Google Scholar]
- 16. Vogels, E. A. (2021). Some digital divides persist between rural, urban and suburban America. Retrieved 10/22/2021, from https://pewrsr.ch/3k5aU6J
- 17. Merriam-Webster. (n.d.). Digital divide. In Merriam-Webster.com dictionary. https://www.merriam-webster.com/dictionary/digital%20divide
- 18. Perrin, A. (2021). Mobile technology and home broadband 2021. https://www.pewresearch.org/internet/2021/06/03/mobile-technology-and-home-broadband-2021/
- 19. Schartman-Cycyk, S., & Krebs, V. (2017). Adoption Persistence: A Longitudinal Study of the Digital Inclusion Impact of the Connect Your Community Project.
- 20. Anderson, M. (2018). About a quarter of rural Americans say access to high-speed internet is a major problem. https://www.pewresearch.org/fact-tank/2018/09/10/about-a-quarter-of-rural-americans-say-access-to-high-speed-internet-is-a-major-problem/
- 21. Smith, A. (2015). U.S. Smartphone Use in 2015, Chapter One: A Portrait of Smartphone Ownership. P. R. Center. https://www.pewresearch.org/internet/2015/04/01/chapter-one-a-portrait-of-smartphone-ownership/ [Google Scholar]
- 22. Van Deursen AJ, Van Dijk JA. The first-level digital divide shifts from inequalities in physical access to inequalities in material access. New Media & Society 2019; 21(2):354–375. doi: 10.1177/1461444818797082. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Horrigan, J. B. (2016). The meaning of digital readiness. https://www.pewresearch.org/internet/2016/09/20/the-meaning-of-digital-readiness/
- 24. Hargittai, E. Second-Level Digital Divide: Differences in People’s Online Skills. First Monday 2002; 7(4). doi: 10.5210/fm.v7i4.942. [DOI] [Google Scholar]
- 25. van Deursen, A., & van Dijk, J. Internet skills and the digital divide. New Media & Society 2011; 13(6), 893–911. doi: 10.1177/1461444810386774. [DOI] [Google Scholar]
- 26. Mackert M, Mabry-Flynn A, Champlin S, Donovan EE, Pounders K. Health literacy and health information technology adoption: The Potential for a New Digital Divide. J Med Internet Res. 2016; 18(10):e264. doi: 10.2196/jmir.6349. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. O’Connor S, Hanlon P, O’Donnell CA, Garcia S, Glanville J, Mair FS. Understanding factors affecting patient and public engagement and recruitment to digital health interventions: a systematic review of qualitative studies. BMC Med Inform Decis Mak. 2016; 16(1):120. doi: 10.1186/s12911-016-0359-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Vollbrecht H, Arora VM, Otero S, Carey KA, Meltzer DO, Press VG. Measuring eHealth Literacy in Urban Hospitalized Patients: Implications for the Post-COVID World. J Gen Intern Med. 2021; 36(1):251–253. doi: 10.1007/s11606-020-06309-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Davies AR, Honeyman M, Gann B. Addressing the Digital Inverse Care Law in the Time of COVID-19: Potential for Digital Technology to Exacerbate or Mitigate Health Inequalities. J Med Internet Res. 2021; 23(4):e21726. doi: 10.2196/21726. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. White, M., Adams, J., & Heywood, P. (2009). How and why do interventions that increase health overall widen inequalities within populations? In S. Babones (Ed.), Health, inequality and society (pp. 64–81). Bristol, UK: Policy Press. doi: 10.1332/policypress/9781847423207.003.0005 [DOI] [Google Scholar]
- 31. Buolamwini, J., & Gebru, T. (2018). Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification Conference on Fairness, Accountability and Transparency, http://proceedings.mlr.press
- 32. Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019; 366(6464):447–453. doi: 10.1126/science.aax2342. [DOI] [PubMed] [Google Scholar]
- 33. Doshi MJ. Barbies, Goddesses, and Entrepreneurs: Discourses of Gendered Digital Embodiment in Women’s Health Apps. Women’s Studies in Communication 2018; 41(2):183–203. doi: 10.1080/07491409.2018.1463930. [DOI] [Google Scholar]
- 34. Mannheim I, Wouters EJM, Van Boekel LC, Van Zaalen Y. Attitudes of health care professionals toward older adults’ abilities to use digital technology: questionnaire study. J Med Internet Res. 2021; 23(4):e26232. doi: 10.2196/26232. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Maguire M. Methods to support human-centred design. Int J Hum-Comput Stud. 2001; 55(4):587–634. doi: 10.1006/ijhc.2001.0503. [DOI] [Google Scholar]
- 36. Israel BA, Coombe CM, Cheezum RR, et al. Community-based participatory research: a capacity-building approach for policy advocacy aimed at eliminating health disparities. Am J Public Health. 2010; 100(11):2094–2102. doi: 10.2105/ajph.2009.170506. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Israel, B. A., Eng, E., Schulz, A. J., & Parker, E. A. (2013). Methods for Community-Based Participatory Research for Health (2nd ed.). San Fransisco, CA: Jossey-Bass. [Google Scholar]
- 38. Chen E, Leos C, Kowitt SD, Moracco KE. Enhancing Community-Based Participatory Research Through Human-Centered Design Strategies. Health Promot Pract. 2020; 21(1):37–48. doi: 10.1177/1524839919850557. [DOI] [PubMed] [Google Scholar]
- 39. Sefelin, R., Tscheligi, M., & Giller, V. (2003). Paper prototyping - what is it good for? A comparison of paper- and computer-based low-fidelity prototyping. CHI03: Human Factors in Computing Systems, Ft. Lauderdale, Florida, USA. doi: 10.1145/765891.765986 [DOI] [Google Scholar]
- 40. Kelley JF. An Iterative Design Methodology for User-Friendly Natural Language Information Applications. ACM Trans Off Inf Syst. 1984; 2(1):26–41. doi: 10.1145/357417.357420. [DOI] [Google Scholar]
- 41. Lazar, J. (2007). Universal Usability: Designing Computer Interfaces for Diverse User Populations. West Sussex, England: Wiley. [Google Scholar]
- 42. Czaja, S. J., Boot, W. R., Charness, N., & Rogers, W. A. (2019). Designing for Older Adults: Principles and Creative Human Factors Approaches (3rd ed.). Boca Raton, FL: CRC Press. [Google Scholar]
- 43. Rotondi AJ, Eack SM, Hanusa BH, Spring MB, Haas GL. Critical design elements of e-health applications for users with severe mental illness: singular focus, simple architecture, prominent contents, explicit navigation, and inclusive hyperlinks. Schizophr Bull. 2015; 41(2):440–448. doi: 10.1093/schbul/sbt194. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44. Visser, M. Digital Literacy and Public Policy through the Library Lens. Maine Policy Review 2013; 22(1):04-113. [Google Scholar]
- 45. Pew Research Center. Public Wants Libraries to Advance Education, Improve Digital Literacy and Serve Key Groups. Washington, DC: Pew Research Center; 2015. https://www.pewresearch.org/2015-09-15_libraries_0-01/ [Google Scholar]
- 46. Program, A. o. U. R. (2021). Citizen Science & Health. NIH National Library of Medicine. https://allofus.nnlm.gov/citizen-science-health [Google Scholar]
- 47. Federal Communications Commission. (2017). Strategies and Recommendations for Promoting Digital Inclusion. Washington, DC. https://www.fcc.gov/document/strategies-and-recommendations-promoting-digital-inclusion [Google Scholar]
- 48. Alcaraz KI, Kreuter MW, Bryan RP. Use of GIS to identify optimal settings for cancer prevention and control in African American communities. Prev Med. 2009; 49(1):54–57. doi: 10.1016/j.ypmed.2009.04.016. [DOI] [PMC free article] [PubMed] [Google Scholar]