Consumer Health Informatics Adoption among Underserved Populations: Thinking beyond the Digital Divide

Summary

Objectives:  Underserved populations can benefit from consumer health informatics (CHI) that promotes self-management at a lower cost. However, prior literature suggested that the digital divide and low motivation constituted barriers to CHI adoption. Despite increased Internet use, underserved populations continue to show slow CHI uptake. The aim of the paper is to revisit barriers and facilitators that may impact CHI adoption among underserved populations.

Methods:  We surveyed the past five years of literature. We searched PubMed for articles published between 2012 and 2017 that describe empirical evaluations involving CHI use by underserved populations. We abstracted and summarized data about facilitators and barriers impacting CHI adoption.

Results:  From 645 search results, after abstract and full-text screening, 13 publications met the inclusion criteria of identifying barriers to and facilitators of underserved populations' CHI adoption. Contrary to earlier literature, the studies suggested that the motivation to improve health literacy and adopt technology was high among studied populations. Beyond the digital divide, barriers included: low health and computer literacy, challenges in accepting the presented information, poor usability, and unclear content. Factors associated with increased use were: user needs for information, user-access mediated by a proxy person, and early user engagement in system design.

Conclusions:  While the digital divide remains a barrier, newer studies show that high motivation for CHI use exists. However, simply gaining access to technology is not sufficient to improve adoption unless CHI technology is tailored to address user needs. Future interventions should consider building larger empirical evidence on identifying CHI barriers and facilitators.

Introduction

The digital divide describes “the gap between those who have and do not have access to computers and the Internet” 1 . Systematic reviews of consumer health informatics (CHI) adoption have placed the digital divide as one of the core barriers to securing equal participation in technology-based health management solutions, specifically among underserved populations—groups that disproportionately experience difficulty accessing care due to social, economic, geographic, racial, or ethnic status 2 3 4 5 .

Internet adoption around the globe, however, has rapidly increased over the past 10 years, including in developing countries 6 , groups with low socioeconomic status, and racial and ethnic minority groups in developed countries 7 8 . Between 2010 and 2012, Hispanic and African-American populations in the U.S. represented the populations with the highest smartphone ownership rates, at 61% and 59%, respectively 9 . Between 2013 and 2015, adults in developing countries who use the Internet at least occasionally or report owning a smartphone increased significantly, from 6% in India up to 31% in Turkey 6 .

Tasks previously only possible on desktop computers, such as Internet and e-Health access, are now widely available through mobile phones and tablets. The global trend of increased Internet access and mobile phone ownership offers low-cost, scalable opportunities for CHI to empower individuals. A randomized controlled trial of MyHealthKeeper, a personal health record system from South Korea that allowed for sharing between patients and healthcare providers, resulted in significant improvements in weight-loss and triglyceride levels among users 10 . The Finnish National Archive of Health Information (KanTa), the national health data repository, was developed with the goal for citizens to access their own health information electronically 11 . In Sub-Saharan Africa, mobile and Internet technology penetration has resulted in increased female economic participation 12 . Mobile health applications in developing countries have shown effectiveness in many areas of medical care: improvement in patient follow-up 13 , uptake of counseling and testing 14 , and improved patient adherence and response to treatment 10 15 .

Given the increased penetration of Internet and mobile technologies across the globe, continuing to assume that basic technology access is the main contributor to health disparity vis-à-vis the digital divide may be in sufficient. This lack of context may potentially increase health disparities over time if left unchecked. Improving CHI adoption requires users to remain highly committed and motivated 2 16 17 18 . Sustained engagement, necessary for the adoption of any technology, has proved challenging in other scenarios 19 20 21 .

In this review, we survey articles that were published between 2012 and 2017 about CHI use. Given the changing atmosphere of underserved populations’ technology use, we concentrate on identifying barriers and facilitators. We suggest facilitators for developing future CHI systems that are sensitive to diverse user populations.

Methods

Our goal is to review the past five years of literature to follow up on persistent qualitative barriers and facilitators to CHI use among underserved populations. The goal is not to systematically review all possible literature, but rather update whether more recent literature continues to view the digital divide and motivation to use as the major barriers. We aim to enrich the discussion regarding what facilitators we can employ for future work in developing and evaluating CHI and overcome barriers to this end.

Operationalizing the Terms CHI and Underserved Population

We adapted the definition of CHI from the most recently updated publication on CHI ontologies 22 , which describes CHI as a technology that is: (1) consumer facing, where consumers refer to patients, caregiv-ers, or healthy individuals with prevention needs; (2) interactive for the consumer, including features such as buttons or links that enable retrieval of further information initiated by the consumer; and (3) providing tailored information, where the tailoring should happen for each consumer (e.g., providing personal health records, rather than general health information from Internet search) or the consumer group (e.g., tailored for cancer survivors group).

Defining underserved populations varies by country-specific political, cultural, and socioeconomic factors. Our review of a Cochrane study and national and international agencies publications settled around a common definition of underserved, or medically underserved, as those groups experiencing barriers to basic health needs due to social (including racial/ethnic minorities), economic, and geographic factors 23 24 25 26 . In the U.S., some agencies like the National Institute for Minority Health and Health Disparities (NIMHD), and the Health Resources and Services Administration's Medically Underserved Areas (HRSA-MUA), which defines underserved as those experiencing a lack of access to basic health care, have operationalized these key characteristics linked to social, economic, and geographic vulnerabilities that are in-step with these global definitions for purposes of research and funding 27 28 29 .

For the present review, we define under-served populations as racial/ethnic minorities in the context of country (e.g., Turkish in Germany, Hispanics and African-Americans in the U.S.), social (e.g., education, literacy, language), economic (e.g., employment, poverty, insurance), or geographic (e.g., rural) barriers.

Data Sources and Searches

We searched PubMed from October 2012 to October 2017 for full-text studies published in the English language regarding barriers and facilitators to CHI use in underserved populations. Staying within the scope of a survey, we restricted keywords to only Medical Subject Headings (MeSH) terms representative of “barriers and facilitators of CHI use among underserved populations” as operationalized above and we avoided custom keywords that might potentially bias the search results if not systematically chosen. We summarize the search strategy in Table 1 .

Table 1. Medical Subject Headings terms used for searching abstracts on PubMed.

Consumer health informatics technology	▪ Consumer facing ▪ Interactive, such as buttons or links that enable retrieval of further information initiated by the consumer ▪ Providing tailored information, where the tailoring should happen for each consumer (e.g., providing personal health information) or the consumer group (e.g., tailored for Spanish speaking groups)
Consumer	Consumers could refer to patients, caregivers, or healthy individuals with prevention needs
Underserved population	▪ Racial/ethnic minorities in the study context of the country (e.g., Turkish in Germany, Hispanics and African-Americans in the U.S.) ▪ Social barriers (e.g., education, literacy) ▪ Economic barriers (e.g., employment, poverty, insurance) ▪ Geographic barriers (e.g., rural)
Barriers and facilitators	Keywords related to access, tailoring, or user-centered design
Constraints	▪ Full text ▪ Published in the past 5 years ▪ Written in English

Data Extraction and Synthesis

After a reviewer training phase to ensure inter-annotator agreement, we performed a single review for title and abstract screening. A full review of the selected text was performed by MS and JH, and data abstraction was performed by JH.

From each study selected for full-text review, we abstracted the following: population characteristics, setting, number of subjects, health conditions studied, study type, and barriers/facilitators to CHI use. To identify the paper's definition for un-derserved population we abstracted race/ ethnicity, income and insurance indicators, education level, and geography. We followed the thematic analysis commonly used in qualitative research methods 30 .

We searched for themes that “emerge as being important to the description of the phenomenon” 31 such as, in our case, facilitators and barriers to CHI adoption among the target population. We identified the themes through “careful reading and re-reading of the data” 32 . We recognized patterns within the data, allowing emerging themes to become the categories for analysis. We adapted PRISMA guidelines for our protocol specification, data abstraction, and synthesis 33 .

Results

We obtained 639 abstracts after removing missing abstracts from 645 search results. There were no duplicates. We removed 586 abstracts because there was no CHI involved in the study—either because the study did not examine implemented CHI (e.g., interviews and questionnaires about future CHI) or because the CHI being studied did not involve direct interaction with users (e.g., educational videos, one-way text message alert system). We then removed an additional 22 abstracts because the studies reported lacked an underserved population perspective on CHI even if the study did involve CHI.

As a result, after the abstract screening, 31 articles remained, from which we excluded 18 articles through the full-text screening (see Figure 1 ). Reasons for exclusion during full-text review included one or more of the following reasons: no data on the underserved (n=5) or on facilitators and barriers to technology adoption (n=5), the article was only about the study protocol and was lacking outcome data (n=3), the technology was not CHI (n=4), or the study did not empirically test technology on human subjects (n=2). As a result, 13 articles finally remained for full-text analysis.

Fig. 1.

Selecting articles related to the use of CHI by underserved populations. Diagram adapted from PRISMA guidelines.

The CHI solutions evaluated in these 13 articles included patient education tools (n=7) 34 35 36 37 38 39 40 , patient portals (n=4) 41 42 43 44 , and technology-based illness intervention (n=2) 45 46 . Nine articles presented mixed methods 35 36 37 38 39 42 43 44 46 , nine included interviews and focus groups 34 35 36 37 38 39 43 44 46 , five included surveys 2 36 39 42 44 , three included trials 37 45 46 , and three included a cohort analysis 41 42 46 . The total sample size ranged from 21 patients to more than 200,000 patients. The studies were from two countries: U.S. (n=12) and Netherlands (n=1) 34 . Six health areas were covered: cancer (n=4) 35 36 38 39 , cardiometabolic risk and nutrition (n=2) 34 40 , HIV/AIDS (n=1) 44 , environmental health for prenatal patients (n=1) 37 , medication adherence (n=1) 46 , and dementia (n=1) 45 . Two articles covered the use of general patient portals, not specific to an illness 41 42 and two specified “chronic conditions” without further detail 42 43 . Eligibility criteria included older adults in five studies 40 42 43 45 46 . Four articles focused only on Hispanic populations 35 37 44 46 , four only on African-Americans 36 38 39 43 , two on Hispanic and African-Americans 40 45 , and the rest on one or more ethnic minorities 34 41 42 . Nine focused on low-income patients 36 37 38 40 41 42 43 44 46 , and 11 on low health literacy or low education populations 34 35 36 37 38 39 40 42 43 44 45 . Tables 2 and 3 display a summary of the inal articles.

Table 2. Characteristics of CHI and population in the selected studies.

Author, year	Type of CHI applications	Total number of participants	Reported age of participants	Indicators for underserved populations
				Ethnicities of interest	Income indicators	Reported education level	Reported insurance status
Ancker, 2017 41	Patient portal: Medline connected links in medical records	12,877	18-24: 18.5% 25-44: 43.5% 45-64: 30% 65+: 8%	Latinos by language preference, Black, White, Other, Unknown	Patients at Federally Qualified Health Centers (FQHC)	n/a	Private, Medicaid, uninsured, Medicare
Gordon, 2016 42	Patient portal: Kaiser North California	231,082	65-79	English speaking non-Hispanic White, Black, Hispanic, Filipino, and Chinese	30.3% had low income	[Survey] 22% Latino seniors and 4% others did not graduate from high school	Kaiser HMO
Damman, 2016 34	Patient education tool: Web-based cardiometabolic disease risk calculator and information	23	40-66	n/a	n/a	Low health literacy
Kukafka, 2015 35	Patient education tool: Web-based decision aid for breast cancer prevention	34	Mean age: 53.4 (SD=n/a)	Hispanic	n/a	41% had low numeracy	n/a
Owens, 2015 36	Patient education tool: Computer-based decision aid	21	37-66	African-American	More than half had income lower than $39,999/yr	14% finished high school	20% Medicaid or no coverage or other insurance
Smith, 2015 43	Patient portal: Registration and utilization of a patient portal	534	55-74	African-American	FQHC and ambulatory care clinic	17.4% had low health literacy, 14.8% graduated from high school or less	n/a
Rosas, 2014 37	Patient education tool: Kiosk, interactive game for prenatal and environmental health	152	n/a	Hispanic	Low income, FQHC	Low literacy	n/a
Odlum, 2014 44	Patient portal: Internet-based electronic personal health, management tools	42 [Survey], 15 [Focus groups]	24-63	Hispanic	80.9% earned less than $10,000/yr	83% graduated from high school or General Educational Development (GED)	Medicaid Special Needs Plan
Cogbill, 2014 38	Patient education tool: Online colorectal cancer education website	18 [Focus groups], 60 [Trial]	45-75	African-American	[Focus groups] 16.7% earned $10,000/yr or less, [Trial] 33.3% earned $10,000/yr or less	[Focus groups] 33.3% graduated from high school or less, [Trial] 31.7% graduated from high school or less	n/a
Czaja, 2013 45	Technology-based intervention: In-home and videophone technology, multi-component psychosocial intervention	110	Mean age: 60.9 (SD: 12.8)	Hispanic, African-American	n/a	40% of the intervention group and 36.5% of the control group had less than high school education	n/a
Bass, 2013 39	Patient education tool: Low-literacy computer touch-screen colonoscopy decision aid	102	50-74	African-American	n/a	Low literacy; less than 6th grade REALM score 87	n/a
Lapane, 2012 46	Technology-based intervention: Tailored DVDs on medication adherence	326 [Telephone survey], 106 [First focus group], 16 [Second focus group]	Eligibility: At least 65 years old; n/a for actual participants	Hispanic	Low-income	n/a	n/a
Neuen-schwander, 2012 40	Patient education tool: Web-based nutrition education program	123	18-30: 48% 31-50: 39.8% 51-70: 10.6% 71+: 1.6%	Hispanic, African-American	SNAP-Ed eligible	43% graduated from high school or less	n/a

Table 3. Barriers, facilitators, and study context of the reviewed articles.

Author, year	Health condition of interest	Study or instrument type	Barriers	Facilitators
Ancker, 2017 41	n/a	Cohort study	Low health and computer literacy	Needs for more information
Gordon, 2016 42	Chronic disease	Cohort study, survey	Low health and computer literacy	Help from proxy users
Damman, 2016 34	Type 2 diabetes, cardiovascular disease, chronic kidney disease	Interviews	Low health and computer literacy, challenges accepting the presented information
Kukafka, 2015 35	Breast cancer	Focus groups, survey	Challenges accepting the presented information
Owens, 2015 36	Prostate cancer	Focus groups, survey		Early user engagement In design
Smith, 2015 43	One or more chronic conditions	Interviews	Low health and computer literacy, challenges accepting the presented information, poor usability and clarity	Early user engagement in design
Rosas, 2014 37	Pregnancy, environmental health	Pre/post test, open-ended interviews		Early user engagement in design
Odlum, 2014 44	HIV/AIDS	Focus groups, survey	Challenges accepting the presented information, poor usability and clarity	Needs for more information
Cogbill, 2014 38	Colorectal cancer	Focus groups, a 3-week feasibility trial	Low health and computer literacy, challenges accepting the presented information, poor usability and clarity	Early user engagement in design, needs for more information
Czaja, 2013 45	Caregivers of patients with dementia	A 5-month randomized clinical trial		Early user engagement in design, needs for more information, proxy users’ help
Bass, 2013 39	Colorectal cancer	Focus groups, survey, segmentation analysis		Early user engagement in design
Lapane, 2012 46	Medication adherence	Cohort study, focus groups, survey	Challenges accepting the presented information, poor usability and clarity	Early user engagement in design
Neuenschwander, 2012 40	Nutrition	1-month randomized, block equivalence trial	Low health and computer literacy

Barriers

We found three main barriers to CHI adoption among underserved populations: (1) low health literacy 34 40 41 42 43 and lack of experience with information technology use 38 42 43 ; (2) challenges in accepting the presented information 34 35 38 43 44 46 ; and (3) poor usability and clarity of content 38 43 44 46 .

• Low health literacy and lack of experience with information technology use

Many study participants did not have experience in using the Internet from school or work, lacked cellphones with Internet access, and had little contact with Internet technology 38 42 43 . This inexperience hindered their ability to adopt and use CHI solutions without appropriate training. In a trial involving nutrition education websites in a low-income community in the U.S. Midwest 40 , understanding medical language, or health literacy, was a barrier to using the tool. Conversely, an in-person meeting was perceived as being more useful than the CHI intervention. Furthermore, participants did not think they could use a CHI application 38 42 . This result was supported by a patient portal use study among older adults in Northern California 42 and a feasibility trial of an online colorectal cancer education program with African-American older adults 38 . Focus groups conducted in the latter study showed that texting might not be feasible for the population because the participants either did not own cell phones or, when they owned one, did not know how to use it or feared that texting would constitute a financial burden 38 .

• Challenges in accepting the presented information

In some studies, even when participants had access to technology, some of them did not find the presented information useful 38 44 . In another study 34 , the presented information contradicted what participants believed about their own health, or they misinterpreted the materials. For instance, in a Dutch study involving low health literacy individuals, participants were provided their cardiovascular risk using technology-based educational materials 34 . The participants either did not believe or misconstrued their risk based on how the information was visualized. Similarly, a study displayed breast cancer risk using a web-based decision aid tool where the majority of participants were Hispanic women with low numeracy. The participants, who felt uncertain about the models presented to them, attributed the reason of their distrust with their healthcare providers to their past interaction with the providers 35 . Odlum et al. 44 studied the use of Internet-based electronic personal health management tools among a mostly minority, low-income HIV/AIDS urban clinic population. The participants preferred to enter their own health history rather than accepting the data generated by the clinic, which they found confusing.

• Poor usability and clarity of content

Lastly, usability problems and a lack of message clarity hindered CHI adoption. For instance, losing an access code after registration deterred older adults from using a patient portal 43 . Confusing user interfaces made it difficult to use patient portals or to benefit from patient education materials 38 43 44 . Tailored educational materials (DVD) for medication adherence aimed at older adults were mainly critiqued for their background color rather than their content 46 . In a study of a colorectal cancer screening education tool with African-American men, the participants felt that the messages were vague and should be further tailored—otherwise they would not motivate behavior change 38 .

Facilitators

We found three main CHI facilitators: (1) early user engagement through iterative user-centered design 36 37 38 39 43 45 46 ; (2) engaging users early in the design development process and identifying their health information needs 38 41 44 45 ; and (3) proxies, such as caregivers or family members, who are more familiar with technology, and use CHI on behalf of the users 42 45 .

• Early user engagement through iterative user-centered design

Participants were more willing to use CHI when the system was usable, engaging, trusted, and tailored 36 37 38 39 43 45 46 . To meet these requirements, one aspect frequently discussed was that CHI should allow for customized communication modes. In a study evaluating an environmental health education intervention for pregnant Hispanic women via a kiosk, researchers communicated information through both audio and text on the screen. Some participants preferred voice to text whereas others preferred reading the information on-screen 37 . Conversely, participants in a colorectal cancer screening study that used text messages versus emails to assess an educational tool had contrasting preferences for receiving reminders and learning materials 38 . Factors influencing their decisions included perceived cost of texting, ease of use, annoyance, and likelihood to grab attention. Bass et al. developed a colonoscopy decision aid for African-American men 39 ; survey and focus group results showed that photographs were preferred over graphics in depicting educational materials. As a result, the aid included photographs coupled with testament videos from the actual clinic patients. While web-based nutrition education was as effective as in-person counseling for low-income participants in Neuenschwander et al.'s study 40 , some topics (e.g., nutrition facts labeling) benefited from a combination of web-based and in-person approaches.

Three studies used varied methodological contexts and study scales (sample size ranging from 21 to 534) 36 43 46 . These studies emphasized the importance of early engagement of end-users into the design process through user-testing and improvement of functionality 36 43 . This process helped add tailored information that met user needs. For instance, a mixed methods study presented the development of a touch-screen decision aid for low health literate African-Americans with colorectal cancer 39 . The study revealed that psychosocial issues related to the colonoscopy rather than medical information on colorectal cancer were the more critical factors in decision-making. In the health-education study with prenatal Hispanic women, adding games helped children, partners, and all family members engage in learning about environmental exposures using a kiosk 37 . Further examples of tailoring included adding actors for a video intervention, who were relatable to the user population 46 .

• Intrinsic needs for more information

Consumer health informatics use was facilitated when participants had an intrinsic need for more information 44 . The works by Ancker et al. 41 and Odlum et al. 44 demonstrated that participants found that practical tips for provider engagement and health management were most useful. In a study with Medicaid users, participants felt that information on facilitating provider visits was useful as a personal health management tool 44 . African-American participants from a colorectal cancer screening tool study expressed that they wanted tips on free or low-cost screening 38 . A videophone-based intervention for dementia patients and their caregivers showed that having access to a support group was helpful and that some participants wanted more information on accessing support groups as part of these interventions 45 .

• Proxy users’ help

For those with low computer literacy, having a delegated person who could help use the CHI had an impact on CHI adoption 42 45 . As evidenced by the videophone study involving dementia patients and their caregivers, caregivers felt motivated to use the technology when it allowed them to better understand their patients’ illness. CHI could contribute to an increase in caregivers’ abilities to help patients take care of the illness 45 .

Discussion

The barriers and facilitators discussed in the present review—low health literacy, tailoring, and the digital divide —have all been considered at length in prior literature 3 46 47 48 49 . Studies published in the past 5 years shows a lag in CHI adoption among the underserved when compared to the general public. Additionally, our results further show that a digital divide persists 50 51 52 53 . At the same time, increased mobile technology adoption by underserved populations has slowly changed the state and nature of the digital divide 9 54 . Furthermore, newer studies suggest an increased willingness to engage with CHI tools among underserved populations 41 55 . Our findings lead us to discuss how the digital divide, literacy, and user-centered design of CHI should be approached.

Re-thinking the Digital Divide, Motivation, and Perceived Usefulness

A 2008 systematic review report to the Agency for Healthcare Research and Quality (AHRQ) 3 concluded that users found the majority of the evaluated CHI tools to be usable. A 2010 report to the Office of the National Coordinator for Health Information Technology (ONC) 4 showed that health and technological literacy, culture and language, level of comfort in interacting with the health care system, and digital divide added to the evidence of these factors as barriers to CHI adoption. A more recent systematic review in 2011 5 further confirmed these findings, identifying perceived benefits of health information technology, and conversely highlighting a lack of trust, technical problems, limited access to computers or hardware, technology fears, and cognitive and physical disabilities as persistent barriers to CHI adoption among underserved populations. These reports emphasize a lack of user-motivation and barriers to technology access rather than the design of CHI systems as the main drivers of the low adoption rate.

However, 10 years later, the digital divide does not appear to persist due to a lack of technology adoption, especially given the increased mobile technology use and Internet adoption among underserved populations globally 9 54 56 . Instead, the digital divide is driven by more complex, multi-dimensional factors. Ancker et al. showed from their 2017 study on patient portals at Federally Qualified Health Centers (FQHCs) 41 , that ethnic minorities, such as Hispanics and African Americans, were more likely to use hyperlinked patient education materials in patient portals than were Caucasian users. Ancker et al. posited two possible causes: first, users’ motivation to understand the medical jargon, and second, a strong association between low health literacy and the need for further explanations. In either case, the study shows intrinsic motivation to learn by those often labeled a ‘disengaged’ population 57 58 59 . All people, regardless of their privilege status, were highly motivated, had intrinsic needs for information 41 , wanted to manage their own health information and to share it with their providers 44 , wanted tips on free or low cost ways of accessing care 38 , or wanted to learn more on how to use CHI 41 44 60 .

Our review also found that motivated use was not limited to the target user of CHI. If the patients themselves could not use the CHI tool, caregivers became highly motivated, assuming CHI user roles 42 45 . This concept of a proxy user aligns with past findings that refer to caregivers’ effective use of technology either for themselves 60 or as helpers for patients 61 62 . A systematic review of older adults’ use of patient portals identified technical assistance and family and provider advices as the main facilitators for patient portal use 51 . In an interview study about patient portal use in safety net hospitals, caregivers expressed interest in using patient portals to interpret health information, advocate for quality care, or manage health behaviors and medical care of patients 63 .

Engaged and motivated users should receive sustainable and culturally appropriate support to help improve computer and health literacy. A recent study by the University of Kansas 64 found that low-income African-Americans wanted to learn how to use computers but study participants felt that the educators were condescending and hence they lost their motivation to return to the education sessions. A similar report was published in 1991, where African-American women engaged in a literacy improvement program were discouraged from going back to the classes because “the instructors were too mean” 65 . Such breach of trust between innovation disseminators and end-users must be repaired, and assumptions around technology adoption in underserved populations should be reconsidered and addressed in design and dissemination.

Institutional response to health disparities, or variation in access, quality, and care, has been incremental and piecemeal with respect to underserved populations. For example, language services are mandated for all hospitals that receive federal support like Medicare reimbursement, which is nearly all hospitals in the U.S.; however, uptake is less than 70% 66 . Thus, CHI interventions must not only consider barriers related to technology adoption, but also culturally competent care delivery, health equity, and significant institutional barriers. Understanding the values and beliefs of underserved populations must be a priority given the growing racial, ethnic, and linguistic diversity of many countries.

Internationally, similar principles apply. For instance, McBride et al. used SMS to help with maternal health among ethnic minorities in Vietnam 67 . A U.K. based systematic review reported, when developing ethnic-specific dietary assessment tools, using customized portion sizes by sex and age, household utensil usage, and literacy levels are critical 48 . The growing focus on patient-centered care serves as an opportunity to secure institutional buy-in to tailor healthcare 68 69 70 . Culturally and linguistically appropriate care could aid CHI diffusion for underserved populations across the globe given universal health care, along with rapid Internet for low-resource and rural areas 66 .

Although the digital divide still exists, barriers to technology access will likely diminish, but general computer literacy will likely continue to impede progress globally. Research groups such as the 2G Lab at the University of Michigan 71 are responding to emerging digital literacy gaps by redesigning and repurposing older technologies (e.g., non-smart phones). Such endeavors consider tailoring needs to the individual a technology innovation in and of itself. This focus on the individual may be the key to CHI development for underserved populations.

It is no longer sufficient to state that the core barriers to CHI adoption by under-served populations are the lack of access to technology, or the lack of motivation or perceived usefulness of CHI applications by the end users as past studies suggested 72 73 74 . Rather, it is currently more important to determine how CHI can be tailored to support culturally relevant, intrinsic, and personalized information needs.

Re-thinking CHI Usability Evaluations for Underserved Populations

Usability and design problems can discourage even highly motivated users. In some of the studies reviewed 38 43 44 46 , participants did not find information useful because the display was confusing or they could not relate to the content or the actors communicating the information. These findings contradict the general consensus of the AHRQ report 3 , which found the reviewed CHI systems’ usability to be high.

This contradiction may be explained by differing variables and tasks chosen for the usability evaluation. For instance, Greenberg and Buxton described this phenomenon in their seminal article, “Usability Evaluation Considered Harmful (Some of the Time)” 75 . They discussed the importance of choosing appropriate evaluation techniques to the problem and the stage of the design cycle. Otherwise, the results can be meaningless. Depending on the user groups tested and the tasks chosen, the results may not reflect how the technology would actually evolve for its intended audience and actual use. Aspects of user-centered design other than usability, such as understanding requirements, considering cultural aspects, and developing and showing stakeholders design alternatives, should be taken into account when evaluating technology use.

Many “quick and dirty” usability evaluation solutions exist that have proven to be as equally reliable and powerful as more comprehensive measurements 76 77 78 79 . However, these methods should be carefully chosen when involving populations who may have linguistic, cultural, and literacy challenges. For instance, Bangor et al. discussed how simpler usability measurements, such as the System Usability Scale Survey (SUS) 80 , should be accompanied by other measurements 81 . The language used for the survey items in the SUS, because of its terseness, can cause comprehension problems for non-native English speakers 82 83 84 . Peres et al. warned that shorter surveys meant to be designed for non-usability specialists can in fact hinder a correct interpretation of the results if a facilitator was not present for clarifications 85 . These studies demonstrated the need for careful consideration while interpreting scores in evaluating a system.

Lessons learned for improving CHI adoption among the underserved include assuring user-centered design has been deployed before dissemination and evaluation. In their 2012 systematic review 47 , Montague and Perchonok suggested providing tailored, relevant, and contextu-ally situated health technology to enable behavior change among underserved populations. Simply translating English to Spanish, for instance, has been shown to be an ineffective solution to increasing technology adoption 86 . Personalizing the tool for each individual and understanding intrinsic needs of users and utilizing proxy users, such as caregivers or younger family members, should help motivate CHI adoption by underserved populations.

Limitations and Future Directions

We confined our search to PubMed indexed publications and our search strategy was very specific. We adopted a definition of underserved population that, though promulgated by U.S. health agencies 27 28 29 , is race/ethnicity neutral. However, we acknowledge that this perspective might not agree with other definitions developed outside the U.S. While we attempted to be as inclusive as possible of the international context, several studies were excluded during the abstract or full text screening process. We restricted publications to English only, which may have eliminated studies of CHI in underserved populations. We did not employ pre-specified procedures to assess the risk of bias in individual studies. Nevertheless, we have referenced selected studies within this review outside of the context of our data synthesis. Future research may consider using inclusion and exclusion criteria that are specifically designed to address non-U.S. contexts to complement what might have been lost in this review. We need more empirical research reporting facilitators and barriers that can apply to a broader international context and address the ‘research divide’ shown from the results of our screening process.

On the 31 screened articles, many did not discuss factors that impacted CHI adoption. However, studies that incorporated qualitative methods, such as conducting focus groups after a trial or coupling surveys with interviews provided insights into what might have been barriers or facilitators to CHI adoption but were not generalizable. These studies did not empirically confirm factors around barriers and facilitators for generalization, perhaps because they were derived from qualitative feedbacks from a small number of individuals. Future studies should consider testing the effectiveness of facilitators and barriers in CHI adoption in a larger, confirmatory study setting to understand scalability and generalizability issues that are predominant among under-served population groups. Furthermore, user-centered design techniques that result in reliable methods for tailoring, such as expected scenarios of use, reflections, case studies, and participatory critique should be considered in addition to usability methods.

Conclusion

The digital divide and few perceived benefits of CHI use were previously considered as the dominant barriers to CHI adoption among underserved populations. The narrowing digital divide, due to increasing technology access, will not by itself solve the problem of low adoption rates. Digital divide can come from a variety of factors, including lack of net neutrality and geographic constraints that require resolution before asserting improved technology access as a solution. Contrary to misleading assumptions that underserved populations, who may suffer from low health and computer literacy, are largely disinterested in engaging with technology, studies published in the past five years indicate high motivation to adopt technology and improve literacy. CHI development should benefit from varied user-centered design techniques that address context and individualized needs of each user. At the same time, there is still much to be learned about underserved populations’ CHI use. Future studies should develop systematic methods of evaluating effective user-centered design and adoptability of CHI use among underserved populations.