How is digital health suitability measured for communities? A systematic review

Abstract

Background

Digital health services continue growing in usage and popularity, with patients and healthcare organizations benefiting from their use. Despite this, no mechanism exists to measure a patient's and community's suitability to leverage these services.

Objective

This systematic review aims to evaluate the extent and nature of measuring the overall suitability of individuals and communities within the digital health landscape.

Methods

Database searches in February 2024 across PubMed, Cumulative Index of Nursing and Allied Health Literature (CINAHL), Embase, and IEEE Xplore yielded 1044 unique references. Two screening stages resulted in 10 articles that met all evaluation criteria for review inclusion.

Results

This systematic review found a gap in the ability to holistically assess a patient's and community's suitability to access digital health services. Myriad indices and tools identify isolated factors contributing to digital health accessibility (e.g., broadband availability); however, no comprehensive mechanism adequately informs providers, policymakers, and researchers.

Conclusion

A comprehensive index that accurately reflects suitability for digital health services is needed. Index factors should include a combination of indicators related to socioeconomic status, digital accessibility, such as device and internet access, and social determinants of health. Together, these form the predominant driving factors related to one's ability to participate in digital health services.

Introduction

Digital health inequity persists. Once considered a universally accessible solution to healthcare access, digital health can increase challenges to access for populations with lower socioeconomic status (SES) ¹ and in rural areas, ² which account for 20% of the population ³ in the United States. Therefore, mechanisms must exist to measure digital health accessibility, which can inform providers, policymakers, and researchers accurately.

The digital health landscape has grown in scale and acceptance with the COVID-19 pandemic, illuminating its manifold benefits to patients and practitioners. ⁴ Digital health tools such as telehealth, mobile health (mHealth), and remote patient care are vital to load-balancing healthcare encounters, enabling increased flexibility, democratization, and options for all parties involved. ⁵ This wide range of components, coupled with its exponential growth, has caused a paradigm shift in the healthcare industry. For this systematic review, we defined digital health as any technology for healthcare service delivery or patient engagement outside of traditional care settings.^1,6,7

However, digital health services are not equitably available. Socioeconomic, sociodemographic, digital literacy levels and geographic factors significantly contribute to a patient's likelihood of leveraging digital health services.^1,4 For example, the lack of nearby healthcare facilities coupled with transportation barriers in rural areas, which account for 20% of the US population, reduces the frequency and quality of healthcare^3,8–10. Yet populations with low social determinants of health (SDOH), low income, and lack of higher education are more challenged to incorporate specific modalities in their households.^1,11 These contributing factors exemplify metrics that must be assessed and accounted for to inform healthcare entities on appropriate care modalities for individuals and communities.

Numerous frameworks related to digital health inequity assessment exist. Examples include the Digital Health Equity Framework, ⁶ the NQF Telehealth Measurement Framework, ¹² the SPROUT STEM Framework, ¹³ and PIDAR. ¹⁴

The DHEF identifies the digital determinants of health (DDoH) and their links to digital health equity, highlighting the unintentional impacts of pivoting to digital health services on vulnerable populations and the potential of digital health services to mitigate health risks and improve equity. It also emphasizes how DDoH impacts digital health equity, compounding with SDOH variables. ⁶ However, the DHEF does not quantify these concerns due to a lack of available data, suggesting that further work must be done to develop systems and mechanisms to identify gaps and increase attention to digital health inequity.

Like the DHEF, the Partner, Identify, Demonstrate, Access, Report (PIDAR) 5-point framework can be used to avoid furthering digital health inequities. PIDAR raises awareness of such inequities, providing guidance and strategies for reference when developing or testing digital health interventions, ¹⁴ but has no associated mechanisms operationalizing these strategies.

Furthermore, the National Quality Forum's Telehealth Measurement Framework identifies the measures and concepts that form a conceptual foundation for assessing the care quality of digital health modalities. ¹² While this framework expertly identifies the critical areas for measurement within domains such as experience, financial impacts, and access to care as they relate to specific clinical areas, it only serves as a guide to which measures should be considered. It does not provide a mechanism or data that translates to metrics that can inform actions. Like the NQF's framework, the SPROUT STEM framework has no associated mechanisms that directly measure digital health services. The Supporting Pediatric Research on Outcomes and Utilization of Telehealth (SPROUT) Telehealth Evaluation and Measurement (STEM) framework evaluates care delivery within the digital health landscape, examining ways to integrate digital health into established workflows. ¹³ Most relevant to this review is its objective of identifying the appropriate usages of telehealth based on patient cohorts, particularly within the factors of healthcare delivery, access, and equity.

While these examples have implications for policy research and practice development, they do not offer or integrate any index or tool to measure their parameters. This systematic review aimed to evaluate the extent and nature of measuring the overall suitability of individuals and communities within the digital health landscape. Our review evaluated existing indices, frameworks, and technological tools that assessed digital health service accessibility.

Methods

Research questions

The evaluated literature was screened to assist in answering two primary questions. First, is there an index that measures a patient's suitability for digital health services? If so, what metrics and parameters are used, how is the index used, and by whom? Second, does a tool exist to integrate such an index for healthcare and research organizations? If so, how is it used, and by whom?

Databases

In February 2024, PubMed, Cumulative Index of Nursing and Allied Health Literature (CINAHL), Embase, and IEEE Xplore were searched to identify relevant literature regarding indices, frameworks, and measurement tools related to digital health.

Eligibility criteria

Studies published before 2014, those whose full text was unavailable during the review period, and those published in languages other than English were excluded. Also excluded were studies that discussed the evaluation of digital health programs without using an index and studies that discussed indices unrelated to digital health. Studies that developed indices and frameworks related to digital health or discussed technological tools associated with digital health indices were included.

Search strategy

LB and JG identified key search terms such as inequity, disparity, digital health, telemedicine, index, framework, measurement, and geographic-related terms. These terms were supplemented by topical controlled vocabulary terms in each database to construct a preliminary search strategy. Upon review of this strategy and its results, recommendations from SK were incorporated to improve its specificity and include known relevant studies. Grey literature sources were not referenced. A complete listing of final search result counts by database is included in Appendix 1.

Study selection

Database searches across PubMed, Cumulative Index of Nursing and Allied Health Literature (CINAHL), Embase, and IEEE Xplore yielded 1394 references, with the final search performed on February 22, 2024. Covidence online screening software was used to facilitate two screening stages. Upon removal of duplicate results, 1044 articles underwent a title and abstract screening by JG and LB. Decision conflicts that emerged during this stage were resolved via discussion of rationale, with 31 ultimately meeting the criteria for full-text review, of which 3 were unretrievable. JG and LB conducted the full-text screening, resolving any conflicts via discussion, excluding 15 studies for lack of index or tool implementation and 3 due to insufficient information. Therefore, 10 articles met all evaluation criteria to be included in this review. Figure 1 illustrates this process with the PRISMA 2020 flow diagram. ¹⁵ This figure can also be accessed in Appendix 2. Our collaborative selection strategy, coupled with using the Covidence screening software, allowed for reproducible results, producing a low risk of bias for this review.

Results

Evaluation

A careful review of the literature was conducted to identify the existence of indices or frameworks that examine the suitability of digital health services for a population. Inclusion criteria included studies that developed indices and frameworks related to digital health or discussed technological tools associated with digital health, were published in English, and were published since 2014. Studies were excluded if they were not published in English or did not use an index to evaluate digital health programs and services. Upon application of evaluation criteria, 10 articles were selected from the original 1044. Adhering to the PRISMA-P resource, a matrix has been included within Appendix 3 detailing the literature reviewed. Retrospective studies were the predominant research method, often aggregating patient electronic health records (EHRs) and geographic information against an indexing tool, such as the Area Deprivation Index (ADI). None of these articles focus solely on an existing index, such as the ADI, and an analysis and table of all referenced and relevant indices are included later in this review.

Index measurement performance

The extent to which selected indices and methods evaluated digital health accessibility was reviewed in 10 articles. The most common factor assessed revolved around broadband metrics, such as availability, durability, and affordability^8,16–19. All articles referenced the Federal Communications Commission's (FCC) standard of sufficient broadband speed to support telehealth with a minimum 25Mbps download and 3Mbps upload speed. ²⁰ Increased broadband availability was correlated with increased access to healthcare, but conversely, added burdens (so-called double or triple burdens) to rural communities, 50% of which lack broadband access on top of existing physician shortages.^2,17 Access to digital devices to facilitate digital health participation was examined via screening questionnaires and not through existing EHR data.^18,21 By contrast, digital literacy was the predominant focus of the two international studies (Poland and Australia) included in this review.^22,23 Although it is well documented that low SES is associated with lower digital health accessibility, ¹ it is worth noting that only one study included affordability as a metric, ² citing internet subscription subsidies as a potential barrier removal. Moreover, it was shown that the higher the need or risk associated with a study's index score (e.g., ADI), the less accessible that region is for digital health services.^2,16,21 One study found that 71% of clinics offering digital health services in California resided in regions with the greatest need, according to a modified Community Needs Index (CNI) score. ²⁴ However, no digital health metrics were included in the CNI score, nor was there an assessment of actual telehealth usage or healthcare needs of the communities. Additionally, digital health modality preference was only solicited in one study in which the participants were asked by a scheduler upon intake, with responses captured in their EHR. ²¹ Most studies echo the importance of examining digital health accessibility to properly inform interested parties on the appropriate interventions and solutions based on community metrics.

Index and method review

Various indices and methods were used across the 10 studies. Each study's methodology presented a low risk of bias and a high confidence in their reported results. Retrospective studies were the predominant research method, often aggregating patient EHRs and geographic information against an index tool, such as the Area Deprivation Index (ADI).^2,25 The others include the Community Needs Index (CNI), ²⁴ Social and Access Inequity Score (SAIS), ⁸ Digital Divide Index (DDI),^18,26 Ophelia method,^22,27 the 2-Step Virtual Catchment Area (2SVCA) ¹⁷ and Enhanced-2SVCA (E2SVCA), ¹⁹ Distressed Communities Index (DCI),^21,28 and the Theil Index.^23,29 Three studies adopted or modified indices to meet their needs. For example, Kaufman et al. modified Dignity Health's Community Needs Index by customizing and combining multiple data points from the US Census within one domain to reflect the category more accurately (e.g., the cultural domain includes ethnicity and spoken language as factors). ²⁴ Roj employed the Theil Index, which typically measures economic inequality, to measure digital health inequity within and between regions in Poland, identifying the key drivers of disparity. ²³ Furthermore, Cheng et al. implemented the Ophelia method (a 3-phase process that focuses on health literacy) to measure digital health literacy across 8 geographic clusters in Australia, co-designing customized solutions for each area. ²² The 2SVCA and E2SVCA provide a formula for measuring digital health access within a geographic region, albeit bandwidth availability and durability are the only metrics assessed.

Table 1 outlines the referenced indices and methods from the 10 studies and other relevant indices for comparison, such as the Social Vulnerability Index (SVI) ³⁰ and the Child Opportunity Index (COI). ³¹ Constructed in MS PowerPoint, Table 1's data was gathered from the index's source documentation or the relevant study in which it was implemented. Extracted data included the measurement scope, focus areas, the existence of an associated tool, and if it directly measured digital health.

Table 1.

Index and method reference table.

Index | Method	Scope of measurement	Content area	Associated tool?	Directly measures digital health suitability?
Area Deprivation Index 25	17 indicators scoring national and state regions with percentile and decile rankings at the census block group level updated annually.	Education, occupation, economic opportunities, economic resources, wealth, social resources, housing resources, transportation	Geospatial web tool	No
Child Opportunity Index 31	A composite index that assesses 44 indicators that capture the variation in opportunity across neighborhoods in the US, updated annually.	Early childhood education, elementary education, secondary and post-secondary education, education resources, pollution, healthy environments, safety- and health-related resources, economic opportunities, economic resources, concentrated inequity, housing resources, social resources, wealth	Geospatial web tool	No
Community Needs Index 24	Provides a score for every populated zip code in the USA on a scale of 1 to 5 that assesses income, culture/language, education, housing, and insurance coverage updated annually.	Modified original CNI elements with evidenced-based barriers to healthcare access via US Census data.	No	No
Digital Divide Index 26	Considers 9 indicators considering digital infrastructure and computing devices as well as SE factors at the Census tract level. This index is not routinely updated.	Bandwidth speeds, device availability, population over 65, HS education, poverty rate, disabilities, and digital inequality/internet income ratio.	Geospatial web tool	Yes
Distressed Communities Index 28	7 indicators measuring disparities in economic well-being from zip code to state level into quintile rankings normalized into a distress score between 0 and 100. Updated annually.	HS education, housing vacancy, unemployed adults, poverty rate, median income ratio, changes in employment, and changes in business establishments.	Geospatial web tool	No
Ophelia (Optimizing Health Literacy and Access) Method 27	A method for co-designing health literacy solutions to improve outcomes, access, and equity. An eHealth questionnaire was used to adopt the Ophelia process into the digital context, addressing digital health literacy needs.	eHLQ assessed the use of technology to process information, motivation to engage with digital devices, device access, digital services that suit individual needs, health literacy, and active engagement with digital services.	No	No
Social and Access Inequity Score 8	11 total measures, including 7 social and 4 accessibility factors measured at the ZCTA level to formulate a combined social inequity score.	Race, ethnicity, poverty rate, single-female HH w. children receiving food stamps, HH age 60 + on food stamps, Medicare, Medicaid, roadway access, proximity to ED/UC, broadband availability, and vehicle access.	No	Yes
Social Vulnerability Index 30	Incorporates 16 census variables that help interested parties identify communities that need support from external stressors on human health, such as disease outbreaks or natural/human-caused disasters. It is updated every two years.	Education, economic opportunities, economic resources, household composition, disability, racial/ethnic composition, housing resources, transportation	Geospatial web tool	No
Theil Index 23	Measures economic inequality on a scale of 0 to 1, with 1 indicating maximum inequality that is applied to a variety of domains to assess performance.	Examines lack of diversity, isolation, segregation, inequality, and non-randomness.	No	No
2SVCA 17	Builds upon 2SFCA to incorporate broadband accessibility to measure geographic access to telehealth.	Broadband durability and availability, accounting for download and upload speeds at patient and provider level.	Equation/Method	Yes
E2SVCA 19	Builds upon 2SVCA by using a step-wise function that more precisely reflects broadband durability at the Census tract level to fulfill typical telehealth requirements.	Divides broadband download/upload speeds into multiple categories, assigning weights to various categories.	Equation/Method	Yes

Discussion

Principal findings

Our findings indicate that no formalized index holistically measures a patient's or community's suitability for digital health services. Moreover, no enduring technological tool operates to integrate such an index for clinicians, healthcare organizations, and researchers. The various approaches considered in this systematic review offer diverse and novel ways to measure digital health accessibility. However, in isolation, they fail to capture the multitude of factors that contribute to this measure. For example, Alford-Teaster et al. and Shao et al.'s studies enhance the ability to measure broadband availability and durability accurately, but do not account for the actual usage of telehealth, cellular broadband availability within their examined regions, or include outpatient clinics that might also perform digital health services.^17,19 Additionally, 9 of the 10 studies focused at the state level or below as their geographic focus area, 3 of which were at the institution/clinic level.^16,18,21

It is unclear why studies chose one index over another to assess a specific region's deprivation or needs, as many have similar indicators. One such index is the COI. With its 44 unique indicators, the COI is objectively more inclusive and comprehensive than the ADI and SVI, capturing a profusion of elements at the community level that incorporate data sources in addition to the American Community Survey (ACS), such as the physical and school environment. ³¹ Therefore, the COI is more reflective of reality than other indices, providing insights beyond its label of child opportunity. Perhaps a change in name will broaden the COI's appeal with the research community, as its outputs exceed the bounds of its name.

Only four studies directly measure elements of digital health suitability via the DDI, SAIS, and the 2SVCA and E2SVCA methods; however, none are comprehensive in measure and scale. While the DDI incorporates relevant variables from a digital infrastructure and socioeconomic standpoint, it is not inclusive of geographic access barriers, and other elements of SDOH. Similarly, the SAIS provides an effective framework for measuring digital health, yet has no associated tool and was only implemented at the state level. Last, the 2SVCA and E2SVCA methods only measure broadband availability and durability. Taken together, these results suggest a critical gap in the ability to measure digital health suitability comprehensively. There is a need for a composite assessment tool that identifies patients and populations at risk, considers SDOH, addresses geographic barriers, reduces inequities, and tailors Virtual Care implementation based on data-driven evidence. The implementation of such a mechanism would offer a great opportunity for conducting large-scale research in the field of digital health. It would allow for easy comparison across communities, enabling policymakers and investigators to make data-driven decisions that can potentially shape the future of digital health.

Impact of implementing an Index Tool

A common theme amongst the reviewed indices was using geoinformatics solutions to represent their content visually. Geoinformatics encompasses various disciplines that combine science, technology, and geospatial information to help us better understand and visualize spatial data. ³² The healthcare industry's advanced technological and information system architecture enables the ability to comprehend the spatial dynamics that factor into improved service delivery. ³³ Geospatial technologies have been used to successfully measure physical accessibility to healthcare centers,^34,35 a critical metric contributing to the likelihood of an individual seeking care. Similar to how spatial epidemiology can visually depict the distribution of disease, ³⁶ the digital health landscape can also be depicted.

Three of the nine indices and methods discussed have an associated geospatial web tool accessible online: the ADI, DCI, and DDI. Such tools offer an interactive means by which interested parties can explore at scale the diverse needs of specific regions in the United States. Moreover, a majority of the studies reviewed integrated some form of geospatial element within their research strategy or to depict their findings visually.^{2,8,16–19,21,24} Continued use of operationalizing digital health measurements within geospatial technologies will only benefit digital health inequity reduction. The ability to visually communicate disparities regarding digital health accessibility will more broadly inform those in a position to influence positive change, improving overall healthcare accessibility and, hopefully, outcomes.

Limitations

This review included studies that developed indices and frameworks related to digital health or discussed technological tools associated with digital health, were published in English, and were published since 2014. Studies were excluded if they were not published in English or did not use an index to evaluate digital health programs and services. Additionally, we examined articles that discussed elements of digital health, such as eHealth and telehealth, with the understanding that access to digital health services is interrelated and affected by the same social determinants, geographic location, and infrastructure constraints (e.g., broadband infrastructure).

Future work

This systematic review aimed to evaluate the extent and nature of current methods examining the overall digital health suitability of individuals and communities. While numerous frameworks help shape and inform policy regarding digital health,^6,12,14,37 they have no associated index or measurement tools. Future research should aim to develop a more comprehensive index that more precisely measures digital health suitability, similar to how the SAIS and DDI integrated social determinant factors with accessibility factors (physical and digital). Moreover, accounting for the dynamic nature of certain metrics, such as temporal variations with broadband durability, ³⁸ will increase accuracy and better represent reality.

The number of nuanced elements influencing one's capacity to participate in digital health services will ensure a margin of error in any index, no matter how comprehensive. Since digital health is mostly used as a supplement to in-person care, ³⁹ another approach to improving statistical fidelity could be to mandate certain data be captured in EHRs. Similar to how Finkelstein et al. implemented a five-question screening tool the scheduler oversaw at patient intake, ²¹ comparable questions could be more widely adopted. Medical codes in ICD-10 now capture SDOH within patient EHRs, as the World Health Organization attributes up to 55% of health outcomes to these. ⁴⁰ In the same way, digital health determinants, such as internet access and digital literacy, have been referred to as “super” social determinants, as these affect the capacity to address all other determinants. ⁴¹ Such digital health metrics could be coded in an EHR to help healthcare providers understand their patients holistically and assist researchers and policymakers in identifying broader trends with this increased fidelity.

Conclusion

This systematic review surveyed the digital health landscape for tools and methods measuring digital health service suitability for individuals and communities. A thorough review of the literature has determined that no index exists that performs such measurement holistically. Furthermore, no technological tool operates to integrate such an index for clinicians, healthcare organizations, and researchers. Rather, various indices and tools are implemented to identify isolated factors contributing to digital health accessibility (e.g., broadband availability). There is a need for a comprehensive, operationalized index that accurately reflects inequitable access to digital health services. Index factors should include a combination of indicators related to socioeconomic status, digital accessibility, such as device and internet access, and social determinants of health. Together, these form the predominant driving factors related to one's ability to participate in digital health services.

This review has reaffirmed that inequity remains an issue within the digital health domain, disproportionately affecting those with low SES and those living in rural areas. More research and development must occur to bring this much-needed solution to fruition.

Abbreviations

ACS

American Community Survey

ADI

Area Deprivation Index

CINAHL

Cumulative Index of Nursing and Allied Health Literature

CNI

Community Needs Index

COI

Child Opportunity Index

DCI

Distressed Communities Index

DDI

Digital Divide Index

E2SVCA

Enhanced 2-Step Virtual Catchment Area

EHR

Electronic Health Record

Ophelia

Optimizing Health Literacy and Access

PIDAR

Partner, Identify, Demonstrate, Access, Report

SAIS

Social and Access Inequity Score

SDOH

Social Determinants of Health

SES

Socioeconomic Status

SVI

Social Vulnerability Index

2SVCA

2-Step Virtual Catchment Area