Abstract
Telehealth has the potential to improve access to health care by mitigating barriers related to geography, time, and finances. However, the increased adoption of ambulatory telehealth has inadvertently widened access gaps for socially disadvantaged and marginalized populations. Quality improvement approaches are a valuable strategy to address health care access inequities and disparities, involving data-driven implementation, assessment, and adaptation of tests of change over time. Because these iterative changes and interventions are data-driven, a critical element of quality improvement requires ongoing data collection and monitoring. This perspective describes the development and validation processes of a telehealth equity dashboard. This dashboard is currently available for use by our health system leaders, providers, and clinic staff. The overall objective of this dashboard is to identify and track inequities and to improve equitable ambulatory telehealth access across diverse patient groups. Lessons learned from creating this dashboard can inform other health care systems of how to develop and validate telehealth data feedback systems to promote quality improvement efforts to advance telehealth equity and accessibility.
Keywords: dashboard systems, digital health, health equity, telemedicine, quality improvement
Introduction
Video visits are ambulatory telehealth encounters that leverage real-time, two-way audio and video communication between patients and health care providers.1 Video visits can enhance access to health care by mitigating barriers such as geography, time, and cost.2 Many individuals favor telehealth for its convenience, reducing clinic visits, and offering flexibility for those with mobility, scheduling, and caregiver challenges.3,4 Unfortunately, research demonstrates that the increased use of ambulatory telehealth following the COVID-19 pandemic has exacerbated telehealth access inequities among populations with advanced age, low income, limited English proficiency, and low digital literacy.5–11
Quality improvement (QI) methods are well-suited to address telehealth access inequities because the contributors to these inequities are primarily at the sociocultural and health care system levels.12,13 QI utilizes data-driven approaches to implement, assess, and adapt tests of change, necessitating continuous real-time data collection and monitoring. This perspective outlines the development and validation processes of a telehealth equity dashboard. This dashboard is currently available for use by our health system leaders, providers, and clinic staff. The objective of this dashboard is to organize data to support improvement efforts that enhance equitable access to video visits for diverse patient populations. The lessons learned can inform other health care systems in developing and validating telehealth data feedback systems to promote QI efforts to advance telehealth equity and accessibility.
Designing a Dashboard for QI to Improve Equitable Telehealth Access
QI requires local buy-in
Our team conducted a mixed methods evaluation of ambulatory telehealth encounters, demonstrating lower completion rates among groups preferring languages other than English and those with public insurance.14 The study’s qualitative phase identified strong provider buy-in to address these access inequities; providers desired systematic improvement initiatives to expand telehealth access. A key recommendation was to prioritize real-time data transparency and develop a user-friendly, widely accessible dashboard for telehealth to drive QI efforts.
QI dashboard design
We formed a multi-disciplinary team to codesign an ambulatory telehealth equity dashboard. The team included members with expertise including QI, telehealth, digital equity, data curation, data validation, and project management. The dashboard includes three figures that each pertain to standard measures for assessing QI initiatives:
Outcome measure: Successfully completed telehealth visits as a percent of all completed ambulatory visits
Process measure: Scheduled telehealth visits as a percent of all scheduled ambulatory visits
Balance measure: Failed telehealth visits as a percent of scheduled telehealth visits
For each measure, the data are presented as a stacked bar graph, showing both telephone visits and video visits. The dashboard also includes a fourth section that is a run chart of the outcome measure, allowing viewers to easily visualize changes in data patterns over time. Figures 1 and 2 illustrate example data for the dashboard. Figure 1 displays the process measure categorized by race/ethnicity, and Figure 2 shows a run chart categorized by preferred language.
FIG. 1.
A graph of the process measure (“Scheduled Telehealth Visits as Percent of All Ambulatory Visits by Month”) stratified by patient race/ethnicity. The data are drawn from a random combination of clinics.
FIG. 2.
A run chart of the outcome measure (“Successfully Completed Telehealth Visits as Percent of All Completed Ambulatory Visits by Month”) stratified by patient language preference. The data are drawn from a random combination of clinics.
Measures are stratified by patient age, insurance, race and ethnicity, language preference, and neighborhood health condition.15 The dashboard also allows users to specify population, timeframe, department, clinic, and provider. It includes sharing capabilities through icons for downloading and sharing screenshots for professional collaboration. The dashboard retrieves data through structured query language (SQL) code sourced from the Epic electronic health record (EHR). The dashboard displays real-time data and is widely accessible to the members of our health system via Tableau, a secure data visualization software.
Dashboard Data Validation
Four subject matter experts who are familiar with Epic EHR documentation worked with technical experts to curate the data. Data curation involved sourcing data from the EHR and applying business rules into the SQL coding logic. For example, through discussion, the team determined what constituted a “successful” versus “failed” visit. The team defined these terms and the associated SQL coding logic through a series of collaborative sessions from January to December 2022. The team then detailed the validation process in a series of collaborative planning sessions in late December 2022. The validation objectives were to achieve parity with other similar health system reports, identify and explore potential outlier scenarios, and validate data against patient charts in the EHR. The team sampled a random selection of 200 charts for successfully completed visits and 50 charts for failed visits. The team compared data—including encounter type, patient demographics, insurance, California Healthy Places Index (CA HPI) quartile,15 visit completion status—between the dashboard and EHR, ensuring accurate data curation and preventing misinterpretation in the dashboard.
The comparison process identified discrepancies and allowed our team to resolve identified issues. Issues included discrepancies regarding race and ethnicity categorization, distinguishing gender from sex, incomplete CA HPI data, and missingness for preferred language. Our team investigated to reveal the source of each issue and subsequently adjusted the SQL code. Ultimately, validation efforts yielded 97.2% accuracy. Regarding resources, this process required an estimated 2–5 min per encounter per person. Overall, this validation process was performed from January to July 2023.
Iterative Improvements Based on End User Feedback
After optimizing data accuracy, we gathered feedback from stakeholders, including clinic and telehealth program managers, researchers, and communication specialists. We sought feedback to enhance its usability and relevance. Suggestions included adjusting verbiage, graph colors, and titles to improve readability and interpretation. We iteratively refined the dashboard content, layout, and labels until no further issues or suggestions for changes were identified.
Implications
Our telehealth equity dashboard development and validation processes illustrate the potential of dashboards to enhance telehealth accessibility in health care systems. These dashboards offer institution-specific data, helping stakeholders identify areas for improvement. This insight enables health system leaders to understand existing access disparities among patient populations, a pivotal step in QI efforts for advancing health equity.13 This strategic dashboard usage acts as an informative tool and catalyst for targeted interventions, paving the way for impactful enhancements in telehealth accessibility and service delivery.
Teams can utilize the dashboard to systematically monitor outcome, process, and balance measures across groups. Sustained data monitoring enables iterative adjustment to interventions, optimizing project outcomes. This approach underscores a commitment to adaptive strategies and continuous improvement, helping teams align with their institution’s best practices and specifical health care improvement goals for their community.
The utilization of data for tailored interventions extends beyond telehealth, permeating multiple health care areas. Clinics can adapt interventions based on their data, priorities, and resources, to support patients in underserved communities. Identifying specific factors affecting these populations is essential for optimizing telehealth and easing the burden on medical providers. This focused, adaptable approach contributes to lasting improvements that tackle immediate challenges and promote long-term improvements.
Upon widely disseminating the telehealth equity dashboard to members across the health system, our team received feedback that the dashboard is user-friendly and innovative. The dashboard has inspired various improvement initiatives. For example, one project aims to address language-based telehealth access inequities. Project leads are partnering with community organizations to conduct focus groups with individuals who have Spanish, Russian, or Cantonese language preference. Lessons learned from these focus groups will inform the key driver diagram for the QI project. The team will then conduct plan-do-study-act cycles using the dashboard’s real-time data to guide the tests of change.
In summary, the development of the telehealth equity dashboard underscores the significance of data transparency in driving continuous enhancements aimed at promoting digital health equity. We aim to carry forward the valuable lessons we’ve learned, continuing to strive to transform our health system into the most accessible and efficient resource it can be.
Authors’ Contributions
L.R.M. co-designed the dashboard, assisted with conceptualization of the perspective piece, drafted and revised the article, and approved the final version of the article. J.P.M. co-designed the dashboard, assisted with conceptualization of the perspective piece, revised the article, and approved the final version of the article. Z.W. co-designed the dashboard, revised the article, and approved the final version of the article. K.M.L. co-designed the dashboard, revised the article, and approved the final version of the article. M.A. co-designed the dashboard, revised the article, and approved the final version of the article. J.L.R. co-designed the dashboard, led the conceptualization of the perspective piece, revised the article, and approved the final version of the article.
Author Disclosure Statements
The authors have no conflicts of interest to disclose.
Funding Information
The project described was supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development, National Institutes of Health (grant number K23HD101550 to Dr. Rosenthal). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
Cite this article as: Meisel LR, Marcin JP, Wu Z, Lopez KM, Avdalovic M, and Rosenthal JL. (2024) Advancing equitable ambulatory telehealth through dashboard development, Telemedicine Reports 5:1, 219–223, DOI: 10.1089/tmr.2024.0033.
References
- 1. Gelburd R. Examining the State of Telehealth During the COVID-19 Pandemic. United Hospital Fund; 2021. [Google Scholar]
- 2. Myers CR. Using telehealth to remediate rural mental health and healthcare disparities. Issues Ment Health Nurs 2019;40(3):233–239. [DOI] [PubMed] [Google Scholar]
- 3. Anaya YBM, Mota AB, Hernandez GD, et al. Post-pandemic telehealth policy for primary care: An equity perspective. J Am Board Fam Med 2022;35(3):588–592. [DOI] [PubMed] [Google Scholar]
- 4. Lyu M, Zhao Q, Yang Y, et al. Benefits of and barriers to telehealth for the informal caregivers of elderly individuals in rural areas: A scoping review. Aust J Rural Health 2022;30(4):442–457. [DOI] [PubMed] [Google Scholar]
- 5. Rodriguez JA, Saadi A, Schwamm LH, et al. Disparities in telehealth use among California patients with limited English proficiency: Study examines disparities in telehealth use among California patients with limited English proficiency. Health Aff (Millwood) 2021;40(3):487–495. [DOI] [PubMed] [Google Scholar]
- 6. Westby A, Nissly T, Gieseker R, et al. Achieving equity in telehealth: “Centering at the margins” in access, provision, and reimbursement. J Am Board Fam Med 2021;34(Suppl):S29–S32. [DOI] [PubMed] [Google Scholar]
- 7. CDC. Using Telehealth to Expand Access to Essential Health Services during the COVID-19 Pandemic. Available from: https://www.cdc.gov/coronavirus/2019-ncov/hcp/telehealth.html [Last accessed: December 16, 2020].
- 8. Edmiston KD, AlZuBi J. Trends in telehealth and its implications for health disparities. Center for Insurance Policy & Research Report. Natl Assoc Insur Comm 2022. [Google Scholar]
- 9. Pasquinelli MM, Patel D, Nguyen R, et al. Age-based disparities in telehealth use in an urban, underserved population in cancer and pulmonary clinics: A need for policy change. J Am Assoc Nurse Pract 2022;34(5):731–737. [DOI] [PubMed] [Google Scholar]
- 10. Rivera V, Aldridge MD, Ornstein K, et al. Racial and socioeconomic disparities in access to telehealth. J Am Geriatr Soc 2021;69(1):44–45. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Blundell AR, Kroshinsky D, Hawryluk EB, et al. Disparities in telemedicine access for Spanish-speaking patients during the COVID-19 crisis. Pediatr Dermatol 2020;38(4):947–949. [DOI] [PubMed] [Google Scholar]
- 12. Lion KC, Raphael JL. Partnering health disparities research with quality improvement science in pediatrics. Pediatrics 2015;135(2):354–361. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Lion KC, Faro EZ, Coker TR. All quality improvement is health equity work: Designing improvement to reduce disparities. Pediatrics 2022;149(Suppl 3). [DOI] [PubMed] [Google Scholar]
- 14. Rosenthal JL, O’Neal C, Sanders A, et al. Differential use of pediatric video visits by a diverse population during the COVID-19 pandemic: A mixed-methods study. Front Pediatr 2021;9:645236. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Maizlish N, Delaney T, Dowling H, et al. California healthy places index: Frames matter. Public Health Rep 2019;134(4):354–362. [DOI] [PMC free article] [PubMed] [Google Scholar]