Abstract
The purpose of this study is to describe telehealth experiences and quality of HIV care provided to an urban population of people with HIV (PWH) in Washington, DC. We used self-reported survey data from a cohort of PWH in the DC Cohort longitudinal study linked to medical records (October 26, 2020–December 31, 2021). Analyses followed a mixed-methods approach, including prevalence estimates and multivariable logistic regression of telehealth use by demographic and HIV characteristics. We measured primary motivation, modes of engagement, and telehealth satisfaction. Qualitative responses to open-ended questions were coded using collaborative coding. A framework developed by the National Quality Forum (NQF) was applied to the results. Among 978 participants, 69% reported using telehealth for HIV care during the pandemic. High school graduates were less likely to use telehealth compared to those with college education (aOR 0.69, 95% CI 0.48, 0.98). PWH with > 1 co-morbid condition were more likely to use telehealth compared to those without (aOR 1.42, 95% CI 1.02, 1.95). The majority reported satisfaction with telehealth (81%). Qualitative analysis of telehealth satisfaction found that most responses were related to access to care and technology, effectiveness, and patient experience. PWH using telehealth during the pandemic were satisfied with their experience though use differed demographically. Telehealth was used effectively to overcome barriers to care engagement, including transportation, costs, and time. As we transition away from the emergency pandemic responses, it will be important to determine how this technology can be used in the future in an equitable manner to further strengthen HIV care engagement.
Keywords: Telehealth and medicine, COVID-19, HIV/AIDS
Introduction
The COVID-19 pandemic dramatically changed the practice and delivery of medical care [1]. Telehealth was used historically to provide HIV care in rural settings and pre-exposure prophylaxis in settings with limited access and slowly transitioned to urban settings [2]. Prior to the COVID-19 pandemic it was estimated that only 15% of physicians utilized telehealth [3]. However, during the pandemic, telehealth was rapidly scaled to maintain continuity of care [4] with private insurers allowing reimbursement and changing restrictive telehealth polices. Medicare telehealth visits increased greatly from 0.1 to 44% during the year 2020 [5]. The Infectious Disease Society of America, National Institutes of Health, and US Centers for Disease Control and Prevention separately issued guidelines for scale-up of telehealth for HIV care delivery [5–8].
Despite the above guidelines, reviews of telehealth quality of care and experience in the general population have been mixed. In a pre-pandemic study, providers perceived telehealth as lessening patient barriers to care, yet this advantage was were countered by technical difficulties and perceived lower quality of care [9]. A study describing telehealth trends during the pandemic indicated high levels of patient satisfaction [10]. Studies of telehealth among persons with HIV (PWH) are conflicting, with some showing engagement in care remaining constant while others demonstrating more patients lost to follow up (LTFU) during the pandemic [11]. A study of PWH demonstrated an increase in patients LTFU and decrease in medical visit frequency, yet viral load suppression (VS) remained high [11]. A large urban study showed maintenance of engagement in HIV care with telemedicine based on visits and VS [12]. One mixed methods study of a large urban HIV cohort during the pandemic found that telehealth was helpful in retention in care, re-engaging patients out of care, delivering patient-centered care, and engaging patients’ family members and multidisciplinary teams [13]. Disadvantages included technology challenges, privacy concerns, loss of routine clinical experience and in-person interactions, physical exam and remote monitoring of symptoms, and reimbursement concerns [13].
Given the inconsistency of data on telehealth and the scarcity of studies evaluating telehealth specifically among PWH, we sought to quantify the use of and satisfaction with telehealth among a cohort of PWH in Washington, District of Columbia (DC) using a mixed-methods approach. Quantitative analyses of telehealth usage, mode, and satisfaction were supplemented with qualitative analyses utilizing the National Quality Framework (NQF) framework as basis for measuring quality of care delivered by telehealth during the pandemic [14].
Methods
Data for this mixed-methods analysis came from responses to a cross-sectional COVID-19 survey of participants in the DC Cohort study, a longitudinal HIV cohort study of PWH receiving care at 14 HIV clinics in Washington, DC. Detailed information about the DC Cohort has been published previously [15]. In brief, the DC Cohort aggregates electronic medical record (EMR) data from 14 participating HIV clinics, including labs, medical visits, prescription drugs, and social and demographic characteristics. Participants’ survey data were linked to the DC Cohort database.
Of the 14 clinics included in the DC Cohort, there are 6 community clinics and 8 hospital-based clinics. Based on internal surveys conducted prior to the pandemic, community clinics report seeing an average of 71 patients per week while hospital clinics see an average of 60 patients per week. Eighty-nine percent of hospital-based clinics have an onsite pharmacy compared to only 50% of community-based clinics. Community and hospital-based clinics are similar in their offering of urgent care, case management, and transportation services. A higher proportion of community clinics compared to hospital-based clinics offer substance abuse counseling (83% vs. 38%, respectively) and opioid treatment programs (50% vs. 13%, respectively). A higher proportion of hospital-based clinics offer nurse navigation (75% compared to 33% of community-based clinics).
The COVID-19 survey was initiated on October 26, 2020 through May 31, 2023. DC Cohort participants ages 18 and older from the 13 adult HIV clinics included in the DC Cohort (one adolescent clinic was excluded) were invited to complete an electronic survey about their COVID-19 experiences. Survey participants were compensated with a $25 gift card. The survey was available in English and Spanish. The survey was created and managed in Research Electronic Data Capture (REDCap) [16, 17]. Electronic informed consent was obtained, and the survey was approved by the George Washington University Institutional Review Board (IRB) and specific sites’ IRBs (Washington DC Department of Veterans Affairs, Washington Hospital Center, Children’s National Medical Center). Data collected on the survey [18–25] included socio-demographic information; medical conditions; COVID-19 exposures, symptoms, testing, and stigma; depressive and anxiety symptoms; telehealth experiences; tobacco use; sexual risk factors; and ART adherence. We conducted a convergent mixed-methods analysis [26] using survey data through December 31, 2021, linked to the Cohort database to quantify telehealth use among PWH by socio-demographic groups, mode, and satisfaction. This end date was chosen based on data availability at the time of analyses.
Measures
Quantitative measures from the COVID-19 survey included use of telehealth for HIV-related care, sociodemographic variables, household composition, housing status, state of residence, clinic type, underlying medical conditions, telehealth use, access experience, and satisfaction for HIV care. HIV-related measures from the DC Cohort database included years since HIV diagnosis, HIV mode of transmission, CD4 cell count (cells/μL at last measured lab value since January 1, 2020) and HIV RNA suppression (suppressed defined as < 20 copies/mL at last measured time point since January 1, 2020). Qualitative analysis included participant responses to questions about their telehealth usage during the pandemic; positive responders were asked about their mode of telehealth use, motivations for using telehealth, and engagement mode.
Quantitative Statistical Analyses
Quantitative analyses included prevalence estimates of socio-demographic and HIV-related measures stratified by use of telehealth and unadjusted odds ratios and 95% confidence intervals (95% CI) of the association between telehealth and socio-demographic characteristics. Adjusted odds ratios and 95% CI were calculated using multivariable logistic regression analyses to investigate the association between telehealth and HIV-related outcomes controlling for socio-demographic variables. Statistical significance was set at p < 0.05. Quantitative analyses were conducted using SAS 9.4 (Cary, NC, USA).
Qualitative Statistical Analyses
The NQF framework, developed in 2017 and updated in 2021, which measures quality and impact of care provided by telehealth, was used for categorizing qualitative codes [14]. This framework focuses on rural areas and includes five domains of quality measurement: access to care and technology; costs, business models, and logistics; experience; effectiveness; and equity. To our knowledge, no similar framework exists for assessing telehealth in urban healthcare systems; however, these five measurement domains are relevant to urban populations.
Qualitative analyses were performed based on responses to three open-ended questions for survey participants who reported an HIV-related telehealth visit during the COVID-19 pandemic (downsides, advantages, and additional comments/suggestions related to telehealth) using Atlas.ti (version 7) [27]. Open-ended responses to each of the qualitative questions were initially coded for responses between October 30, 2020 and November 1, 2021 by authors SB and KS using a collaborative coding method, and categorized into multiple codes when appropriate. Inductive analysis was performed utilizing open coding simultaneously via virtual conference, and dialogical intersubjectivity was utilized to reach consensus [28]. Duplicative and synonymous codes were reviewed for merging as appropriate, and a coding frame was finalized. Secondary coders (AC and GB) acted as sense-checkers for the coded dataset. Subsequently, data from November 2, 2021 to December 31, 2021 were coded using the coding frame by SB and KS and saturation was achieved.
Results
Quantitative Findings
Among survey participants (n = 978), there were 678 participants who reported using telehealth for HIV care (69.3%) (Table 1). All survey participants submitted their surveys electronically through the online survey. In bivariable analysis, there were no statistically significant differences in telehealth usage by age, race, education status, household composition, or location or residence, median years since HIV diagnosis, viral suppression status, CD4 count, or number of underlying medical conditions. Differences by clinic site type (community clinic versus hospital clinic) are shown in Supplemental Table 1.
Table 1.
Frequency counts of demographic and HIV characteristics of DC Cohort COVID-19 Survey Participants by self-reported telehealth visit status
| Characteristic | Totala (N = 978) n (%) |
Had a telehealth visit (n = 678, 69.3%) n (col %) |
Did not have a telehealth visit (n = 300, 30.7%) n (col %) |
Chi-square testb | p value |
|---|---|---|---|---|---|
| Sociodemographics | |||||
| Age (median, IQR, Wilcoxon p-value) | 54 (43.0, 61.0) | 54.0 (43.0, 61.0) | 54.0 (44.0, 60.0) | 0.81 | |
| Gender identity | |||||
| Female | 314 (32.1) | 235 (34.7) | 79 (26.3) | 15.72 | 0.003 |
| Male | 640 (65.4) | 421 (62.1) | 219 (73.0) | ||
| Transgender/genderqueer/gender nonconforming | 19 (1.94) | 18 (2.65) | 1 (0.33) | ||
| Declined to state | 3 (0.3) | 3 (0.4) | 0 (0.0) | ||
| Missing | 2 (0.2) | 1 (0.2) | 1 (0.3) | ||
| Race/ethnicity | |||||
| Non-Hispanic White | 129 (13.2) | 85 (12.5) | 44 (14.7) | 1.19 | 0.88 |
| Non-Hispanic Black | 723 (73.9) | 505 (74.5) | 218 (72.7) | ||
| Hispanic | 53 (5.4) | 38 (5.6) | 15 (5.0) | ||
| Otherc | 48 (4.9) | 32 (4.7) | 16 (5.3) | ||
| Missing | 25 (2.6) | 18 (2.7) | 7 (2.3) | ||
| Education | |||||
| Less than high school education | 92 (9.4) | 70 (10.3) | 22 (7.3) | 4.56 | 0.21 |
| High school graduate | 280 (28.6) | 190 (28.0) | 90 (30.0) | ||
| At least some college | 591 (60.4) | 405 (59.7) | 186 (62.0) | ||
| Missing | 15 (1.5) | 13 (1.9) | 2 (0.7) | ||
| Work status (as of 1/1/20) | |||||
| Employed full- or part-time | 541 | 355 (52.4) | 186 (62.0) | 17.43 | 0.002 |
| Unemployed | 129 | 91 (13.4) | 38 (12.7) | ||
| Disabled/retired | 268 | 194 (28.6) | 74 (24.7) | ||
| Otherd | 32 | 31 (4.6) | 1 (0.3) | ||
| Missing | 8 | 7 (1.03) | 1 (0.3) | ||
| Household composition | |||||
| Lives alone | 420 (42.9) | 283 (41.7) | 137 (45.7) | 1.58 | 0.45 |
| Lives with others | 553 (56.5) | 392 (57.8) | 161 (53.7) | ||
| Missing | 5 (0.5) | 3 (0.4) | 2 (0.7) | ||
| Housing status | |||||
| Own | 234 (23.9) | 140 (20.7) | 94 (31.3) | 14.46 | 0.002 |
| Rent | 573 (58.6) | 411 (60.6) | 162 (54.0) | ||
| Homeless | 10 (1.0) | 6 (0.9) | 4 (1.3) | ||
| Othere | 161 (16.5) | 121 (17.9) | 40 (13.3) | ||
| Location of residence | |||||
| District to Columbia | 735 (75.2) | 522 (77.0) | 213 (71.0) | 10.30 | 0.07 |
| Maryland | 181 (18.5) | 114 (16.8) | 67 (22.3) | ||
| Virginia | 48 (4.9) | 33 (4.9) | 15 (5.0) | ||
| Other | 6 (0.6) | 2 (0.3) | 4 (1.3) | ||
| Declined to state | 5 (0.5) | 5 (0.7) | 0 (0.0) | ||
| Missing | 3 (0.3) | 2 (0.3) | 1 (0.3) | ||
| Clinic type | |||||
| Hospital | 324 (33.1) | 158 (23.3) | 166 (55.3) | < 0.0001 | |
| Community | 641 (65.5) | 516 (76.1) | 125 (41.7) | ||
| Missing | 13 (1.3) | 4 (0.6) | 9 (3.0) | ||
| HIV and other medical conditions | |||||
| Median duration of HIV diagnosis (median years, IQR, Wilcoxon p-value) | 17.0 (11.0, 24.0) | 16.0 (11.0, 24.0) | 17.5 (12.0, 25.0) | 0.39 | |
| Viral load as of 1/1/20f | |||||
| Undetected | 497 (49.8) | 352 (51.9) | 145 (48.3) | 7.29 | 0.06 |
| Not undetected | 209 (21.4) | 154 (22.7) | 55 (18.3) | ||
| No HIV RNA labs as of 1/1/20 (labs before this date) | 186 (19.0) | 119 (17.6) | 67 (22.3) | ||
| Missing labs | 86 (8.8) | 53 (7.8) | 33 (11.0) | ||
| CD4 ≥ 200 cells/μL as of 1/1/20 | |||||
| CD4 ≥ 200 cells/μL | 715 (73.1) | 505 (74.5) | 210 (70.0) | 3.83 | 0.28 |
| CD4 < 200 cells/μL | 32 (3.3) | 23 (3.4) | 9 (3.0) | ||
| No CD4 labs as of 1/1/20 (labs before this date) | 177 (18.1) | 118 (17.4) | 59 (19.7) | ||
| Missing | 54 (5.5) | 32 (4.7) | 22 (7.3) | ||
| HIV mode of transmission | |||||
| Men who have sex with men (MSM) | 403 (41.2) | 260 (38.4) | 143 (47.7) | 19.27 | 0.002 |
| High risk heterosexual | 253 (25.9) | 186 (27.4) | 67 (22.3) | ||
| Injection drug use (IDU) | 43 (4.4) | 32 (4.7) | 11 (3.7) | ||
| Perinatal | 13 (1.3) | 7 (1.0) | 6 (2.0) | ||
| Otherg | 216 (22.1) | 164 (24.2) | 52 (17.3) | ||
| Missing | 50 (5.1) | 29 (4.3) | 21 (7.0) | ||
| Number of self-reported underlying medical conditions | |||||
| 0 | 299 (30.6) | 193 (28.5) | 106 (35.3) | 2.12 | 0.15 |
| ≥ 1 | 679 (69.4) | 485 (71.5) | 194 (64.7) | ||
Totals may not sum to N due to missing data
Chi-square or Wilcoxon test
Other includes American Indian/Alaska native, Asian, Native Hawaiian/Pacific Islander, Bi/multiracial
Other includes student, homemaker, other
Other includes lives with parent/friends, lives in rooming/halfway/group home, lives in residential drug facility, lives in assisted living, other
Viral suppression defined as HIV RNA < 20 copies/mL at last measured
Other includes Hemophilia, blood transfusion, other, unknown
In the unadjusted logistic regression analysis, females were more likely to use telehealth compared to males (OR 1.55, 95% CI 1.14, 2.09) (Table 2). Those who rented their homes or were in the ‘other’ living category (e.g., lives with parent/friends, rooming/halfway/group home, residential drug facility, assisted living) were more likely to use telehealth compared to those who owned their homes in the unadjusted model (OR 1.70, 95% CI 1.24, 2.34; OR 2.03, 95% CI 1.30, 3.16, respectively). In adjusted models, high school graduates were less likely to have reported a telehealth visit compared to those with some college education (aOR 0.63, 95% CI 0.43, 0.92). Participants who used community clinics for their HIV care were nearly four times more likely to have reported using telehealth compared to those receiving care at a hospital-based clinic (aOR 3.96, 95% CI 2.88, 5.44). Participants with no CD4 labs on or after 1/1/2020 were more likely to report a telehealth visit compared to those with CD4 labs ≥ 200 cells/μL on or after 1/1/2020 (aOR 1.54, 95% CI 1.02, 2.32). Participants reporting at least one comorbid condition were more likely to report having a telehealth visit compared to those with no comorbid conditions (aOR 1.54, 95% CI 1.09, 2.17). More specifically, there were no differences in telehealth usage among those reporting hypertension, smoking, overweight/obese, asthma, cancer, chronic lung disease, chronic kidney disease, cardiovascular disease, or cerebrovascular disease (data not shown). Those with diabetes mellitus type 2 were more likely to report using telehealth for HIV visits compared to those without (aOR 1.93, 95% CI 1.14, 3.25).
Table 2.
Unadjusted and adjusted HIV characteristics of DC Cohort COVID-19 Survey Participants by self-reported telehealth visit status
| Characteristic | Unadjusted odds ratios | 95% confidence interval | Adjusted odds ratios | 95% confidence interval |
|---|---|---|---|---|
| Sociodemographics | ||||
| Age (per 5 years)a | 1.00 | 0.95, 1.06 | 1.01 | 0.94, 1.08 |
| Gender identitya | ||||
| Female | 1.55 | 1.14, 2.09 | 1.38 | 0.96, 2.00 |
| Male | Ref | Ref | ||
| Race/ethnicitya | ||||
| Non-Hispanic White | Ref | Ref | ||
| Non-Hispanic Black | 1.20 | 0.81, 1.78 | 0.84 | 0.52, 1.33 |
| Hispanic | 1.31 | 0.65, 2.64 | 1.03 | 0.48, 2.21 |
| Otherc | 1.04 | 0.51, 2.09 | 0.80 | 0.36, 1.77 |
| Educationa | ||||
| Less than high school education | 1.46 | 0.88, 2.43 | 1.00 | 0.53, 1.89 |
| HS graduate | 0.97 | 0.72, 1.32 | 0.63 | 0.43, 0.92 |
| At least some college | Ref | Ref | ||
| Work status (as of 1/1/20)a | ||||
| Employed full- or part-time | Ref | Ref | ||
| Unemployed | 1.26 | 0.83, 1.91 | 0.95 | 0.57, 1.56 |
| Otherd | 1.57 | 1.15, 2.16 | 1.33 | 0.90, 1.97 |
| Household compositiona | ||||
| Lives alone | Ref | Ref | ||
| Lives with others | 1.18 | 0.90, 1.55 | 1.11 | 0.80, 1.54 |
| Housing statusa | ||||
| Own | Ref | Ref | ||
| Rent | 1.70 | 1.24, 2.34 | 1.37 | 0.92, 2.05 |
| Homeless | 1.01 | 0.28, 3.67 | 1.07 | 0.10, 10.92 |
| Othere | 2.03 | 1.30, 3.16 | 1.59 | 0.92, 2.75 |
| Location of residencea | ||||
| DC | Ref | Ref | ||
| Otherf | 0.73 | 0.54, 0.99 | 0.99 | 0.68, 1.42 |
| Clinic type | ||||
| Hospital | Ref | Ref | ||
| Community | 4.34 | 3.24, 5.81 | 3.96 | 2.88, 5.44 |
| HIV and other medical conditions | ||||
| Median duration of HIV diagnosis (per 5 years) | 0.99 | 0.92, 1.08 | 1.04 | 0.93, 1.15 |
| Viral load suppressedg as of 1/1/20b | ||||
| Unsuppressed | Ref | Ref | ||
| Suppressed | 1.15 | 0.80, 1.66 | 0.89 | 0.58, 1.34 |
| No HIV RNA labs as of 1/1/20 | 0.73 | 0.51, 1.05 | 1.16 | 0.77, 1.76 |
| CD4 ≥ 200 cells/μL as of 1/1/20b | ||||
| CD4 ≥ 200 cells/μL | Ref | Ref | ||
| CD4 < 200 cells/μL | 1.06 | 0.48, 2.34 | 1.60 | 0.63, 4.07 |
| No CD4 labs as of 1/1/20 | 0.83 | 0.59, 1.18 | 1.54 | 1.02, 2.32 |
| HIV mode of transmissionb | ||||
| Men who have sex with men (MSM) | Ref | Ref | ||
| High risk heterosexual | 1.53 | 1.08, 2.16 | 1.47 | 0.87, 2.50 |
| Otherh | 1.62 | 1.15, 2.28 | 1.53 | 0.97, 2.39 |
| Number of self-reported underlying medical conditionsb | ||||
| 0 | Ref | Ref | ||
| ≥ 1 | 1.37 | 1.03, 1.83 | 1.54 | 1.09, 2.17 |
Model includes all demographic variables
Model adjusts age, gender, race/ethnicity, employment status, housing composition, housing status, state of residence
Other includes American Indian/Alaska native, Asian, Native Hawaiian/Pacific Islander, Bi/multiracial
Other includes student, homemaker, disabled, retired, other
Other includes lives with parent/friends, lives in rooming/halfway/group home, lives in residential drug facility, lives in assisted living, other
Other includes MD, VA, other, declined, missing
Viral suppression defined as HIV RNA < 20 copies/mL at last measured
Other includes Hemophilia, blood transfusion, other, unknown, perinatal
Among the 678 participants who reported telehealth usage, 31% reported they used telehealth because it was offered, recommended, or required by their healthcare provider (Supplemental Fig. 1). Seventeen percent reported using telehealth because it was faster, more convenient, and comfortable; was easier for follow up care; and reduced their risk of COVID-19 exposure.
When asked how they interacted with providers, most participants reported using telephone calls (45.7%) or a phone video call 38.6% (Fig. 1). We found differences in telehealth mode by gender, household income, race/ethnicity, and age group. Males reported using video calls on their phone more than other genders (27.1%, chi-square = 14.65, p = 0.0021). Telehealth use via telephone call was more common in lower income PWH (household income < $25,000, 58.2%, chi-square = 58.03, p < 0.001). Fifty-two percent of those in the $100–149,000 income group reported using video calls on a computer, a proportion that differed significantly from other income groups (chi-square = 58.03, p < 0.001). Among Hispanics, 44.7% reported using video calls on a computer to engage with providers, a higher proportion compared to other race/ethnicities (chi-square = 23.71, p < 0.001). Sixty-four percent of those in the 65 and older age group reported using telephone calls to engage with providers, a greater proportion than other ages (chi-square = 28.83, p < 0.001).
Fig. 1.
Telehealth mode by demographic groups
Participants were asked to rate their telehealth experiences (Supplemental Fig. 2). Among 678 participants, the majority reported that they would use telehealth in the future (74.1%), that they were satisfied with their telehealth experience (81.0%), and that using telehealth was convenient (86.4%). Using a 4-point Likert scale, slightly more than half agreed that telehealth was as good as an in-person visit (54.3%), while 26.1% disagreed. Among those using video for telehealth (n = 399), most participants disagreed that being on camera made them feel embarrassed or uncomfortable (69.4%); 10% worried about their privacy during telehealth sessions.
Qualitative Findings
Of the 678 survey participants reporting telehealth utilization, there were 917 open-text responses regarding the disadvantages and concerns, advantages, and suggestions for improving telehealth. Results for the five major categories included in the NQF framework are presented along with example quotations (Fig. 2, Table 3). The greatest frequency of coded responses fit into the categories of experience (n = 236, 31.0%), effectiveness (n = 231, 30.3%), and access to care and technology (n = 225, 29.5%). Costs, business models, and logistics represented 6.3% (n = 48) and equity-related responses represent 2.9% (n = 22) of all codes.
Fig. 2.
Qualitative findings applied to the NSQ framework for telehealth priorities using data from the DC Cohort COVID survey, 2020–2021
Table 3.
Selected example quotes categorized into NSQ framework domains
| Advantages/support | Disadvantages/concerns |
|---|---|
| Access to care and technology | |
| “The greatest advantage is the shorten wait time to see my care provider.” | “Concerned about how secure these communications can actually be in light of today’s continual news about computer hacking, etc.” |
| “The advantage is I don’t have to stand in line to get in the clinic there’s no wait time other than to sit and wait for my doctor to call which is also pleasant to hear his voice” | “Worried about getting text messages” |
| Costs, business models, logistics | |
| “Without a doubt, follow up for lab results. There is no need to have to drastically alter one’s schedule for something so mundane. When I used to work, I had to use up precious few sick days/PTO to take off for appointments that honestly only took up five minutes.” | “Maybe adding some remote diagnostics to the telehealth visit might improve the experience.” |
| “It takes away the stress of traveling to doctor’s appointments, costs of gas and parking, etc.” | |
| Experience | |
| “The personal face-to-face connection is missing, but the convenience of beingable to have a telehealth visit without too much interruption from my normal daily work is far more valuable to me.” | |
| “Distracted me, I forgot to, ask important questions or listen carefully” | |
| “Eye to eye connection in person is better for me. There are things that you see and feel when in person.” | |
| “It’s impersonal. I think it would be difficult to diagnose patients without physically touching them.” | |
| “A routine HIV Checkup visit without any lab results doesn’t leave anything but how I am feeling to discuss. Now I have been waiting for a month for the call to get my blood draw appointment. Seems like I fell through the cracks.” | |
| Effectiveness | |
| “Able to keep appointments easier, easy access, convenient not having to leave the house.” | “Can’t track my vital signs and weight” |
| “Convenience, lack of need to travel, time-saving for routine visits with no major needs required. Would be OK for interviews for pre-colonoscopy meeting, for example, as opposed to using all that time traveling to & from” | |
| “I travel 50 miles to [clinic] so it is convenient Works great for follow-up appointments especially when there has been little or no change in your condition” | |
| “It’s great for circumstances that don’t require examinations or close physical inspections.” | |
| Equity | |
| “I have arthritis and walk with a walker. Doing telehealth is convenient for me as I don’t have to find someone to take me to the clinic.” | “Please gather information that would maximize telehealth, to include access to computer or phone with said technology.” |
| “Very critical breakthrough for persons with disabilities.” | “I have fast and reliable internet, so the connection quality video and audio was great. But not everyone is fortunate enough to have reliable internet, so it’s probably not a feasible option for many.” |
| “I could be anywhere to take the call. Even though sometimes I come in and get vouchers for example” | |
Access to Care and Technology
There were 225 coded responses in the category of access to care and technology. Codes falling under the sub-category of clinical use cases included COVID-19 safety, positive comments about staying at home and not having to be around people (n = 166, 73.8%). Eighteen percent (n = 40) of coded responses related to the geographic distance and travel sub-category including flexibility in choosing appointment times, not having to come into the doctor’s office, and positive commuting comments. A small proportion of comments were related to telehealth technology and broadband issues sub-categories: 3.6% (n = 8) were about Wi-Fi-specific issues, 4.4% (n = 10) were about telehealth technology and capacity for communication.
Approximately 6% (n = 48) of coded responses were related to costs, business models and logistics. Fifty-two percent (n = 25) of codes fell under the sub-category of adaptability and system readiness (positive and negative comments about technology), and 48% (n = 23) were categorized as costs to patients, caregivers, and insurers (insurance issues and both negative and positive comments related to telehealth cost effectiveness).
There were 236 (31%) coded responses related to the telehealth experience category. Seventy-four percent (n = 174) of this category included codes in the patient experience with telehealth sub-category. Within the patient experience and telehealth sub-category, 18% (n = 31) were coded as “loss of intimacy with provider”, 48% (n = 83) were coded as psychological-related (positive and negative comments about stress, anxiety, comfort with telehealth visits), and 9% (n = 15) were related to a positive provider relationship with telehealth visits. A small percentage were related to clinician experiences with telehealth (n = 3, 1.3%). Nearly 19% (n = 44) of comments under the experience sub-category were related to patient choice (remote versus in-person visits), with most of these comments (n = 28, 64%) coded as patients preferring in-person visits to telehealth. Just over 6% (n = 15) were related to patient trust of the health care system, with the majority regarding telehealth-related privacy concerns.
Thirty percent (n = 231) of all coded comments were related to the effectiveness of telehealth. Eighty-five percent (n = 194) of these comments were about time to care delivery and receipt of specific care. Most were positive in nature and related to convenience, efficiency of visit, and saving personal time, and positive access to their doctor. Sixty-one percent (n = 141) of comments were coded into the sub-category of quality of care for clinical issues addressable through telehealth. Fifteen percent (n = 34) of the comments were related to planning around clinical issues not addressable through telehealth, including labs, difficulty accessing medications, vital sign measurements, and testing.
The prevalence of responses falling into the equity category was minimal (n = 22, 2.9%). Most related to the sub-category how quality of care and outcomes differ by intersection of factors, such as elimination of health barriers, patient not missing work, and children being at home (n = 18, 82%). Some participants mentioned a telehealth benefit being the removal of transportation barriers, falling into the social determinants of health sub-category.
Discussion
In this large urban population of PWH, there was high use of telehealth with most participants reporting satisfaction and plans for future use. We found that PWH with higher education, those receiving HIV care at community clinics versus hospitals, those with no CD4 labs after January 2020 were more likely to use telehealth. Those with at least one comorbid condition were more likely to use telehealth in adjusted models; consistent access and adherence to medications during the pandemic among those reporting no telehealth usage may help explain this observation. Conversely, those accessing telehealth may have experienced challenges getting labs and prescriptions as supported by the qualitative analysis. Additionally, in our analysis, telehealth use did not differ significantly based on race, gender, or HIV measures such as viral load, or mode of transmission. These results differ from other studies that reported disparities in telehealth usage in the general population, with decreased use among Blacks and older ages [29, 30]. In contrast to our findings, one study in an urban HIV setting found greater telehealth usage among older persons compared to younger and Whites compared to Blacks [31]. These findings warrant further exploration, especially among patients with comorbidities where telehealth may facilitate keeping patients engaged in care.
We noted demographic and clinical differences between community and hospital-based clinics in Supplemental Table 1. Females, non-Hispanic Blacks, unemployed, and those in lower education groups were more likely to utilize community-based HIV care. Additionally, we found that PWH receiving care at community clinics were more likely to use telehealth compared to those at hospital-based clinics in unadjusted and adjusted models. These findings indicate that increased offering of telehealth as an HIV care option in community-based clinics fulfills a need for demographic groups at greatest need for easily accessible HIV care.
As we consider the future of telehealth use and how to integrate it into routine patient care, it is important to consider which patients may benefit from continued access. Importantly, over half of participants felt that telehealth visits were just as good as in-person visits. In the context of the pandemic, PWH with comorbidities were more likely to engage in telehealth, which was further confirmed with a large percentage of qualitative participants discussing increased perception of COVID-19 safety by utilizing telehealth. Additionally, in bivariable analyses, we found that females and those who may be predominantly at home (retired, disabled, homemakers) utilized telehealth more frequently, though these findings were attenuated when adjusting for other socio-demographic variables. Competing priorities such as childcare responsibilities may explain the increased telehealth use among women and homemakers. Those who are retired or disabled may have mobility or transportation challenges which may increase their likelihood of using telehealth compared to those who are employed. PWH with less stable housing were more likely to utilize telehealth in unadjusted models, suggesting that telehealth may be a tool to promote follow-up in some high-need patients [32].
While telehealth use and satisfaction were high, distinct differences in mode of telehealth were observed. We found that very low-income patients as well as non-Hispanic Whites and non-Hispanic Blacks used the telephone (audio) as their telehealth platform. Hispanics, younger age groups, and those in higher income groups were more likely to use video calls compared to other races, older PWH, and those in lower income groups, who were more likely to use telephone calls for telehealth. These differences highlight potential disparities in technology access and may widen the digital divide [29, 33–35]. To counter this growing divide, Grove et al. [33], suggest providing devices and internet options along with related training as part of a hybrid model of care focusing on in-person visits for rapport-building and telehealth instruction with virtual follow-up visits. A successful hybrid model would consider the frequency of minimum necessary in-person HIV-related visits for purposes of checking vital signs, performing physical exams, and collecting lab specimens.
We found the NQF framework a good fit for analyzing the qualitative responses in this large urban cohort. Most participants mapped to the quality markers of experience, effectiveness, and access to care. In the urban environment, many patients remarked telehealth removed travel barriers. Participants largely felt that telehealth provided good and timely access to their care delivery; however, this was balanced by some patient-centered concerns that aspects of care (e.g. laboratory testing), are not easily addressed by telehealth, which is consistent with findings from other telehealth studies [32, 36].
Our mixed-methods approach indicates that in our large urban population of PWH, telehealth was well-received during the pandemic and may continue to be used in the future. Utilizing the NQF framework, from the patient perspective, telehealth does provide good quality HIV care. Our study also provides valuable information about certain groups who may benefit from integrating telehealth into their care. However, this must be balanced by concerns that certain aspects of care may be better delivered via in-person visits.
We recognize several limitations of our study including the use of a web-based cross-sectional survey data which may result in response and recall bias. Participants who had COVID-19 may have been more likely to participate in the survey. We do not have information about the DC Cohort participants who refused to participate in the COVID survey. Results from this study among PWH in an urban setting may not be generalizable to the larger PWH population. We did not aim to assess whether telehealth usage was associated with improved HIV measures (viral suppression and CD4 counts ≥ 200 cells/μL) in this analysis. Additionally, participants who had strong positive or negative telehealth experiences may have been more likely to provide responses to the open-ended questions. Strengths of this study include the ability to link survey responses to the larger longitudinal DC Cohort study database to triangulate our findings. Further, our ability to capture telehealth use and perceptions among an urban and largely minority population of PWH provide unique insights into PWH receiving care in these settings. Finally, our mixed-methods approach allowed us to enhance and better contextualize our quantitative findings.
With COVID-19 becoming a more endemic infection and the gradual return to in-person care, it will be important to monitor whether interest and motivation for telehealth changes among PWH as pandemic restrictions subside. Determining which groups of PWH benefit most from telehealth visits will help with developing targeted care models post-pandemic [37]. Additionally, eliciting provider perspectives regarding the impact of and satisfaction with telehealth will be essential if we are to consider how to support its continued use [38]. The HIV Medicine Association of the Infectious Diseases Society of America recently recommended telehealth as a method of expanding access to care for PWH [39]. Since virtual visits will likely be an essential component of continuing HIV care, quality of care metrics and guidelines are needed to monitor and evaluate this component of care.
Understanding the limitations and challenges of telehealth care can help providers best balance the use of in-person and virtual visits [40, 41]. Innovations to overcome telehealth barriers have been used in other PWH populations, such as rideshare support for lab testing in conjunction with telehealth visits [42] and use of an mhealth application, PositiveLinks, for increasing engagement in care [43]. Differentiated service delivery models tailored to the local context should incorporate telehealth for certain aspects of HIV care [44] and may lead to increased continuity of care. Based on this urban and racially diverse cohort of PWH, telehealth was well received with high patient satisfaction and interest in future use. As we transition away from our emergency pandemic responses, it will be important to determine how this technology can be applied and available in an equitable manner to further strengthen HIV care engagement going forward.
Supplementary Material
Acknowledgements
Data in this manuscript were collected by the DC Cohort Study Group with investigators and research staff located at: Children’s National Hospital Pediatric clinic (Natella Rakhmanina); the Senior Deputy Director of the DC Department of Health HAHSTA (Clover Barnes); Family and Medical Counseling Service (Angela Wood); Georgetown University (Princy Kumar); The George Washington University Biostatistics Center (Tsedenia Bezabeh, Vinay Bhandaru, Asare Buahin, Nisha Grover, Lisa Mele, Susan Reamer, Alla Sapozhnikova, Greg Strylewicz, and Marinella Temprosa); The George Washington University Department of Epidemiology (Shannon Barth, Morgan Byrne, Amanda Castel, Alan Greenberg, Shannon Hammerlund, Olivia Kirby, Paige Kulie, Anne Monroe, Lauren O’Connor, James Peterson, Bianca Stewart, and Mark Storey); The George Washington University Medical Faculty Associates (Jose Lucar); Howard University Adult Infectious Disease Clinic (Jhansi L. Gajjala) and Pediatric Clinic (Sohail Rana); Kaiser Permanente Mid-Atlantic States (Michael Horberg); La Clinica Del Pueblo (Ricardo Fernandez); MetroHealth (Duane Taylor); Washington Health Institute, formerly Providence Hospital (Jose Bordon); Unity Health Care (Gebeyehu Teferi); Veterans Affairs Medical Center (Debra Benator, Rachel Denyer); Washington Hospital Center (Adam Klein); and Whitman-Walker Institute (Stephen Abbott). The DC Cohort and COVID-19 supplement were funded by the National Institute of Allergy and Infectious Diseases, 1R24AI152598-01.
Funding
The DC Cohort and COVID-19 supplement were funded by the National Institute of Allergy and Infectious Diseases, 1R24AI152598-01. This research is funded by NIH Grant R24AI152598-01.
Abbreviations
- EMR
Electronic medical record
- NQF
National Quality Forum
- PWH
Persons with HIV
Footnotes
Supplementary Information The online version contains supplementary material available at https://doi.org/10.1007/s10461-023-04198-7.
Declarations
Competing Interests Dr Barth: none; Dr Saulters: Alliance for Academic Internal Medicine, American College of Physicians 2021–2022, Paid to institution; Magnolia Regional Medical Center, Corinth MS 2020 and 2021, payment made to individual; Support for travel to Alliance for Academic Internal Medicine 2022, payment made to individual; Dr. Balba: Gilead, GlaxoSmithKline personal stocks; Dr. Kumar: Consulting fees from ViiV/GSK, Merck, Thera Technology, Johnson and Johnson, paid to individual; personal stocks in Pfizer, Gilead, Merck, Moderna, Johnson and Johnson; Dr. Greenberg: Invited speaker, Davidson College, paid to individual; DSMB/AB UW CFAR EAB (all unrelated), JHU CFAR EAB, ERC CFAR EAB, paid to individual; Dr. Castel: Research award from Gilead Sciences, Funding from National Institutes of Health, paid to individual; NIH NIDA Clinical Trials Network; personal stocks in Pfizer, Viatris, Inc.; Receipt of medications through Gilead Sciences grant as mentioned above, paid to institution.
Ethical Approval The survey was approved by the George Washington University Institutional Review Board (IRB) and specific sites’ IRBs (Washington DC Department of Veterans Affairs, Washington Hospital Center, Children’s National Medical Center).
Informed Consent Electronic informed consent was obtained from all participants.
References
- 1.Johns Hopkins University. The center for systems science and engineering. COVID-19 dashboard 2021. [cited 2022 Nov 28]. Available from: https://www.coronavirus.jhu.edu/map.html. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Ohl M, et al. Mixed-methods evaluation of a telehealth collaborative care program for persons with HIV infection in a rural setting. J Gen Intern Med 2013;28(9):1165–73. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Kane CK, Gillis K. The use of telemedicine by physicians: still the exception rather than the rule. Health Aff (Millwood) 2018;37(12):1923–30. [DOI] [PubMed] [Google Scholar]
- 4.Appropriations, House Budget, Coronavirus Preparedness and Response Supplemental Appropriations Act, 2020, in Public Law No: 116–123 (03/06/2020), H.R.t.t.C.− Congress, editor. 2020. [Google Scholar]
- 5.US Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation. Updated national survey trends in telehealth utilization and modality (2021–2022) 2023. Feb 7. Available from: https://aspe.hhs.gov/reports/updated-hps-telehealth-analysis-2021-2022. [PubMed] [Google Scholar]
- 6.Young JD, et al. Infectious Diseases Society of America position statement on telehealth and telemedicine as applied to the practice of infectious diseases. Clin Infect Dis 2019;68(9):1437–43. [DOI] [PubMed] [Google Scholar]
- 7.Centers for Disease Control and Prevention. Telehealth and telemedicine 2023. Feb 7. Available from: https://www.cdc.gov/phlp/publications/topic/telehealth.html. [Google Scholar]
- 8.Telehealth.HHS.GOV 2023. Feb 7. Available from: https://telehealth.hhs.gov/. [Google Scholar]
- 9.Nitiema P Telehealth before and during the COVID-19 pandemic: analysis of health care workers’ opinions. J Med Internet Res 2022;24(2): e29519. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Miller MJ, et al. Patient experience after modifying visit delivery during the COVID-19 pandemic. Am J Manag Care 2021;27(2):e54–63. [DOI] [PubMed] [Google Scholar]
- 11.Fadul N, et al. A Midwestern academic HIV clinic operation during the COVID-19 pandemic: implementation strategy and preliminary outcomes. J Int Assoc Provid AIDS Care 2021;20:23259582211041424. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Mayer KH, et al. Rapid migration to telemedicine in a Boston community health center is associated with maintenance of effective engagement in HIV care. Open Forum Infect Dis 2020;7(Suppl 1):S337–8. [Google Scholar]
- 13.Harsono D, et al. Experiences with telemedicine for HIV care during the COVID-19 pandemic: a mixed-methods study. AIDS Behav 2022;26(6):2099–111. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.National Quality Forum. 2021. Rural telehealth and healthcare system readiness measurement framework. Washington, DC. Available at: https://www.qualityforum.org/Publications/2021/11/Rural_Telehealth_and_Healthcare_System_Readiness_Measurement_Framework_-_Final_Report.aspx. [Google Scholar]
- 15.Greenberg AE, et al. Development of a large urban longitudinal HIV clinical cohort using a web-based platform to merge electronically and manually abstracted data from disparate medical record systems: technical challenges and innovative solutions. J Am Med Inform Assoc 2016;23(3):635–43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Harris PA, et al. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42(2):377–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Harris PA, et al. The REDCap consortium: building an international community of software platform partners. J Biomed Inform 2019;95:103208. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Permanente Kaiser. KP Research Bank COVID-19 surveys. 2020 2023. Feb 9. Available from: https://researchbank.kaiserpermanente.org/wp-content/uploads/2021/05/KPRB-COVID-19-Survey_-summary.pdf. [Google Scholar]
- 19.Wilbourn B, et al. Characterization of HIV risk behaviors and clusters using HIV-transmission cluster engine among a cohort of persons living with HIV in Washington, DC. AIDS Res Hum Retrovir 2021;37(9):706–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Stanford. COVID19 interview items for vulnerable populations. 2020 2023. Feb 9. Available from: https://clelandcm.github.io/COVID19-Interview-Items/COVID-Items.html#stanford. [Google Scholar]
- 21.Behar-Zusman V, Chavez JV, Gattamorta K. Developing a measure of the impact of COVID-19 social distancing on household conflict and cohesion. Fam Process 2020;59(3):1045–59 (9 Feb 2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.ATN. COVID Questionnaire Draft. 2020 2023. Feb 9. Available from: https://clelandcm.github.io/COVID19-Interview-Items/COVID-Items.html#atn. [Google Scholar]
- 23.CDC. Human infection with 2019 novel coronavirus case report form interviewer information case classification and identification 2023. Feb 9. Available from: https://www.cdc.gov/coronavirus/2019-ncov/downloads/pui-form.pdf. [Google Scholar]
- 24.City University of New York Institute for Implementation Science in Population Health. Baseline Questionnaire for the Communities, Households and SARS/COV-2 Epidemiology (CHASE) COVID Study March 31, 2020. Available from: https://cunyisph.org/wp-content/uploads/CHASE-COVID_baseline_V2.1.pdf. [Google Scholar]
- 25.Kalichman S Kalichman Covid-19 Assessment 2020. Available from: https://clelandcm.github.io/COVID19-Interview-Items/COVID-Items.html#kalichman-covid-19-assessment. [Google Scholar]
- 26.Creswell JW, et al. National Institutes of Health. 2011. Best practices for mixed methods research in the health sciences. Bethesda Available at: https://obssr.od.nih.gov/sites/obssr/files/Best_Practices_for_Mixed_Methods_Research. [Google Scholar]
- 27.ATLAS.ti Scientific Software Development GmbH [ATLAS.ti 22 Windows] 2022. [Google Scholar]
- 28.Smaling A Varieties of methodological intersubjectivity—the relations with qualitative and quantitative research, and with objectivity. Qual Quant 1992;26:169–80. [Google Scholar]
- 29.Labisi T, et al. HIV care meets telehealth: a review of successes, disparities, and unresolved challenges. Curr HIV/AIDS Rep 2022;19(5):446–53. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Ennis N, Armas L, Butame S, Joshi H. Factors impacting video telehealth appointment completion during COVID-19 pandemic among people living with HIV in a community-based health system. AIDS Behav 2021;6(2):407–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Friedman EE, Devlin SA, Gilson SF, Ridgway JP. Age and racial disparities in telehealth use among people with HIV during the COVID-19 pandemic. AIDS Behav 2022;26(8):2686–91. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Auchus IC, et al. Transitioning to telehealth during the COVID-19 pandemic: patient perspectives and attendance at an HIV clinic in San Francisco. AIDS Patient Care STDS 2021;35(7):249–54. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Grove M, Brown LB, Knudsen HK, Martin EG, Garner BR. Employing telehealth within HIV care: advantages, challenges, and recommendations. AIDS 2021;35(8):1328–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Smith E, Badowski ME. Telemedicine for HIV care: current status and future prospects. HIV AIDS (Auckl) 2021;13:651–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Budak JZ, et al. The impact of COVID-19 on HIV care provided via telemedicine-past, present, and future. Curr HIV/AIDS Rep 2021;18(2):98–104. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Spears CE, et al. Intersecting epidemics: the impact of coronavirus disease 2019 on the HIV prevention and care continua in the United States. AIDS 2022;36(13):1749–59. [DOI] [PubMed] [Google Scholar]
- 37.Dandachi D, et al. Integration of telehealth services in the healthcare system: with emphasis on the experience of patients living with HIV. J Investig Med 2019;67(5):815–20. [DOI] [PubMed] [Google Scholar]
- 38.Bouabida K, et al. Healthcare professional perspectives on the use of remote patient-monitoring platforms during the COVID-19 pandemic: a cross-sectional study. J Pers Med 2022;12(4):529. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Thompson MA, Horberg MA, Agwu AL, Colasanti JA, Jain MK, Short WR, Singh T, Aberg JA. Primary care guidance for persons with human immunodeficiency virus: 2020 update by the HIV Medicine Association of the Infectious Diseases Society of America. Clin Infect Dis 2021;73(11):e3572–605. [DOI] [PubMed] [Google Scholar]
- 40.Patel MC, Young JD. Delivering HIV subspecialty care in prisons utilizing telemedicine. Dis Mon 2014;60(5):196–200. [DOI] [PubMed] [Google Scholar]
- 41.Castaño PM, Stockwell MS, Malbon KM. Using digital technologies to improve treatment adherence. Clin Obstet Gynecol 2013;56(3):434–45. [DOI] [PubMed] [Google Scholar]
- 42.Koay WLA, et al. Brief report: supporting access to HIV care for children and youth during the COVID-19 pandemic with telemedicine and rideshare. J Acquir Immune Defic Syndr 2021;88(4):384–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Campbell BR, et al. PositiveLinks and the COVID-19 response: importance of low-barrier messaging for PLWH in non-urban Virginia in a crisis. AIDS Behav 2021;25(11):3519–27. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Collins LF, et al. The COVID-19 pandemic as a catalyst for differentiated care models to end the HIV epidemic in the United States: applying lessons from high-burden settings. AIDS 2021;35(2):337–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.