Abstract
Background
SARS CoV-2 virus (COVID-19) impacted the practice of healthcare in the United States, with technology being used to facilitate access to care and reduce iatrogenic spread. Since then, patient message volume to primary care providers has increased. However, the volume and trend of electronic communications after lockdown remain poorly described in the literature.
Methods
All incoming inbox items (telephone calls, refill requests, and electronic messages) sent to providers from patients amongst four primary care clinics were collected. Inbox item rates were calculated as a ratio of items per patient encountered each week. Trends in inbox rates were assessed during 12 months before and after lockdown (March 1st, 2020). Logistic regression was utilized to examine the effects of the lockdown on inbox item rate post-COVID-19 lockdown as compared to the pre-lockdown period.
Results
Before COVID-19 lockdown, 2.07 new inbox items per encounter were received, which increased to 2.83 items after lockdown. However, only patient-initiated electronic messages increased after lockdown and stabilized at a rate higher than the pre-COVID-19 period (aRR 1.27, p-value < 0.001). In contrast, prescription refill requests and telephone calls quickly spiked, then returned to pre-lockdown levels.
Conclusion
Based on our observations, providers experienced a quick increase in all inbox items. However, only electronic messages had a sustained increase, exacerbating the workload of administrators, staff, and clinical providers. This study directly correlates healthcare technology adoption to a significant disruptive event but also shows additional challenges to the healthcare system that must be considered with these changes.
Keywords: inbox, in-basket, electronic messages, patient-provider communication, electronic health record, EHR, patient portals, telehealth, COVID-19, pandemic, primary care
Introduction
Patient-provider communication is a cornerstone for primary care that is critical for improving and maintaining patient health. Sub-optimal communication is associated with poorer outcomes in chronic diseases, particularly among patients with poor health literacy.1–3 Effective relationships between healthcare providers and patients increase self-care efficacy and result in less burden related to treatment.4 However, the traditional patient-provider communication channels were disrupted during the SARS CoV-2 virus (COVID-19) pandemic. This disruption contributed to issues related to patient care, particularly among those already experiencing health disparities, through disruption of traditional communication channels, reduced routine screening for health conditions, and an unprecedented amount of health misinformation.5–8
Over the past several years, electronic communication has become a prominent form of patient-provider communication and has increased since the COVID-19 pandemic began. As demonstrated by Nath et al.9 via a piece wise linear regression, messaging and calls increased for primary care and specialists during the pandemic. However, the sustained impact of this change was not clearly defined or evaluated over time. Electronic messages were first introduced in 2000 with the creation of patient portals for electronic record systems.10 They became widespread within years of inception and are now a feature of most electronic health record (EHR) systems in use today.11 Within the first decade, the number of electronically derived messages that physicians received from patients nearly tripled.12 Physicians have noted that electronic messages contribute to enhancing patient care and communication, cost saving, improved productivity, and saved time.13 However, negative attributes of patient messaging include inadequate reimbursement, assuming additional liability, and patients using electronic services incorrectly.14 All of these problems may have been compounded during the COVID-19 pandemic due to increasing volume and use of these services by patients unfamiliar with these types of communication.12,15,16 The additional time responding and sorting these messages may burden the healthcare provider who has increasing administrative duties in addition to patient care.
In March of 2020, the COVID lockdown mandate in the United States (US) impacted millions of individuals and businesses, but arguably, none more so than healthcare. Across the country, medical care providers and ambulatory clinics quickly adapted by utilizing telemedicine services to continue the care and treatment of their patients. Although rapid, this sudden shift to telemedicine was largely successful and perceived favorably by patients,17–20 it continues to be a part of routine clinical care in most practices, despite relaxation of lockdown policies.
We suspect that in addition to providers offering more services through telemedicine, patients may have adopted and continued utilizing other established forms of electronic communication that became necessary during the lockdown period. The purpose of this study is to understand how the volume of electronic communications between patients and providers in outpatient primary care clinics changed after the US COVID-19 pandemic lockdown in March 2020. We examined trends in electronic communications 1 year before and after lockdown to evaluate if utilization of different electronic communication methods had a significant and sustained change after lockdown.
Methods
Study design and population
This retrospective study evaluated the frequency of electronic inbox items sent from patients to providers in outpatient primary care clinics between March 01, 2019 and March 31, 2021. Deidentified data were collected in the year directly before and the year after the US COVID-19 national lockdown. Within the primary care clinics being evaluated, in-person visits were restricted and transitioned to video visits in the 1st week of March 2020 and used as the point of interruption in the study. Inbox items were defined as patient-derived electronic messages, refill requests, and telephone call memos. These items were collected from the clinic's EHR data system, along with patient demographics, clinical diagnosis codes, and clinic encounter details. In order to capture comorbidities experienced by patients, we utilized the Elixhauser co-morbidity index to quantify the clinical burden of patients. Elixhauser co-morbidity index has been determined to perform better or similar to the Charlson co-morbidity index to predict health outcomes in various patient populations.21 Information was obtained from patients 18 years of age or older from four different academic ambulatory primary care clinics (three from Family Medicine and one from Internal Medicine) in Houston, Texas. Only the initial item from each string of communications was counted, with no items in the count being replicated in the study. This was used to better reflect the number of times patients contact their providers but does not specifically demonstrate the total work burden or complexity of the items, nor does this study include practitioner-initiated communication. Multiple messages were often derived from individual patients and was accounted for in the descriptive analysis. However, in the trends assessment, our analyses included all unique inbox instances regardless of the number of requests from individual patients.
Population-level clinical information was obtained from the Houston Health Department regarding the number of COVID-19 cases identified in the Houston Metroplex during the study period.22 COVID-19 case counts were included in the analyses in order to understand whether surges of infection cases were correlated with the rate of messages received. Lastly, there was a notable outlier during the week of February 15th, 2021, due to Winter Storm Uri which caused a statewide power outage. Clinics were closed during that week. Despite the winter emergency, a significant number of messages were still sent to providers. As a result of the extreme outliers obtained, data from that week were excluded. This study was assessed and given exemption status from full review by the Baylor College of Medicine's Institutional Review Board (protocol number H-50797).
Statistical analyses
In order to account for temporal fluctuations, inbox items were adjusted as weekly rates defined as: number of inbox items per number of clinical encounters (in person or telemedicine) each week during the period assessed. It should be noted that due to the academic setting, many providers had teaching and administrative responsibilities making their day-to-day clinic volume fluctuate frequently and vary between providers. Thus, to standardize rates of inbox items we chose to use encounters per week as the denominator instead of per provider. These rates were visualized and assessed over time as a whole and for each different sub-type of inbox item (messages, refills, and telephone memos). The timeline was divided into Pre-COVID Lockdown and Post-COVID Lockdown as messages or encounters that occurred before or after March 1, 2020. For all patients seen during the study period, demographic and clinical specific details were compared pre and post lockdown using student's T-test and chi-square tests for bivariate analyses. Unique encounter features and inbox items were similarly tabulated between pre and post lockdown dates. We hypothesized that there would be a significant level change after lockdown with a sustained or elevated slope change. Thus, a segmented linear regression model was used to assess if the lockdown event had a significant impact on the rate of inbox items and each subtype that fit the expected trend. Due to slightly elevated variance of the data, a scaling adjustment to the analysis was made by using a Quasi-Poisson model. Number of reported COVID-19 cases in the community was added to the model to control for potential confounding of infection rate fluctuations. The final results were reported as adjusted risk ratios (aRR) with their corresponding 95% confidence intervals (95%CI) and P-values with an alpha of 0.05 determining significant differences. All analyses were done using R statistical software version 4.1.2 with the Rstudio environment (R Foundation for Statistical Computing, Vienna, Austria).23
Results
Over the study period, there were 50,297 unique individuals seen and a total of 107,633 distinct clinical visits observed in all four clinics (Table 1). During the Pre-COVID Lockdown period there were 25,504 distinct patients. In the Post-Lockdown period there was a slight decline in unique patients to 24,793. During this time there was a small but statistically significant increase in the mean age of patients (50.0 vs 50.6, p < 0.001) and a slightly higher prevalence of African Americans and Hispanic patients (20.9% vs 21.1% and 16.1% vs 17.1%, respectively; p < 0.001). There was no significant difference in gender frequencies before and after lockdown. There were no telemedicine encounters during the Pre-COVID Lockdown, but these types of visits increased rapidly after it became accessible in early March, accounting for 32.2% of all clinical encounters during the Post-COVID Lockdown period (Supplement 1). There was a moderate change in the proportion of patients seen in Family Medicine (70.9% vs 67.8%) and Internal Medicine (29.1% vs 32.2%, p < 0.001). Clinic visit frequency also increased from an average of 2.13 visits per year to 2.16 (p = 0.04). However, overall patient complexity measured by the Elixhauser Score showed no significant change before and after lockdown (1.59 vs 1.6, p = 0.97).
Table 1.
Pre-Covid lockdown | Post-Covid lockdown | |||||
---|---|---|---|---|---|---|
n = 25504 | n = 24793 | p-value | ||||
Age (mean, sd) | 50.03 | (17.12) | 50.6 | (17.28) | <0.001 | |
Sex (n, %) | 0.79 | |||||
Female | 14797 | (58.0%) | 14414 | (58.1%) | ||
Male | 10707 | (42.0%) | 10379 | (41.8%) | ||
Race and ethnicity (n, %) | <0.001 | |||||
White | 11491 | (45.1%) | 10776 | (43.5%) | ||
Black/African American | 5332 | (20.9%) | 5222 | (21.1%) | ||
Hispanic | 4092 | (16.1%) | 4230 | (17.1%) | ||
Asian/Pacific Islander | 2406 | (9.4%) | 2192 | (8.8%) | ||
American Indian or Alaska Native | 114 | (0.5%) | 108 | (0.4%) | ||
Other/declined | 2069 | (8.1%) | 2265 | (9.1%) | ||
Encounter type (n, %) | <0.001 | |||||
Office visit | 25504 | (100%) | 16158 | (65.2%) | ||
Telemedicine | 0 | (0%) | 8635 | (34.8%) | ||
Specialty (n, %) | <0.001 | |||||
Family medicine | 18074 | (70.9%) | 16807 | (67.8%) | ||
General internal medicine | 7430 | (29.1%) | 7986 | (32.2%) | ||
Clinical features (mean, sd) | ||||||
Elixhauser score | 1.59 | (1.86) | 1.6 | (1.84) | 0.97 | |
Patient visits per year | 2.13 | (1.64) | 2.16 | (1.82) | 0.04 | |
Inbox items per patient (mean, sd) | ||||||
Total items | 4.08 | (6.00) | 5.79 | (7.77) | <0.001 | |
Electronic messages only | 1.08 | (2.62) | 1.74 | (3.69) | <0.001 | |
Refill requests only | 1.67 | (3.17) | 2.09 | (3.80) | <0.001 | |
Telephone calls only | 1.34 | (2.68) | 1.96 | (3.30) | <0.001 |
The year before lockdown, each patient would send an average of 4.08 inbox items in total (messages, refills, and telephone calls). Per subtype, there were an average of 1.08 electronic messages, 1.67 refill requests, and 1.34 telephone calls. In the Post-COVID Lockdown period, average number of inbox item increased significantly, in total and per each subtype (5.79, 1.74, 2.09, 1.96, respectively; p < 0.001). This translated to a total of 112,240 inbox items during the Pre-COVID Lockdown period (Table 2). There was an average of 2158 inbox items per week with a corresponding average of 1042 clinical encounters each week (Figure 1), giving an inbox item rate of 2.07 per encounter. During the Post-COVID Lockdown period there were 151,108 inbox items, with average number of 2906 items per week and 1027 encounters per week. This equated to a post lockdown rate of 2.83 items per encounter.
Table 2.
Total | Pre-Covid lockdown | Post-Covid lockdown | |||||
---|---|---|---|---|---|---|---|
Total number of encounters, n (%) | 107633 | — | 54216 | — | 53417 | — | |
Type of encounters | |||||||
In-clinic | 86044 | (79.9) | 54216 | (100) | 31828 | (59.6) | |
Telemedicine | 21589 | (20.1) | 0 | (0) | 21589 | (40.4) | |
Specialty of encounters | |||||||
Family medicine | 72625 | (67.5) | 37476 | (69.1) | 35149 | (65.8) | |
General internal medicine | 35008 | (32.5) | 16740 | (30.9) | 18268 | (34.2) | |
Total number of inbox messages, n (%) | 263348 | — | 112240 | — | 151108 | — | |
Type of inbox messages | |||||||
Electronic messages | 73067 | (27.7) | 28795 | (25.7) | 44272 | (29.3) | |
Refill requests | 102679 | (40.0) | 47176 | (42.0) | 55503 | (36.7) | |
Telephone calls | 87602 | (33.3) | 36269 | (32.3) | 51333 | (34.0) | |
Number of all messages per number of encounters, rate (%) | 2.4467217 | — | 2.0702376 | — | 2.8288373 | — | |
Number of electronic messages per encounters | 0.6788531 | (27.7) | 0.5311163 | (25.7) | 0.8287998 | (29.3) | |
Number of refills per encounters | 0.9539732 | (40.0) | 0.870149 | (42.0) | 1.0390512 | (36.7) | |
Number of telephone calls per encounters | 0.8138954 | (33.3) | 0.6689723 | (32.3) | 0.9609862 | (34.0) |
Temporal trends showed that the rate of all inbox items had a temporary level change that increased immediately after lockdown but quickly returned back to the counterfactual projected rate (Figure 2). A similar trend was found amongst the refill and telephone message rates. However, electronic messages showed a distinctive slope change that was sustained, and this was the only appropriate trend for further model fitting. After adjusting for COVID cases in the community, the final adjusted model for electronic messages showed an aRR of 1.27 (95% CI: 1.15−1.42, p < 0.001) (Figure 3). The final model was assessed for autocorrelation and associated residuals, which did not show any concerns about the independence of the observations (Supplement 2).
Discussion
Between the Pre and Post-COVID Lockdown periods, there was a slight decline in the total number of unique patients and total visits. There were modest changes in demographic and clinical details amongst patients between the two study periods as well. Telemedicine video visit adoption was implemented quickly and accounted for approximately one-third of all clinical visits in the year following lockdown. There was a substantial increase in the total number inbox items and each sub-type in the Post-COVID Lockdown period. When adjusted for the number of encounters, rates of total inbox items and each subtype were statistically higher. However, we found that this was largely due to an immediate spike in electronic communication tools and a temporary lull in encounters. Telephone calls and refill requests gradually returned to the pre-lockdown rates after approximately 6 months. The notable exception was electronic messages, which had a significant upward shift immediately after lockdown, and then maintained an elevated rate through the post-COVID Lockdown period. The final model indicated a 27% increase in the relative risk of electronic messages after adjusting for COVID cases in the community.
To put this into context, if a provider saw 20 patients per day in the Pre-COVID Lockdown period, they would expect to receive 41 inbox items per day. After lockdown, this would have increased to an average of 57 inbox items daily, representing a 26.9% increase. The Post-COVID Lockdown message rate was slightly misleading, as the majority of the difference occurred in the first half of the Post-COVID Lockdown study period, with higher rates of inbox items immediately after lockdown, demonstrating an even greater burden placed on faculty and staff during that period. Fortunately, with refills and telephone calls returning to baseline, the demand became less but still elevated due to electronic messages, which are arguably more time-consuming to the individual provider who often has requirements to respond in a short amount of time.
Prior to the COVID-19 lockdown, inbox items were a point of controversy in healthcare. Since the adoption of EHRs, there has been little to no assessment of the impact of the digital inbox on clinical practice and overall workflow in the outpatient setting.14 Electronic messaging directly from patients to their providers has contributed to burnout in primary care, particularly among female providers who receive more patient messages than male providers.24,25 Even specialty physicians, such as orthopedic surgeons, are now spending more time in record keeping and clerical work than they are in face-to-face patient interaction.26 These issues not only affect mental health and burnout, but also contributes to reduction in billable services and frequent interruptions in the clinical environment for providers, clinical extenders, and staff.26–28 Alternatively, patients find these systems to be time-saving, and are satisfied with their use.29 As messaging becomes more broadly adopted, more considerations about triage of messages, privacy, and time management need to be considered.29
As reported by many clinical specialties, the social and clinical changes due to COVID lockdown was significantly disruptive to routine clinical practice for patients and providers.30–32 This sudden change correlated with an adoption of healthcare related technologies, most notably telemedicine.33,34 The emergent nature associated with the COVID-19 outbreak helped with overcoming many of the initial barriers associated with adoption of telehealth tools.35,36 In one of the oldest conceptual models for understanding technology adoption, the classic technology acceptance model (TAM) would describe this shift as a new external variable changing the perceived usefulness of the technology, and thus leading to substantial change in actual use (Supplement 3).37,38 It has also been argued that the increased use of digital conferencing tools in businesses, schools, and routine social interactions impacted the Perceived Ease of Use for this type of technology as well, further driving actual usage.39–41 However, once lockdown restrictions gradually receded, we observed gradual reduction in telemedicine visits. A similar trend was seen with telephone calls and refills to the clinic, both of which had an increase in perceived usefulness until COVID lockdown and social behaviors slowly returned to normal. However, electronic messages volume increase was more sustained. Using the TAM as a reference, we concluded lockdown spurred an increase in technology utilization while it maintained its perceived usefulness despite reduction of COVID lockdown policies. During the study window, the messaging mechanism through the EHR did not have any major changes to its functionality or features, thus its technical ease of use did not change. As with telemedicine, we suspect that increased use of digital tools outside of healthcare may be contributing as an external factor. Telemedicine visits themselves may have also contributed to increase comfort with using other healthcare related tools as a proximity effect, but the sustained use of electronic messaging is a feature that is unique to itself.42
Electronic messaging maintained a stable rate of occurrence during the pre-lockdown period. Although message volume increased significantly after adoption of messaging platforms, the rate of messaging per patient was maintained at a consistent level.12 However, special considerations should be given during and after times of crises, such as the COVID pandemic, when patients increase their adoption of new technology and then become accustomed to utilizing it in future interactions. Furthermore, while there is substantial evidence that shows the safety and efficacy of telehealth services, most of these use cases are studied in specialty clinical settings or specific diseases.43,44 Literature that captures the use and effect in a primary care setting remains sparse. Because of the pandemic, adoption of telehealth services has been added quickly to traditional clinical workflows with unknown implications. Additional support to reduce the time-burden of these tasks should be given during these types of events, and continuing assistance should be evaluated throughout and beyond special circumstances to ensure positive outcomes and reduce time spent in these activities.45
Programs have been developed to help reduce the amount of time that primary care providers spend doing administrative tasks through workflow interventions that reduce waste and increases activities that add value to the clinic. One of these programs was associated with a 30% improvement in documentation and closure of patient charts as well as less time needed for refill requests.45 Using non-physician personnel, such as scribes, to respond to inbox messages for primary care providers has been shown to have no significant effect on time to completion of these tasks.46 Although improved response time would be ideal, reducing the burden of responding to these messages could improve provider morale and improve the amount of time they can devote to direct patient care, patient phone calls, or consultation with other provider staff.
This study benefited from a large time window with a robust sample size of encounters and inbox items at each measured time point. However, our results are only a small representation of the entire volume of clinical work outside of direct patient care. Per the study design, communications that required further clarification, forwarding/replying to other staff, or new issues being addressed through previously established communication threads were not represented in our results. Nor do they capture other types of inbox items that may be unique to the clinic's or EHR's preferences, or communications outside the EHR entirely (faxes, emails, text messages, etc.). Overall, our study likely only represents a fraction of non-encounter communication and non-clinical work that occurs in primary care.
Electronic communications provide a useful tool to enable better healthcare management in the primary care setting, however, as adoption and use of this technology increases, the amount of time and work that it is associated with needs to be accounted for. As these tools continue to change how healthcare is practice, so to must the workflow and resources in the clinic setting. Disruptive events such as the COVID-19 Lockdown demonstrate the value of these tools but continues to exacerbate the unaddressed work burden that they also introduce.
Supplemental Material
Supplemental material, sj-docx-1-jtt-10.1177_1357633X221146810 for Impact of COVID-19 lockdown on patient-provider electronic communications by Michael A. Hansen, Rebecca Chen, Jacqueline Hirth, James Langabeer and Roger Zoorob in Journal of Telemedicine and Telecare
Abbreviations
- US
United States
- EHR
Electronic Health Record
- COVID-19
SARS CoV-2 virus
- TAM
Technology Acceptance Model
- CI
Confidence Interval
- aRR
Adjusted Risk Ratio
Footnotes
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding: The author(s) received no financial support for the research, authorship, and/or publication of this article. This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
ORCID iD: Michael A. Hansen https://orcid.org/0000-0001-8787-0006
Supplemental material: Supplemental material for this article is available online.
References
- 1.Seligman HK, Wang FF, Palacios JL, et al. Physician notification of their diabetes patients’ limited health literacy: a randomized, controlled trial. J Gen Intern Med 2005; 20: 1001–1007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Piette JD, Schillinger D, Potter MBet al. et al. Dimensions of patient-provider communication and diabetes self-care in an ethnically diverse population. J Gen Intern Med 2003; 18: 624–633. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.White RO, Chakkalakal RJ, Presley CA, et al. Perceptions of provider communication among vulnerable patients with diabetes: influences of medical mistrust and health literacy. J Health Commun 2016; 21: 127–134. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Eton D, Ridgeway J, Linzer M, et al. Healthcare provider relational quality is associated with better self-management and less treatment burden in people with multiple chronic conditions. Patient Prefer Adherence 2017; 11: 1635–1646. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Chong YY, Cheng HY, Chan HYLet al. et al. COVID-19 pandemic, infodemic and the role of eHealth literacy. Int J Nurs Stud 2020; 108: 103644. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Gray DM, Anyane-Yeboa A, Balzora Set al. et al. COVID-19 and the other pandemic: populations made vulnerable by systemic inequity. Nat Rev Gastroenterol Hepatol 2020; 17: 520–522. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Bakouny Z, Paciotti M, Schmidt ALet al. et al. Cancer screening tests and cancer diagnoses during the COVID-19 pandemic. JAMA Oncol 2021; 7: 58. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Gujral UP, Johnson L, Nielsen J, et al. Preparedness cycle to address transitions in diabetes care during the COVID-19 pandemic and future outbreaks. BMJ Open Diabetes Res Care 2020; 8: e001520. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Nath B, Williams B, Jeffery MM, et al. Trends in electronic health record inbox messaging during the COVID-19 pandemic in an ambulatory practice network in new England. JAMA Netw Open 2021; 4: e2131490. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Halamka JD, Tripathi M. The HITECH era in retrospect. N Engl J Med 2017; 377: 907–909. [DOI] [PubMed] [Google Scholar]
- 11.Adler-Milstein J, Holmgren AJ, Kralovec Pet al. et al. Electronic health record adoption in US hospitals: the emergence of a digital “advanced use” divide. J Am Med Inform Assoc 2017; 24: 1142–1148. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Crotty BH, Tamrat Y, Mostaghimi Aet al. et al. Patient-To-Physician messaging: volume nearly tripled as more patients joined system, but per capita rate plateaued. Health Aff (Millwood) 2014; 33: 1817–1822. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Police R, Foster T, Wong K. Adoption and use of health information technology in physician practice organisations: systematic review. J Innov Health Inform 2010; 18: 245–258. [DOI] [PubMed] [Google Scholar]
- 14.Rathert C, Mittler JN, Banerjee Set al. et al. Patient-centered communication in the era of electronic health records: what does the evidence say? Patient Educ Couns 2017; 100: 50–64. [DOI] [PubMed] [Google Scholar]
- 15.Houston TK, Sands DZ, Nash BRet al. et al. Experiences of physicians who frequently use E-Mail with patients. Health Commun 2003; 15: 515–525. [DOI] [PubMed] [Google Scholar]
- 16.Kittler A, Pizziferri L, Volk Let al. et al. Primary care physician attitudes towards using a secure web-based portal designed to facilitate electronic communication with patients. J Innov Health Inform 2004; 12: 129–138. [DOI] [PubMed] [Google Scholar]
- 17.de Oliveira Andrade A, Soares AB, de Andrade Palis A, et al. On the use of telemedicine in the context of COVID-19: legal aspects and a systematic review of technology. Res Biomed Eng 2022; 38: 209–227. [Google Scholar]
- 18.Hincapié MA, Gallego JC, Gempeler Aet al. et al. Implementation and usefulness of telemedicine during the COVID-19 pandemic: a scoping review. J Prim Care Community Health 2020; 11: 215013272098061. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Chu C, Cram P, Pang Aet al. et al. Rural telemedicine use before and during the COVID-19 pandemic: repeated cross-sectional study. J Med Internet Res 2021; 23: e26960. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Reeves JJ, Pageler NM, Wick EC, et al. The clinical information systems response to the COVID-19 pandemic. Yearb Med Inform 2021; 30: 105–125. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Moltó A, Dougados M. Comorbidity indices. Clin Exp Rheumatol 2014; 32: S-131–S-134. [PubMed] [Google Scholar]
- 22.Houston Health Department. Harris County / City of Houston COVID-19 Data Hub. https://covid-harriscounty.hub.arcgis.com/pages/cumulative-data.
- 23.R Core Team. R: A language and environment for statistical computing. Published online 2018. https://www.R-project.org/.
- 24.Rittenberg E, Liebman JB, Rexrode KM. Primary Care Physician Gender and Electronic Health Record Workload. J Gen Intern Med 2022; 37: 3295–3301. Published online January 6, 2022. doi: 10.1007/s11606-021-07298-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Bittar PG, Nicholas MW. The burden of inbox-messaging systems and its effect on work-life balance in dermatology. J Am Acad Dermatol 2018; 79: 361–363.e1. [DOI] [PubMed] [Google Scholar]
- 26.Verret CI, Nguyen J, Verret Cet al. et al. How do areas of work life drive burnout in orthopaedic attending surgeons, fellows, and residents? Clin Orthop Relat Res 2021; 479: 251–262. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Somerson JS, Patton A, Ahmed AAet al. et al. Burnout among United States orthopaedic surgery residents. J Surg Educ 2020; 77: 961–968. [DOI] [PubMed] [Google Scholar]
- 28.Todd B, Shinthia N, Nierenberg Let al. et al. Impact of electronic medical record alerts on emergency physician workflow and medical management. J Emerg Med 2021; 60: 390–395. [DOI] [PubMed] [Google Scholar]
- 29.Wallwiener M, Wallwiener CW, Kansy JKet al. et al. Impact of electronic messaging on the patient-physician interaction. J Telemed Telecare 2009; 15: 243–250. [DOI] [PubMed] [Google Scholar]
- 30.Chang AY, Cullen MR, Harrington RAet al. et al. The impact of novel coronavirus COVID-19 on noncommunicable disease patients and health systems: a review. J Intern Med 2021; 289: 450–462. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Carr MJ, Steeg S, Webb RT, et al. Effects of the COVID-19 pandemic on primary care-recorded mental illness and self-harm episodes in the UK: a population-based cohort study. Lancet Public Health 2021; 6: e124–e135. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Singer SJ, Glassman J, Glaseroff A, et al. Impact of COVID-19 on Primary Care Practice Sites and Their Vulnerable Patients. Adv Health Care Manag 2021; 20: 233–271. doi: 10.1108/S1474-823120210000020009. [DOI] [PubMed] [Google Scholar]
- 33.Han SM, Greenfield G, Majeed Aet al. et al. Impact of remote consultations on antibiotic prescribing in primary health care: systematic review. J Med Internet Res 2020; 22: e23482. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Mold F, Hendy J, Lai YLet al. et al. Electronic consultation in primary care between providers and patients: systematic review. JMIR Med Inform 2019; 7: e13042. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Smith AC, Thomas E, Snoswell CL, et al. Telehealth for global emergencies: implications for coronavirus disease 2019 (COVID-19). J Telemed Telecare 2020; 26: 309–313. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Thomas EE, Haydon HM, Mehrotra A, et al. Building on the momentum: sustaining telehealth beyond COVID-19. J Telemed Telecare 2022; 28: 301–308. [DOI] [PubMed] [Google Scholar]
- 37.Taherdoost H. A review of technology acceptance and adoption models and theories. Procedia Manuf 2018; 22: 960–967. [Google Scholar]
- 38.Nadal C, Sas C, Doherty G. Technology acceptance in Mobile health: scoping review of definitions, models, and measurement. J Med Internet Res 2020; 22: e17256. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Bokolo AJ. Exploring the adoption of telemedicine and virtual software for care of outpatients during and after COVID-19 pandemic. Ir J Med Sci 2021; 190: 1–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Golinelli D, Boetto E, Carullo Get al. et al. Adoption of digital technologies in health care during the COVID-19 pandemic: systematic review of early scientific literature. J Med Internet Res 2020; 22: e22280. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Wosik J, Fudim M, Cameron B, et al. Telehealth transformation: COVID-19 and the rise of virtual care. J Am Med Inform Assoc JAMIA 2020; 27: 957–962. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Schreiweis B, Pobiruchin M, Strotbaum Vet al. et al. Barriers and facilitators to the implementation of eHealth services: systematic literature analysis. J Med Internet Res 2019; 21: e14197. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Snoswell CL, Stringer H, Taylor MLet al. et al. An overview of the effect of telehealth on mortality: a systematic review of meta-analyses. J Telemed Telecare 2021. Published online June 29, 2021:1357633X2110237. doi: 10.1177/1357633X211023700 [DOI] [PubMed] [Google Scholar]
- 44.Snoswell CL, Chelberg G, De Guzman KR, et al. The clinical effectiveness of telehealth: a systematic review of meta-analyses from 2010 to 2019. J Telemed Telecare 2021. Published online June 29, 2021:1357633X2110229. doi: 10.1177/1357633X211022907 [DOI] [PubMed] [Google Scholar]
- 45.Hung DY, Truong QA, Liang SY. Implementing lean quality improvement in primary care: impact on efficiency in performing common clinical tasks. J Gen Intern Med 2021; 36: 274–279. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Zallman L, Altman W, Chu L, et al. Do medical scribes help primary care providers respond more quickly to out-of-visit tasks? J Am Board Fam Med 2021; 34: 70–77. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplemental material, sj-docx-1-jtt-10.1177_1357633X221146810 for Impact of COVID-19 lockdown on patient-provider electronic communications by Michael A. Hansen, Rebecca Chen, Jacqueline Hirth, James Langabeer and Roger Zoorob in Journal of Telemedicine and Telecare