Barriers and Facilitators to the Collection and Aggregation of Electronic Health Record HIV Data: An Analysis of Study Recruitment Venues Within the Adolescent Medicine Trials Network for HIV/AIDS Interventions (ATN)

Abstract

Electronic health record (EHR) data can be leveraged for prospective cohort studies and pragmatic clinical trials, targeting youth living with HIV (YLH). Using EHRs in this manner may minimize the need for costly research infrastructure in service to lowering disease burden. This study characterizes HIV prevention and care continua variables and identifies factors likely to impede or facilitate EHR use for research and interventions. We conducted telephone-based qualitative interviews with National Experts (n =10) and Key Stakeholders (n = 19) from subject recruitment venues (SRVs), providing care services to YLH and youth at risk for HIV. We found 17 different EHR systems being used for various purposes (e.g., workflow management and billing). Thematic content analysis of interviews highlighted six broad categories of perspectives on barriers to and facilitators of EHR use: specific variable collection, general use barriers, and facilitators, general data collection barriers and facilitators, EHRs for surveillance and research, EHRs for personnel and resource management and capture of HIV specific variables. These findings may inform implementation strategies of future studies, in which we conduct routine monitoring of the youth HIV prevention and care continua using EHRs and test an eHealth intervention.

Introduction

In North America, health care practitioners are making the transition to electronic health records (EHRs)—patient health data stored in a digital format (Dean et al., 2009). Also driving the transformation is the possibility of using EHRs to support health research (Otero Varela et al., 2019). EHRs may be used to link patient health charts with diagnostic and administrative codes, forming a rich resource for epidemiologic and surveillance studies, aside from administrative and billing purposes (Hyman et al., 2017). Consequently, EHR-derived data, in theory, would be comprehensive and better structured, compared with historical forms of patient health data storage and documentation. Lastly, EHR data would not suffer the lag time problems of claims data, where there is a need to aggregate and process submitted claims (Dean et al., 2009).

The Adolescent Medicine Trials Network for HIV/AIDS Interventions (ATN) is a domestic, multicenter research network dedicated to improving the health outcomes of youth living with HIV (NICHD, 2019). A cornerstone of the network is its subject recruitment venues (SRVs). The SRVs are clinical sites, which often have differing capacities for EHR use. These differing EHR capacities can foster investigations into the costs of research infrastructure (Cowie et al., 2017), efficient regulatory paradigms (Bettger et al., 2018), and enhancements to HIV care and treatment cascades in the ATN (Lin et al., 2017; Naar et al., 2019). Such investigations may help narrow the evidence-to-practice gap in HIV care and care networks like the ATN.

Despite the potential of EHRs to support research, there remain some challenges. Organizational characteristics may affect the adoption of EHRs (Lambooij & Koster, 2016), and EHR system designs may create data errors, negatively impacting research (Vanderhook & Abraham, 2017; Colicchio et al., 2019). Also, there is a lack of standardization in EHRs across different health sectors (O’Leary et al., 2009) and even between population settings. EHR end-users (e.g., providers) struggle with interface engagement and the interoperability of EHRs (Acharya & Werts, 2019). Such problems can lower the quality of EHR-generated data and reduce its usability (Leibowitz & Desmond, 2015).

In the ATN, the Electronic Health Record Monitoring of the Continuum of Youth HIV Prevention and Care (EHR-COC) protocol (Protocol No. ATN 162) seeks to leverage EHR data for prospective cohort studies and pragmatic clinical trials. Thus, the protocol aims to reduce the costs of developing research infrastructure and lower the disease burden of HIV for clients and their providers. The protocol also aims to facilitate HIV prevention efforts and develop new paradigms for regulatory review of low-risk trials. It comprises two phases, with phase 01 focused on defining HIV prevention and care continua variables and a qualitative assessment of the barriers and facilitators impacting data collection and data downloads from EHRs. Consequently, we aimed to identify the factors likely to impede or support the collection and aggregation of EHR data from SRVs in the Network.

Methods

Study Design and Procedures

We employed an embedded mixed methods design (Creswell & Clark, 2017), with qualitative interviews serving as the primary method to guide the project and a secondary quantitative data gathering process, “embedded” within the interviews. Clinical trials in the ATN are implemented through its SRVs. So, we identified two sets of study participants who could offer insights into HIV health services provision and medical record administration, both at SRVs and at non-ATN facilities. These participants were classified as “National experts” and as “Key Stakeholders.” We conducted telephone interviews with these participants, capturing their views on EHR use. National Experts included principal investigators (PIs) and research program officers, all with HIV prevention, treatment, and EHR implementation experience. Conversely, Key stakeholders were EHR end-users, personnel from SRVs typically responsible for decisions about EHR administration or data downloads. We rationalized that Stakeholders could describe barriers and facilitators of EHRs, while National Experts could confirm those perspectives and provide a broader outlook on EHR implementation and use.

Recruitment

National Experts were recruited based on recommendations from the research team. To recruit Key Stakeholders, contact lists of personnel specific to each SRV were circulated to a primary liaison (e.g., Site PI at an SRV) to elicit recommendations on who could speak with the research team. Eligible participants then received an email inviting them to schedule an hour-long telephone interview. Our efforts yielded interviews with 10 National Experts and 19 Key Stakeholders. We conducted interviews between August 2018 and February 2019. At the start of each interview, every participant was read an informed consent statement, approved by the Florida State University, Institutional Review Board (IRB; Approval No: IRB00000446).

Data Collection

The interview structure and the accompanying data collection sheets were formulated a priori during study protocol development. The interviews had two parts; the first consisted of specific questions about HIV prevention and care variables, and the second focused on an open-ended discussion about EHRs. Trained interviewers conducted the discussions by following a semi-structured interview script, which highlighted themes of interest. At least one independent coder from the research team (JMD, LG, and/or SAB) sat in on each interview and completed a data collection form during the discussion. All the interviews were recorded.

The Key Stakeholder interviews began with a prompt to enumerate the type and number of EHR systems employed at their respective SRVs and discuss how EHRs were implemented. Then followed a discussion on HIV prevention and care variables, commonly associated with the prevention and care continua, comorbidities, and co-infections. Key Stakeholders were prompted to note if a specific variable was captured (i.e., Y/N), data capture format (e.g., paper and/or electronic), and data capture modalities (e.g., radio buttons versus free-text). The second half of the interviews focused on four areas: organizational structure and EHRs, elucidation of HIV continua terms used in EHRs, EHR data collection and extraction, and managing interventions with EHRs. National Expert interviews followed the same format, but discussion prompts were generalized and not focused on a specific SRV.

Data Analysis

Using an inductive approach, trained coders affected a thematic content analysis of the interviews. First, they cataloged all the EHR systems discussed during the interviews and tabulated counts of HIV prevention and care variables by SRV (i.e., whether variables are collected, how the variables are collected, and how they are stored). Coders then categorized perspectives on EHRs into three broad themes: individual experience factors, EHR system-specific factors, and institutional factors. The coders then stratified the perspectives according to barriers and facilitating features related to data collection and extraction methods. Consequently, interview summary reports were produced showing a thematic grid, which described the barriers and facilitators highlighted in the interviews. The coders resolved discrepancies in their understanding of participant perspectives via discussion.

Results

EHR Systems Common to ATN SRVs

Interviews highlighted several challenges to mobilizing EHRs in the ATN. We cataloged 17 different EHRs across all the represented SRVs, with EHRs utilized for several purposes, including data and workflow management (refer to Table 01 for a complete list). Some SRVs use a combination of EHRs, to capitalize on their specialized capabilities, such as billing and patient management. Alternatively, combining EHRs sometimes serves to address fiscal requirements stipulated by funders. For example, recipients of funding from the U.S. Health Resources and Services Administration (HRSA) often used CAREWare 6® in addition to their own EHR. Finally, public health reporting standards from local or federal government agencies can compel the use of certain EHRs. For example, some SRVs employ EvaluationWebâ to report information to the Centers for Disease Control and Prevention (CDC).

Table 1.

EHR systems, data management tools, and analytical software packages utilized by Surveyed SRVs within the ATN (as noted by Key Stakeholders)

Electronic Health Record (EHR) system	Manufacturer/ Purveyor/ Distributor		Description of program
General EHR Systems
Allscripts®		Allscripts Healthcare, LLC	General medical record keeping and practice management.
Centricity®		GE Healthcare	General medical record keeping and pharmacy database management.
Cerner EHR platform (Millennium®)		Cerner Corporation	General medical record keeping and practice management.
Citrix		Citrix Systems, Inc
eClinicalWorks®		Eclinicalworks, LLC
Epic (Clarity database)		Epic Systems Corporation	General medical record keeping and practice management.
iConnect®		IBM Watson Health
McKesson EHR®		McKesson Corp.	General medical record keeping and practice management.
Qlik		QlikTech	Conducting data analytics within existing EHR system.
Rx30® Pharmacy Management System		Transaction Data Systems, Inc.
Slice
McKesson STAR system		McKesson Corp.
HIV/AIDS & ATN Specific Systems
Casewatch Millennium®		ACMS Inc.	Tracking and managing care and services provided to patients by providers.
CAREWare 6®		U.S. Health Resources & Services Administration (HRSA) - HIV/AIDS Bureau	Provides electronic and social support services to patients and providers (strictly within the Ryan White HIV/AIDS Program)
Commcare		Dimagi, Inc.	Mobile data collection platform used for data entry during patient visits.
AIDS Institute Reporting System (AIRS)		Netsmart Technologies and licensed by Health Research, Inc	Used in conjunction with Ryan White Part A providers to catalogue services provided.
EvaluationWeb®		Centers for Disease Control and Prevention (CDC)	Web-based data entry platform.

SRV Data Collection

Before the interviews, we had identified 7 groups of HIV prevention variables: demographic characteristics, behavioral risk factors, behavioral health status, other medical status factors, pre-exposure prophylaxis (PrEP) levels, variables based on the CDC checklist for PrEP, and additional variables (see Table 02). Most of the SRVs captured the range of prevention variables in some capacity. However, the prevention variables, based on the CDC checklist, were not collected in several instances. This may be because SRVs are not aware of the CDC checklist or how to operationalize the guidelines for data collection.

Table 2.

Table denotes how sites handle HIV prevention variables (reported as a percentage of the 19 ATN sites queried)

HIIV Prevention Variables	Site capture of variable (%)		Data Capture Format (%)		EHR capture (%)		Paper Documents (%)
	Yes	No	Paper	Electronic	Templates	Open fields
Demographic Variables
Age	95	0	21	74	95	0	0
Gender	95	0	21	74	95	0	0
Sexual Orientation	95	0	32	63	74	0	0
Race	95	0	26	68	95	0	0
Ethnicity	95	0	21	74	95	0	0
Zip Code	89	0	21	63	84	0	0
Behavioral Risk Factors						0
Smoking Status	74	11	21	53	63	0	0
Substance Abuse	79	5	32	37	42	0	0
Behavioral Health Status
Mental Health	84	11	16	58	42	0	0
Sexual Risk	79	5	26	37	21	0	0
STI Results	84	0	5	79	79	0	0
Other Medical Status
Height/ Weight	79	16	5	74	79	0	0
BMI	79	16	5	68	74	0	0
Pregnancy Results	84	11	16	63	74	0	0
Levels of PrEP		0	0	0	0	0
PrEP Eligibility	84	5	42	32	16	0	0
PrEP Acceptance	74	5	42	26	11	0	0
PrEP Status	68	5	42	21	11	0	0
CDC Checklist for PrEP
PrEP basic education	47	0	21	5	0	5	5
Patient medical history	42	5	16	16	21	5	5
Assess acute HIV symptoms	47	0	11	32	32	11	11
HIV associated lab tests	53	0	0	47	47	0	0
Truvada prescriptions	53	5	42	47	47	0	0
PrEP counseling	47	0	32	5	0	0	0
Assess substance abuse/mental health	53	0	26	11	11	5	5
Communicate need for clinic follow-ups	47	5	21	11	0	11	11
Schedule 1-month follow-ups	42	5	5	32	32	0	0
Start appropriate vaccine series	42	11	5	32	37	5	5
Additional Variables of Interest
HIV Testing	84	0	11	68	79	0	0
PrEP Risk Assessment	74	5	37	26	5	0	0
PrEP Referrals	53	11	32	11	21	0	0
PrEP Follow-up Visits	47	5	11	37	32	5	5
Behavioral health referrals	63	0	21	26	26	5	5
Outreach/ case management visits	58	0	32	16	5	0	0
STI Testing	74	0	0	68	68	0	0
Pregnancy testing	58	0	5	47	47	11	11

Note. This table illustrates variables related to the prevention of HIV and how the subject recruitment venues (SRVs) in the described study collect and store data. Take the demographic variable of ‘Age.’ In the interview, we asked subjects whether their site captured age as a variable in their EHR. The first column denotes the percentage of SRVs capturing the age variable in the context of HIV prevention. The second column denotes what form the data capture took (i.e., did they use a paper form or electronic entry). The third column denotes the percentage of SRVs using either a specific template question or an open field in their electronic record system to capture the age variable. The final column denotes the percentage of SRVs using paper documents to capture that particular variable. Therefore, for the ‘age’ variable, the table should read as follows: 95% of respondents captured ‘age,’ and 21% of those used paper forms, and the remaining 74% used electronic data capture. Of those using electronic capture, 95% used a specific template. Finally, none used paper documents to store ‘age’as a variable.

There was some variation in the modalities employed for the collection and storage of patient data. For example, SRVs comfortably used EHRs to capture patient demographic information. However, for other prevention categories, some utilized EHRs while others relied on paper forms. When EHRs were used, specific templates or radio buttons were often employed, but data collection for some variable categories was better suited to open text fields (see Table 02). Paper forms were often used to collect prevention variables under the CDC checklist.

We also identified HIV care and treatment variables: demographic characteristics, diagnosis characteristics, use of antiretroviral therapies (ARTs), HIV and general health management, behavioral risk factors, behavioral health status factors, and other medical status factors (see Table 03). Most of those variables were captured using EHRs via specific templates. Some SRVs used paper forms for information capture in addition to their EHRs. Behavioral variables or variables needing additional context (e.g., sexual orientation) were sometimes captured using open-text fields. Most SRVs did not rely on paper documentation as a means of data storage.

Table 3.

Table denotes now sites handle HIV prevention variables (reported as a percentage of the 19 ATN sites queried)

HIIV Care Variables	Site capture of variable (%)		Data Capture Format (%)		EHR capture (%)		Paper Documents (%)
	Yes	No	Paper	Electronic	Templates	Open fields
Demographic Variables
Visit Date	100	0	26	68	95	0	0
Current Age	100	0	26	68	95	0	0
Date of Death	79	16	21	58	68	11	0
Gender	100	0	26	68	89	0	0
Sexual Orientation	100	0	32	63	63	26	0
Race	100	0	26	68	95	0	0
Ethnicity	100	0	26	68	95	0	0
Zip Code	100	0	26	63	89	0	0
Diagnosis Characteristics
Age at HIV Diagnosis	89	5	21	63	74	16	0
Date of HIV Diagnosis	84	5	21	58	68	16	0
Mode of Transmission	84	5	26	53	63	21	0
Levels of ART
ART Acceptance	89	5	16	63	58	32	0
ART Status (e.g., adherence)	84	5	11	68	58	26	0
HIV Treatment
Date of Entry into Care	100	0	16	79	84	16	0
ART Rx	95	0	5	84	89	5	0
Viral Load Results	95	0	5	84	89	5	0
CD4 Cell Count	95	0	5	84	95	0	0
Other Rx/Non-HIV	89	5	5	79	79	11	0
Behavioral Risk Factors
Smoking Status	84	11	0	79	63	16	0
Substance Abuse	79	16	0	74	53	21	0
Behavioral Health Status		0	0	0	0	0	0
Mental Health	95	5	11	79	63	32	0
Sexual Risk	84	5	16	63	37	47	0
STI Results	89	0	0	84	84	11	0
Other Medical Status
Other Diagnosis Codes	95	5	0	84	89	0	0
Height/ Weight	100	0	0	89	95	0	0
BMI	100	0	0	84	89	5	0
Blood Pressure	95	0	5	74	84	0	0
Cholesterol Panel	95	0	0	79	84	0	0
Blood Glucose	89	5	0	79	84	0	0
Pregnancy Results	95	0	0	84	89	0	0

Note. This table illustrates variables related to the care of HIV and how the subject recruitment venues (SRVs) in the described study collect and store data. Take the demographic variable of ‘date of death.’ In the interview, we asked subjects whether their site captured ‘date of death’ as a variable as part of theirh HIV care modalities. The first column denotes the percentage of SRVs capturing the ‘date of death’ variable in the context of HIV care. The second column denotes what form the data capture took (i.e., did they use a paper form or electronic entry). The third column denotes the percentage of SRVs using either a specific template question or an open field in their electronic record system to capture the ‘date of death’ variable. The final column denotes the percentage of SRVs using paper documents to capture that particular variable. Therefore, for the ‘date of death’ variable, the table should read as follows: 79% of respondents captured ‘date of death,’ and 21% of those used paper forms, and the remaining 58% used electronic data capture. Of those using electronic capture, 68% used a specific template. Finally, none used paper documents to store ‘date of death’ as a variable.

General Use Barriers and Facilitators

We identified three groups of EHR barriers: system-specific, user-specific, and institutional or organizational barriers. EHRs that were not user-friendly, thought of as “difficult” or “cumbersome” to use, were categorized as having system-specific barriers. EHR implementation was described as having little oversight or input from clinical staff or was based on idiosyncratic decision-making, and sometimes dictated by EHR content and processes. Barriers hampering EHR use included personnel preferences, generational gaps between worker classes, and the need for specialized training to manage and use EHRs.

For institutional/organizational barriers, EHRs served general clinical care needs instead of HIV specific care and treatment needs. Furthermore, institutional priorities, controls, and guideline limitations were said to stifle input on EHR implementation, policy, and vendor updates. Other barriers included a lack of internal EHR infrastructure and support systems, as well as regulatory hurdles, such as data sharing agreements and data security guidelines.

Long-term EHR users pointed to facilitating factors, such as improvements in usability and accessibility, and the development of standards (e.g., Fast Healthcare Interoperability Resources (fhir®) standard (Health Level Seven, Inc., 2019)). EHR developers appear to be moving beyond fundamentals, such as simple workflow and billing and are striving to meet complex health systems’ needs. Consequently, there have been improvements in template designs, data fields, and checklists, making EHRs better suited for capturing patient characteristics and psychosocial variables. Also, EHRs offer data entry and extraction options to promote consistent and accurate data entry and integration across units (e.g., clinical and pharmacy).

Data Collection Barriers and Facilitators

We identified five data collection barriers: forms and templates, general data management, provider/personnel challenges, institution level, and patient population. EHRs typically have multiple data collection forms. These data forms often lack critical fields and standard templates, often designed to facilitate data collection. Furthermore, the absence of such templates may result in burdensome data entry procedures for personnel. Data management barriers include laborious and tedious entry procedures and a lack of standardization across and within clinical units. Within SRVs, different units were noted to have varying procedures and conflicting data priority needs. For example, a clinical unit could have one set of data needs, compared with a research unit.

Time constraints within clinical environments were barriers, compelling providers to balance data entry requirements with meaningful patient interactions. Similarly, provider biases during data entry may affect data type and overall quality. Some patient populations exhibit domestic transience, homelessness, multiple residences due to bi-nationality, and incarceration. These kinds of patient characteristics impose barriers to the collection of certain types of data. Similarly, characteristics such as patient health literacy, and a hesitance to disclose sensitive information, may impact the type and quality of data entered into EHRs by providers.

At the institutional level, some SRVs had an absence of guidelines to support data collection. Furthermore, innate organizational characteristics, such as multiple reporting steps, may impose barriers to data collection. Also, institutional characteristics such as residency programs that experience turnover of personnel may impose barriers due to a need for continuous training of new staff on existing EHR modalities. Lastly, the organizational mission or its for-profit or not-for-profit status may impose additional barriers to data collection.

Facilitating factors were categorized into four groups: institutional facilitators, “people” factors (i.e., provider, personnel, and patient-related facilitators), facilitators to enhance data collection tools, and procedural facilitators. Some SRVs were noted to have guidelines for data collection, and others offered onsite technical assistance to improve data capture. Similarly, evolving leadership has led to balancing data capture standards with realistic expectations of what can be accomplished in a demanding clinical environment. We categorized these observations as institutional facilitators.

Regarding “people” factors, several SRVs now encourage provider input during EHR implementation while also providing training on new procedures and systems. Patient altruism was highlighted as a facilitating factor. Appealing to the goodwill and understanding of patients regarding the value of their input to research data can increase the quality and quantity of data collected. Other stakeholders reported allocating time for meaningful patient-provider communication and relationship building. Additionally, personal outreach, rather than relying solely on an automated response system like text alerts, could enhance information gathering.

To support data collection, stakeholders and experts suggested modeling templates after official guidelines, designing EHRs to mimic widely used official databases (e.g., health department databases), and harmonizing variable definitions. Lastly, they suggested incorporating data collection into routine care workflows, as such workflows are critical to data monitoring and quality assurance practices.

EHRs for Surveillance and Research

We identified four categories of barriers: barriers specific to EHR systems, institutional-level barriers that impact data extraction, barriers affecting the generation of data, and general personnel barriers. With system-specific barriers, respondents’ SRVs often needed to engage EHR vendors regularly, as they are best suited to extract certain data types. Also, EHR design limitations can impede data access. Within institutions, regulatory and legal impediments can impact what data may be extracted and for what purpose. Also, internal bureaucratic factors can slow or prevent access. On the matter of data generation, interviews highlighted unstructured data generated from open text-fields as a barrier. Furthermore, data extraction could be labor-intensive, imposing demands on personnel time and economic costs on the SRVs. Finally, some SRVs may lack personnel with the time or expertise to support consistent data extraction.

Using EHRs for disease surveillance purposes may benefit from emergent factors. For example, natural language processing (NLP) may be used to develop algorithms for data extraction within institutions. Research demands on EHRs may stoke institutional efforts to harmonize variable definitions across different units (and in the broader health sector). Further, categorizing disease surveillance as an institutional objective may encourage study designs, which appropriately account for the time and costs of using EHRs. Such an objective may also encourage quality assurance measures, such as periodic and systematic data audits of an EHR system.

Personnel and Resource Management in Support of EHRs

Staff turnover leaves gaps in expertise, especially in the use of EHRs, requiring new staff training. When implementing EHRs for data capture, certain categories of staff, particularly those with patient interactions, may experience additional demands on their time. Conversely, opportunities to improve EHR use include ongoing staff training and EHR engagement efforts. Forging and strengthening collaborative relationships with IT personnel may facilitate research and help cultivate expertise over the shared goal of efficient EHR usage. Lastly, having institutional leaders who are “champions for change” can promote EHR use and innovation.

Data generated from EHRs may not adequately serve research needs, partly because EHRs are typically developed for purposes other than research (e.g., billing). Furthermore, several SRVs utilize data capture templates in their EHRs that lack specificity. Also, the scope of some EHRs makes it difficult to aggregate relevant data for research. Conversely, participants noted that SRVs that were part of a larger medical department or clinic often received support (e.g., administrative, financial, or technical) for EHR use.

Capture of HIV Specific Variables

The interviews suggested difficulties in capturing variables often used by HIV researchers (e.g., date of death, gender identity, and sexual orientation). Difficulties were also noted to come from inconsistent capture modalities. Similarly, existing social mores can make it difficult to capture variables representing a specific patient population’s characteristic. However, factors or variables such as laboratory test results, referral information, and appointment history were relatively easy to document, and the consensus perspective from the interviews was that such specific information was consistent even in different EHR systems.

Discussion

The interviews highlighted a fractured EHR landscape within the ATN, and perhaps in the national HIV care system. Furthermore, several of the EHRs employed by Network SRVs, serve different functions, mainly workflow management, billing, and record storage. This variability in EHRs, the functions they serve, and their implementation can hamper their use as tools for research. This investigation clarified some of these barriers and identified facilitating factors to enhance EHR use for research.

Our findings corroborate other EHR use studies. For example, a survey of primary care providers pointed to barriers such as individual-level characteristics (e.g., typing speed), EHR attributes (e.g., computer processing ability), and broad organizational characteristics (Linder et al., 2006). Similarly, Holden (2011) interviewed physicians from community hospitals and highlighted time allowances, organizational environment, and inter-organizational integration as barriers to EHR implementation. Lastly, some research points to the unintended consequences of EHR implementation (Colicchio et al., 2019). Our findings on usability, data entry challenges, and end-user burden corroborate some known unintended consequences.

EHR interoperability is an emergent issue, and the variety of EHRs across the ATN and within SRVs themselves confirm this fact, especially since efforts to aggregate data can pose a challenge. Such impediments to patient data aggregation or integration speak to the need for more coordination between vendors or developers and their end-users. Additionally, research relying on data from multiple SRVs may need to draw on existing guidelines, such as the Office of the National Coordinator for Health Information Technology’s (ONC) interoperability roadmap (ONC, 2015). The ONC is a federal entity responsible for coordinating nationwide efforts to implement advanced health information technologies and electronic health data exchange. EHR interoperability challenges will need to be addressed in the ATN before EHRs can be used for substantiative investigative studies.

The end-users we interviewed highlighted some engagement challenges. For example, some characterized their user experience as cumbersome, and in some cases, rather than easing administrative challenges, merely compounded those issues. The National Experts confirmed the perspectives of end-users and offered some solutions. They suggested integrating end-users into the process of implementing EHRs and consulting with end-users to develop clinical interventions and programs that capitalize on EHRs. These are suggestions supported by the literature, where it is noted that investments in staff training, and the provision of technical experts to guide end-users as they interface with EHRs, are a means to avoid some engagement challenges (Salloum et al., 2018).

The reported findings should be interpreted within the limitations of our study. Firstly, our findings may not apply to a national setting as our study investigated the ATN’s experiences. However, the interviews produced insights applicable to similar settings, describing challenges that youth care sites may encounter as they implement EHR systems. Secondly, we interviewed Key Stakeholders from 19 SRVs and 10 National Experts, all of whom self-selected to participate. Consequently, their views may not represent the variety of perspectives on EHR use and implementation. Furthermore, the reported data may not fully represent the unique perspectives of SRV personnel because we typically spoke with one stakeholder from each site. However, the perspectives provided served our broad objective of identifying the barriers to and facilitators of the collection and aggregation of EHR data. Lastly, the sample size of 19 SRVs with approximately one stakeholder from each, while small, results from the qualitative nature of our endeavor, which made a larger study sample size practically untenable. However, the depth of our interview topics offset the relatively small number of study participants.

Conclusion

The perspectives proffered by National Experts and Key Stakeholders will inform implementation strategies in developing a second phase to the EHR-COC protocol (i.e., ATN 162b). This secondary phase aims to routinely monitor the HIV prevention and care continua across the ATN and conduct a pragmatic clinical trial of an eHealth intervention geared towards improving youth adherence to care. Our findings help clarify HIV specific variables common to SRVs within the ATN and denote how such variables are typically captured. Furthermore, our understanding of the key barriers and facilitators to EHR use has helped us develop a monitoring strategy that pairs remote data capture with regular EHR data downloads from participating studies. Lastly, our understanding of EHR use challenges has informed intervention design in the second phase, serving to tailor it to better suit the characteristics of participating SRVs.

Abbreviations/ acronyms

ATN

Adolescent Medicine Trials Network for HIV/AIDS Intervention

ARTs

anti-retroviral therapies

HRSA

Health Resources and Services Administration

EHR-COC

Electronic Health Record Monitoring of the Continuum of Youth HIV Prevention and Care

NICHD

Eunice Kennedy Shriver National Institute of Child Health and Human Development

ONC

Office of the National Coordinator for Health Information Technology

SRV

Subject Recruitment Venue