Data-Related Processes and Challenges at Level I–IV Trauma Centers in Arkansas: Results of a Mixed-Methods Case Study

Abstract

Background:

Clinical data registries provide a rich source of real-world data that can be leveraged by clinicians, researchers, and public health professionals to address some of the current health challenges faced in society today. The relative usefulness of a registry depends on the ability to gather data and the overall quality of the data. To assess existing datarelated processes, including barriers and facilitators to data collection and submission, we conducted an observational case study to evaluate sites submitting data to a prominent state-based trauma registry.

Methods:

A mixed-methods approach was undertaken to evaluate existing processes and barriers to data collection for the Arkansas Trauma Registry (ATR). A series of interviews and observations were conducted with trauma registry personnel from level I–IV trauma centers across the state of Arkansas to collect data on current data-related processes. To facilitate observations, a think-aloud protocol was used to gather keystroke-level modeling (KLM) data. Additional observational data (qualitative) were collected regarding site processes and workflows pertaining to the collection and submission of registry data to the ATR. Following the observations, informal, semi-structured interviews were conducted to assess the participants' perspectives on current data-related processes, potential barriers to data collection or submission, and any recommendations for improvement. All sessions were recorded, and de-identified transcripts and session notes were used for analysis. Quantitative analyses were performed on the KLM data derived from observations to determine time spent performing end-to-end registry-related activities. Qualitative data from interviews were reviewed and coded by 2 independent reviewers. The qualitative codings were adjudicated by the reviewers using a consensus-driven approach. Themes were then extrapolated to generate the final set of results.

Results:

Seven unique staff members (trauma registrars, coordinators, supervisors, and directors) participated in the study, having completed both observation and interview sessions. These participants were from 5 unique trauma centers (one level I, one level II, two level III, and one level IV). Through the observations, we were able to characterize the typical operational flow for level I–IV trauma centers participating in the ATR, confirming a primarily manual process was used across all sites. Furthermore, the KLM analysis demonstrated that, on average, site staff would need close to 26 total hours to identify, abstract, and transcribe a single, relatively complex trauma registry case (meaning all 288 registry data elements were captured). Results from interviews further emphasized the exhaustive nature of the current data collection processes across sites, regardless of trauma level classification. Five common themes were identified across all 7 interviews: data quality; manual processes; resources and technology; site environment and staffing; and training. Across each theme, interviewees provided their perspectives on various activities and procedures at their site, as well as on the overarching trauma registry program. Many positive perspectives were shared, and several negative perspectives and perceived issues were noted. Interviewees also provided recommendations for improving internal site processes and streamlining the way in which data are collected and submitted to the ATR.

Discussion:

Automation of data entry, improved training resources, and adequate staffing are critical areas of improvement for efficiently abstracting registry data. By streamlining manual inputs into the registry, registrars could minimize time spent consolidating and abstracting recurring data. Conclusion: These results illuminate existing data-related practices of sites participating in the ATR and the outcomes measures for process improvement efforts. These challenges are not unique to the trauma domain and are encountered by registries across a variety of therapeutic areas. Thus, attempts to develop integrated and interoperable solutions to streamline and improve data collection would benefit all registries.

Background

Clinical data registries are a rich source of health information derived from local, state, and national sources. Federal regulations require reporting of health care data to registries to produce a longitudinal dataset for use in clinical practice, research, and public health operations. To be most effective, data submitted to registries must be timely and of high quality if we aim to use this information to improve patient and population health outcomes.¹

Sites participating in registries are required to collect and submit data on an ongoing basis. For trauma registries, the data collection systems are usually composed of uniform data elements that describe the injury event, demographics, prehospital information, diagnosis, care, and outcomes of injured patients. In most cases, sites commit an extensive amount of manpower to collecting, coding, performing quality reviews of, and submitting data—all activities that are typically performed manually. Studies have shown that these manual approaches can be expensive and inefficient, burdensome for sites, and not unique to any one registry.^2–5

Here, we evaluate a prominent state-based trauma registry (which funnels data to a national registry) to better understand existing data-related processes across sites. The Arkansas Trauma Registry (ATR) is a statewide data collection system designed to collect trauma-centric data and evaluate them. The goal of the ATR is to identify and drive process improvement activities that will reduce morbidity and mortality from traumatic events.⁶ The ATR aims to gather, code, and organize this information for analysis and reporting of both individual and collective results.⁶

Trauma centers in the United States are designated into different levels to signify the available resources and the number of patients admitted yearly. They are classified into 5 levels, with level I being the highest and offering the most extensive amount of care, whereas a level V center provides initial evaluation, stabilization and diagnostic services, and preparation for transfer to higher trauma care levels.⁷ Arkansas has level I to IV trauma centers, all of which participate in contributing to the registry entries.⁶

These data are also contributed to the National Trauma Data Bank (NTDB), which is the largest compilation of trauma registry data in the United States.⁸ Various reporting sites often depend on manual medical record abstraction (or chart review) to collect state-level trauma data from electronic health records (EHRs) and emergency medical services (EMS) systems prior to submitting to a statebased and/or national registry. These methods are typically burdensome, costly, and inefficient because the processes are manual and people-driven, lacking automation. The personnel involved in this process include trauma registrars, registry coordinators, and program managers who are directly engaged in data collection. The aim of this study was to incorporate the insights of these experienced trauma team members to identify the current barriers and facilitators to data collection and submission to determine how best to streamline and improve data collection and submission processes.⁹

Methods

A mixed-methods, observational case study design was used to gather information from site personnel at level I–IV trauma centers across the state of Arkansas on current, end-to-end data collection processes for or to the ATR.⁹ This study received a determination of not human subjects research as defined in 45 CFR 46.102 by the University of Arkansas for Medical Sciences (UAMS) Institutional Review Board.

Clinical, research, or data registry personnel active in the day-to-day data collection and submission processes were included in this study. We targeted the inclusion of at least 1 person per site across 4 unique sites representing the different trauma center certification levels (a minimum of 4 total participants, one from each a level I, II, III, and IV trauma center) for a more representative sample. Recruitment was carried out until data saturation was reached.¹⁰

Semi-structured observations and interviews were conducted virtually, and recordings were used for analysis. Sixty minutes were allocated for each session (one 30-minute session for the observation period, and one 30-minute session for the interview). The purpose of the observations was to allow trauma registry personnel to walk through current site-level processes using a think-aloud protocol^11,12 to collect qualitative information to assess current processes, challenges, and perspectives of the data-related practices carried out at each site (primary objective).⁹ An interview guide was developed and referenced to ensure consistency across all observations and interviews.

Keystroke-level modeling (KLM)^13,14 was also conducted on the quantitative data collected during observations to calculate time-on-task for each activity outlined by registry staff (secondary objective). To calculate this measure, we used the 9 KLM operators outlined in Table 1 to aid in translating a task or activity to a time or duration. These were adapted from the work of Card, Moran, and Newell.¹³ Prior to conducting the KLM, we also made a few assumptions based on the information gathered from the observations and interviews, which are noted in Table 2.

Table 1.

Keystroke Level Monitoring (KLM) Operators¹³

Operator	Activity/task	Time (seconds)
T(n)	Type a sequence of n characters on a keyboard	n × Ka
P	Point with mouse to a target on the display	1.1
B	Press or release mouse button	0.1
BB	Click mouse button	0.2
H	Home hands to keyboard or mouse	0.4
M	Mental act of routine thinking or perception	1.2
W(t)	Waiting for the system to respond (time t must be determined)	0b
N	Notation onto a paper form	1.2
K	Average nonsecretarial typist (40 words/min)	0.28c

^aWe assumed 50 characters per line of text. Therefore, for 10 lines of text, the total n is 10 × 50, or 500.

^bIn many cases, with modern computers, the waiting time is essentially negligible. In other cases, such as database retrieval or website usage, the delay might be substantial, but still the same regardless of which interface design is under consideration. In such cases, it may not be necessary to include this operator, either because it takes near-zero time, or it has the same value in all the alternative designs, and so affects only the absolute, not the relative, task times. For this study, we assumed W(t) to be zero (0).

^cWhile we made the assumption that our registrars were able to type at a rate similar to an average, nonsecretarial typist (or 40 words/min), this EMS, emergency medical system; EHR, electronic health record; ATR, Arkansas Trauma Registry; CRF, case report form.

Table 2.

Keystroke Level Monitoring (KLM) Assumptions^a

1	All required registry data live in the EHR and on paper forms (ie, EMS paper form).
2	Registrar has 2 monitors available to alternate between 2 different programs at 1 time.
3	Registrar already has access to the EHR, relevant paper forms, and reports.
4	Registrar abstracts to paper source first.
5	Registrar then enters data from paper form to electronic form.
6	In the registry portal, the form autosaves after each data entry/selection.
7	This was a complex case, requiring all fields to be abstracted and entered into the ATR.
8	There were 288 total data elements (based on the paper registry CRF).
9	There were 13 sections containing all data elements (based on the paper registry CRF).
10	Registrar is an average, nonsecretarial typist (40 words/min).
11	In general, the default character amount used was 10, unless we were able to confirm otherwise.
12	Estimates accounted for (1) case identification, (2) case abstraction, and (3) data entry only and did not account for medical coding or quality control measures.

^aWe used registrar as the default term to cover all roles associated with the ATR.

Interviews and qualitative coding were conducted by trained members of the research team following a rapid qualitative evaluation approach^15–17 to understand staff perspectives on existing processes (primary objective). Additional details regarding the methods used are included in our previously published protocol manuscript.⁹

Results

Participant Recruitment

The study team collaborated with the Arkansas Department of Health (ADH) to identify 7 level I–IV trauma centers across the state to contact for inclusion in our study (one level I; one level II; two level III; and three level IV). ADH was able to provide 1–2 contacts (names, work emails, and work phone numbers) for each site, and the study team made attempts to contact all sites accordingly. Of the 7 sites, we were able to contact personnel from 5 sites, each representing a unique trauma level certification (one level I; one level II; two level II; and one level IV). Two of the 7 sites did not respond to email or phone communications, and after 3 attempts via each mode of communication, we decided to exclude them from the study. In total, we were able to recruit 7 unique staff members across the 5 sites to participate in the observations and interviews (Table 3).

Table 3.

Sites and Personnel Identified for Participation in the Study^a

Site identified	Trauma level	Contacted	Total participants	Participant roles
A	I	X	1	Trauma director
B	II	X	2	Trauma supervisor; registrar
C	III	X	1	Trauma registrar
D	III	X	1	Trauma coordinator
E	IV	–	–	–
F	IV	X	2	Trauma supervisor; registrar
G	IV	–	–	–

^aWe did not seek out additional sites or personnel, as we were able to (1) meet our criteria of having at least 1 representative from each trauma level certification, and (2) reach saturation with this set.

Observations

Through the observations, we were able to generate general, high-level data and workflow diagrams. Figures 1 and 2 combine the data and workflows into a more simplified depiction of an operational flow generated from discussions with all 5 sites. Briefly, the operational flow begins with a trauma case entering the emergency department (ED) and the generation of a paper chart. Cases brought into the ED via an ambulance often had external EMS data that needed to be collected from the EMS service provider. This typically required a telephone call to the EMS service provider to obtain a PDF copy of the patient chart, which was then uploaded into the patient's file in the site's EHR. All other data were typically charted on paper first (given the fast-paced nature of the ED environment) and then transcribed into the EHR later. Using nurses' notes and, in some cases, a preprogrammed EHR report, the trauma registrar was able to sift through the newly admitted patients from the previous day(s) to identify cases for the trauma registry. The trauma registrar was able to refer to both sets of records (paper and electronic) to gather case information for the registry, which was often collected on a paper registry case report form (CRF). Once certain clinical data were abstracted, a medical coder (who may or may not have been the same person as the trauma registrar, depending on the site) was able to perform the coding and document those codes on the paper registry CRF with the rest of the case information. On rare occasions, a site may have waited until the paper CRF was transcribed to the electronic system before performing coding. In these cases, the coding was done directly from the EHR to the electronic registry system. (In the state of Arkansas, the electronic registry system is a state-based database developed and maintained by the ADH. Trauma centers across the state have access to a user interface, either through a web-based system or an application, through which they can enter data electronically to submit to the ATR. In this manuscript, the terms electronic registry system, electronic system, or registry refer to the state-based database managed and maintained by the ADH.)

Figure 1.

High-level Workflow Diagram for the Arkansas Trauma Registry (ATR)

Figure 2.

High-Level Operational Flow Diagram for the Arkansas Trauma Registry (ATR)

CSV, comma-separated values (.csv file); DB, database; EHR, electronic health record; EMS, emergency medical services; ER, emergency room; QC, quality control; NTDB, National Trauma Data Bank.

Once the case was fully abstracted, a data entry person (again, who may or may not be different from the registrar) manually transcribed the case information from the paper trauma CRF to the ATR registry via a user interface (some sites had a local instance of the software, others used a web-based platform). The ATR then reviewed the data and performed quality control (QC) measures as defined by their standard operating procedures. In cases where the ATR identified inconsistencies with the data, the ATR would contact the site. The ATR would send quarterly reports to the site assessing specific quality measures. Sites may exchange data back and forth with the ATR either through the portal's user interface or, in some cases, via CSV data files in response to data queries. At some frequency (which varied by site), the data were then submitted to the NTDB as applicable (note that the ATR and NTDB request nearly the same dataset but with some differences). QC at the site could occur throughout the entire process or in batches at some specific interval (monthly, quarterly, etc).

We also attempted to calculate the approximate time spent carrying out common data-related activities using KLM. From the information gathered through the observations, we were able to identify 5 major categories to classify the workload: case identification, case abstraction, medical coding, QC, and data entry (Figure 1). As the QC processes could vary significantly based on cases and sites, we did not account for that in our KLM calculation and instead focused on a direct identification-to-abstraction/coding-to-data entry process. With that in mind, we were able to estimate that, on average, regardless of trauma level, a complex case requiring all 288 data elements to be completed would take approximately 26.18 hours, or a little over 3 working days, to complete (given the assumptions provided in Table 2). To clarify what is meant by “complex,” this would likely be a case admitted to the intensive care unit on site, and that the site likely submits data to both the state and national registries (as opposed to just the state registry). This is in line with what one site shared in the interview session, noting that, for coding alone, “…it could take you anywhere from say 15 minutes…to a case that might take you all day.” It is important to note that not all sites participate in the NTDB, and some sites participate in both the state and national registry as well as a Trauma Quality Improvement Program. Therefore, the number of required data elements varied across sites, albeit all participating study sites reported to both the state and national registries.

Interviews

There were 8 domains of interest referenced during the analysis of the interviews. Within each domain, a set of categories or themes were identified based on discussions with interviewees. Five common themes affecting data collection and submission to the ATR were identified across all 7 interviews (Table 4): data quality; manual processes; resources and technology; site environment and staffing; and training. Saturation was reached with this set of interviewees.

Table 4.

Common Themes Identified Across All Interviews^a

Themes	Interviewees by site
	A	B1	B2	C	D	F1	F2
1. Data Quality	X	X	X	X	X	X	X
2. Manual Processes	X	X	X	X	X	X	X
3. Resources & Technology	X	X	X	X	X	X	X
4. Site Environment & Staffing	X	X	X	X	X	X	X
5. Training		X	X	X	X	X	X

^aInterviewees are listed by site (see Table 3). “A” is the single participant from Site A, “B1” is the first of 2 participants at site B, etc. An “X” indicates that the theme was discussed during the interview.

Data Quality

The focus of the data quality theme included anything pertaining to the quality of the data collected and submitted to the registry. The quality of data either fuels or restricts the quality of research and trauma care delivered to patients who experience traumatic injury. For example, trauma centers used data-based, computational indexes such as the Trauma and Injury Severity Score to estimate mortality and aid in clinical triage, decision-making, and quality assessment.¹⁸ The data-driven nature of trauma care required data quality to be a key area of interest for the research team, and, therefore, one of the primary domains centered on data quality (general perceptions on overall quality of data submitted to ATR). Each interviewee was asked about site activities or standard procedures specific to data quality management, as well as their individual perspectives on the overall quality of the data their site was submitting to the registry. Overall, all interviewees indicated high confidence in the quality of the data submitted, primarily because each site had robust QC measures in place.

Notably, previous studies have found that QC may positively affect data quality by reducing and maintaining lower error rates in clinical research.^19–21 Here, these QC measures and processes differed across sites, with one interviewee indicating that “80% of trauma [registry] work is QC,” while another indicated that they had a variety of QC “stop-gaps” in place (including monthly case reviews and quarterly QC and performance improvement meetings) to ensure high quality data were submitted, and yet another noted, “10% to 30% of charts are validated per month based on the volume of cases per month.” Several interviewees mentioned that there are mechanisms in place on the registry side (eg, at the health department) that also QC the data and provide a scorecard or report back to the site to review any questionable data points. One interviewee noted that they have consistently performed well, receiving high marks from ADH, “The registry through the health department has a section that gives us feedback, and so they double check what we put in and we get basically a score card from them… [the registrar] gets a really good score card from the health department, and usually even when they've said, please look at this, usually [the registrar] shows them where it is, and [the registrar] actually has done exactly what's supposed to be done.”

While the overall quality of the registry data at each site was considered high (positively perceived by interviewees), there were some negative perceptions held by several interviewees across different sites. Multiple interviewees noted that they often dealt with (the lack of) timeliness of data entry or missingness—data either not entered in a timely manner (requiring multiple days or weeks of rechecking charts) or data were not available at all (due to clinician oversight, delays in charting, or delays in access to EMS data). Data accuracy was also discussed in several interviews, with a few interviewees noting that sometimes they had conflicting data and would often rely on the paper trauma chart over the digital EHR data because of the assumption that the paper chart would have been documented in real-time and therefore should be more reliable. That said, another interviewee noted that the manual paper processes (further elaborated on below) could result in delays or missingness, as some clinicians were not always capturing the data on paper in real-time (refer to the section on manual processes).

A recommendation noted by one of the interviewees pertained to the charting process. Inconsistencies in the way clinicians on site are charting affect the data collection process for the registry, “…it's the same barriers everywhere from all the trauma program people that I've talked to… we just need to have our nurses chart better. I mean, I've been a[n] ER nurse for 30 years, and it amazes me, still, to go through the chart, and there's stuff there that's so important. I mean, it's important for us. But then…[i]t's like, I can't believe they didn't chart that. So, I think right now … we're working on that, and I know a lot of the other trauma supervisors that I've talked to are having the same issues.”

Manual Processes

The activities included in the manual processes theme were those that did not rely on technology or any form of automation, for example, paper-based abstraction. Across all sites interviewed, at least one aspect of the trauma registry data collection process (case identification, abstraction, coding, QC) relied on at least one manual process. Interviewees had both positive and negative perspectives of the current manual processes in place, which varied by site and interviewee. For example, one interviewee noted that the manual back-up procedures in place at their site for ensuring no missed potential cases were helpful and functioned as a “safety net” to the trauma registry team. Another positive shared by an interviewee was the ability to use a “cheat sheet” of manually abstracted paper forms as a reference during internal trauma meetings for review of performance indicators and/or quality checks.

However, in most cases, interviewees had negative perceptions of the current manual processes, often offering recommendations for improvements. For example, all interviewees noted that a patient's trauma chart would often originate as a paper form (the data were not always entered on the paper form in real-time) that was then uploaded to the EHR as a PDF. At some sites, this was standard operating procedure, even though it was noted that the nurses at one site did not like using paper charts, and, often, do not complete the paper charting in real-time. One interviewee noted, “…the nurses would much rather just do everything in the electronic health record. But we're still doing the paper chart for trauma and for codes…I think, because it's not in the computer, we have problems with them, not doing it in real time…I think that's a part of the disconnect is that they're thinking because it's on paper oh, I could do that later, but then they're not getting everything they need onto the paper.” Another interviewee indicated that their site used a courier service to collect paper forms, transfer them to another facility, and then upload the paper forms into the EHR at that facility; and, in some situations, paper charts have gotten lost. “So not only do we have that paper chart, but that paper chart was getting picked up in our ER, transferred by courier to [location], it might get lost and never get into our medical records, or we might not see it for 2 weeks.” Several other interviewees also noted that, often, the EMS team does not leave run sheets (paper forms that document the EMS encounter prior to arriving at the site), which results in the site having to contact the EMS facilities to get that information faxed or mailed to the site for upload into the EHR. “…[M]ost of the time, I have to request them [the EMS data] or go on to ImageTrend…that's where EMS is supposed to download all of their EMS Run Reports… that come to our facility.”

Over half of the sites indicated that they first used a paper CRF to abstract cases (collect case information) and then manually transcribed this into the registry. At least 1 site had a multipaper approach, in which a list of potential cases was initially listed on a spreadsheet or paper log (collecting certain pieces of registry-required information). These cases were then reviewed and any confirmed to be trauma cases were entered on a second spreadsheet/log (reentering of information previously collected). Finally, a paper CRF was recorded for these cases (requiring reentry of information previously collected) and used as the basis for completing the full abstract entered into the registry. This meant that the site was reabstracting some of the same data at least 3 times before it went into the registry. In most cases, the manual processes were seen as barriers to data collection and submission because the data on the paper chart (even when complete and uploaded to the EHR) might contradict data that were entered directly into the EHR later. For example, one interviewee mentioned, “…there's a lot, …a big human part of medical records, and so [a] lot of times I'm looking at the trauma [paper] chart, and then I'm looking at what's put in there in the EMR and seeing if they match up…there's some things on the trauma [paper] chart that would trump something else in the record, and that's usually [where] we go, because they should be writing on the trauma chart in real time.” A second interviewee emphasized some of the challenges faced at their site with regards to the manual case identification and data abstraction, “Well there is a difference in handwriting on a manual abstract. It's a challenge. Variations in how you mark it. You know, do you [use] a little X, or do you check next to it, or do you circle the whole thing. That can be nuances. And just fat finger issues you know…blood pressure of 138, glance at it, you think it's 183. You know transposing numbers… You're depending on correct visualization and putting that point in. [Additional] challenges [are] that occasionally you don't have all the information when you're doing that. For instance, [another staff member] brought it up today that…we have [to wait] up to 24 hours for the weight to be entered…It is a gathering process of the pieces of information because EMS, trauma flow sheet, all of that, it might take weeks to get that information.”

Recommendations for improvement were consistent across interviewees. Most of the interviewees recommended there be some mechanism for auto-populating the registry or automatically exchanging data between the EHR and the registry to reduce double-data entry, abstraction time, and site burden. “…[S]omething that would be really nice would be most of the data points in here are data points that every electronic medical record has. There should be a way…it [c]ould pull in data straight into the registry. But to a point, because you know only the ones that I chose, and then I would upload them to this system…That would help everybody.”

Resources and Technology

The resources and technology theme encompassed elements of the ATR process that pertained to registryspecific resources, tools, aids, and/or technology made available either by the site trauma program or by the ATR/NTDB. In general, the ADH was perceived positively as a useful resource for trauma registry teams: “The resources at the Arkansas Trauma Registry for the health department…I can call them, and they can help walk through if I have a question.” All interviewees across all sites noted that they leveraged the state and national guidelines (provided as a PDF download from ATR/NTDB) to verify activation requirements during the case identification process, and all seemed to look at this resource favorably.

It was mentioned by several interviewees that the registry software was slightly different depending on the site. Some sites had access to a local, stand-alone version of the software that was deployed or installed on-site. Other sites had access to the web-based version of the registry. One site noted they had access to both, but preferred the local, stand-alone version to the web-based version because it allowed for customization of features and reports that were beneficial to the site. Through customizations, “We've found a way to make [the registry] work for us.”

One of the level III sites we interviewed also mentioned a unique feature that they had available to them. At this site, there was a mechanism used that allowed for some of the data to get automatically imported into the registry from their internal systems, although, it is unclear as to how much data or which fields beyond demographics were available. (The other sites interviewed confirmed that they did not have access to such a tool.) This capability was perceived positively by the site, as it made the caseload more manageable for them, having only 2 staff members working the registry, “…like I said before, just us two, dealing with 900 and something patients in the registry [per] year is kind of difficult. If we did not automatically import, we would not be able to do that.”

One of the sites that used the local, stand-alone version of the registry also provided additional positives about the system. For example, there was color-coding within the user interface that allowed sites to distinguish between state and nationally required data elements: “…we have initial information if it is a pink screen, a pink item, it is specific to the Arkansas Trauma Registry. If it is a blue item, it is specific to the National Trauma Data [Bank]. That was one thing that the state advocated, which is really great.” Another positive was the notes section provided in each tab that can be used by the site as they saw fit, this site using it to track QC items (eg, inconsistencies in data elements): “Each of these tabs have note sections. We've used them on and off for different things…we've identified a data element that we found out was not entered correctly and may have not been collected correctly. I've used this page to say a note to just provide a little detail on that. And then I have a report that I run. And that way it serves as a great summary when we meet.” This site, along with a few others, also mentioned that the use of filters and customizable reports within the registry system were beneficial and helped with performance improvement efforts at the sites.

That said, some aspects of the registry system were not as useful to sites. For example, one interviewee noted that the custom tab for “quality assurance tracking” required duplicate data entry, a workaround was used to replace the “tracked events” feature, and the “attachments” functionality was not used by the site. Other interviewees also mentioned that the current process for submitting death data to QSource, a separate, third-party system, was redundant and could (or should) be automatically pulled from the registry so that the site was not having to do double-data entry.

In addition to the registry system, several sites benefited from the use of supplemental software that assisted with case identification and acquiring EMS data. Pulsara is an EMS/ambulance communication system that alerts staff to trauma cases coming to the ED. ATCC is a trauma communication center that notifies of incoming traumas. ImageTrend is a system that provides EMS run reports. Not all sites had access to these tools, but those that did found great value in having access to them.

Site-level resources were also discussed. Many sites leveraged working spreadsheets for tracking potential cases, including patients seen in the ED (via an ED log), and for keeping trauma activations organized. These are Excel spreadsheets, and, in some cases, paper logs, developed and maintained on site by the trauma registry team. Across the board, these logs were used as a primary method for tracking cases and managing workflow and the identification and abstraction process. They were often used to determine which cases need to be entered into the registry, which are not “eligible,” which had already been data entered into the registry, and which cases needed to be reviewed at quality/performance improvement meetings: “I have a spreadsheet that has limited activations, full activations, transfers, incoming, nontrauma activations. And then I have dispositions, transfers out, admissions. And of those admissions, what were OR or ICU floor, and then deaths.” Interviewees also noted that trauma registry staff on site have also created “cheat sheets” or paper checklists to assist with the data collection processes: “I have a laminated copy [of the Trauma Activation Requirements] by my desk…because I don't have them, like, totally memorized in my head, so I don't want to, like, miss something…I have created myself my own little cheat sheet, and so I like to read through the whole record before I do the registry.”

Interviewees made note of one barrier to the data collection and submission process. Several interviewees mentioned having issues with submitting data to ADH using the registry system: “And through those processes we have found issues, you know, where not all the records went, that there was maybe an update that was done with one facility but not the ATR. I mean…there's a variety of things that cause an upload to fail.” It was unclear as to how often these “glitches” were taking place, but it was implied that these did not occur often enough to cause too much disruption. However, because the amount of work it takes to go through the full process through to submission is so great, any time a “glitch” occurred, it was felt greatly by the site. That said, it was also implied that the issue did not fully prevent data submission, but delayed submission and caused a bit of extra work to troubleshoot the issue when it occurred at the site.

Most interviewees offered the same recommendations for improvement: there should be a way to pull in the data automatically into the registry from the EMS and EHR systems, which would allow for registrar review and direct submission to the registry once approved by the registrar. Again, as noted during discussions on Manual Processes, interviewees sought some form of automation of the data collection and submission process to reduce some of the burden (refer to the section on manual processes).

Site Environment and Staffing

The site environment and staffing theme focused on the perceptions interviewees had about their sites, which included staffing for the trauma registry program. All interviewees positively perceived the trauma registry team at their site, several interviewees also noting the benefits of having support from the trauma department and corresponding clinicians, “I also think we … have a very good ER team, OR team, surgeons…” A second interviewee further emphasized the importance of having institutional support by leadership at their site, “I think we also have a big support by the ER leadership team and wanting to make sure that that, you know, we succeed in our trauma program; so they're behind us a lot. But so is our medical director of the hospital… trauma is very important to us at [site]…”

Previous studies found that this variability in organizational features (eg, institutional support, staffing, training of the staff) among trauma center levels is a significant factor in the variability of patient outcomes across those trauma centers.^22,23 Relatedly, most interviewees found it useful to have registrars who were clinicians by training (eg, registered nurses) because of the knowledge and expertise they were able to contribute to the team. Some sites benefitted from having former ED/ER nurses on staff as registrars who were already familiar with the EHR and where the relevant data were located, as well as having familiarity with the workflows and common documentation practices of the trauma team on site.

A few sites did share negative perceptions pertaining to the site environment and staffing. One interviewee in particular indicated that registrar hours at their site had been cut for full-time staff from 40+ hours to 32 maximum hours, which made managing the registry workload more difficult: “Now they did recently cut the trauma registrars' hours to 32 a week…And in that aspect, it's gotten a little harder, just since they've cut our hours, but we're still keeping up and meeting our deadlines.” (This seems to have been a broader cost-cutting measure at the site and not something that was mandated by the ATR.) Further, lower trauma level sites (levels III and IV) struggled with limitations in the number of staff allotted to their sites. At the time these interviews were conducted, the American College of Surgeons had the following criteria for sites: 1 registrar for every 500–750 cases (it was anticipated that this would soon change to 1 for every 500 cases). Both level III and IV sites struggled with this given their respective caseloads, having only a single registrar (and a supervisor) to fulfill all data collection and submission duties, “…like I said before, [it's] just us two, dealing with 900 and something patients in the registry [per] year is kind of difficult.”

Unique challenges were faced at the level IV site. One of the interviewees noted that having the level IV designation was limiting for them because they also take in neurological and spine patients, which means they often operate as a level II. However, in the eyes of the ATR, they are a level IV center, which affects both reimbursement and support: “…one of the things that's really frustrating for us is that we're the only level IV trauma system in Arkansas that does neuro and spine, and we operate a lot like a level II…I have to follow the same guidelines that a level II facility does with neuro and spine. However, we get a whole lot less reimbursement just because…we're only counted as a level IV because we don't have 24/7 general surgery and general ortho…” Another limitation faced is the high staff turnover rate, which makes it difficult to maintain consistency.

One recommendation provided by a few interviewees (primarily at the level III sites) was to increase the number of staff on the trauma registry team. It was stated by multiple interviewees that the work was “all-consuming” for sites that see between 500 and 1,000 cases per year: “…so many people around the state that's only got like 2 people or some that only have 1. You really would like to have 3 people [2 registrars and 1 supervisor]…that'd be nice…And I honestly think because that is going to be all-consuming. You're going to have to have somebody dedicated. If you're doing for 500 or more a year, you've got to have somebody dedicated to just doing just that. As opposed to having that same person working out on the floor for 3 days and then working [on the registry]…” Therefore, the data-related processes required for supporting the registry may be optimized with increased registry staff and job redesign of the current registry team's roles, improving the overall working capacity of each site and the outcomes of their patients.

Training

All aspects of training were captured in the training theme. In general, most sites found that the state-offered training courses were beneficial. These courses were “pretty straightforward,” although, one interviewee did note that it might be beneficial for registrars-in-training to come in with some basic knowledge in medical terminology. One of the interviewees at the level IV site was able to provide a breakdown of some of the courses their trauma registry team members had to take: “So there is a manager class that you have to take for the trauma. There's also … the supervisors have to take the same registry class like [registrar] does, because we have to be able to review [their] data. There's also like a rural trauma course that because we're a level IV that we're required to take. There is a topic course, I can't remember all of them. There was like, probably a total of maybe 10 courses that we took. Some of them are how to abstract data… Each class was anywhere from 1 to 2 days, and so I mean, I would say, probably a total maybe spent 18 days taking classes, but it was spread out over like a year and a half period.” Overall, the training was perceived favorably and seemed manageable. Although, one interviewee shared that having come into the role of registrar with no prior registry background, the training was a bit overwhelming, especially as their site did not have a site-level training program to supplement what the state provided: “If you haven't done the registry before, it was a little overwhelming. When I started, I would probably say I was kind of not comfortable. …so some of the trainings, it was hard to understand what…how they, how they affected the registry. Like, it gave you a lot of information, and, at first, I mean, if you haven't done the registry before, it was a little overwhelming, because I didn't know how they relate, like how does this relate to what I'm going to do?”

Many interviewees noted that beyond the training offered by the state, there was no formal site-level training program at their respective sites. It appeared as though the level I and II sites did have some formalized training process in place, but the level III and IV sites did not. One interviewee at one of the level III sites mentioned that, beyond the ATR required coursework, most of the learning was “trial by fire” and many sites had to rely on the knowledge of external registrars from other sites: “…I'm pretty self-taught… That's why I kind of reached out to…different [people] to kind of see what they're doing…[external registrar] was a very big help…and…was a great resource and gave me how to do codes because I had never done coding before.” An interviewee at the level IV site also noted that the impact of not having a formalized training program on site was heavily felt, as they have been impacted by high staff turnover rates. For example, an experienced registrar may leave the site before the new registrar can be hired or can go through onboarding; the new registrar is then left with no one to train them (beyond the supervisor and state-offered classes) at the site, “…trauma personnel are… usually there's no one to train them, because the person before them is already gone. And so, I think that's a real thing that a lot of people need is that network…”

Interviewees did offer recommendations for improvement. One interviewee suggested improvement to the state-offered trainings, noting that the trainings were difficult for more visual learners: “I wish there was more walk through of everything, because sometimes they miss things, and sometimes I don't know if that's important. But I'm a more visual learner. So, when you read a book, and you have that sometimes it's hard to put that into action. But when someone does a video, and they're walking through the trauma registry and talking about what they're doing there how they do that.” Another interviewee recommended that new registrars (with no prior training) have the option to take a basic medical terminology course before going through the state-offered coding course(s).

Discussion

Our observations confirmed that all trauma centers evaluated relied on manual processes, particularly during data abstraction and entry. Analysis of KLM data collected during observations found that staff (regardless of site and trauma level) spent an average of 26 hours (about 3 working days) completing the full data collection process. Through interviews, 5 common themes emerged (data quality; manual processes; resources and technology; site environment and staffing; and training) that were identified as key factors affecting the collection and submission of real-world data to the registry. These results revealed that all sites, regardless of trauma level, faced similar challenges.

The results of this study underscore the importance of addressing these barriers to enhance the efficiency of trauma registry operations at sites. In general, data quality, manual processes, resources and technology, site environment and staffing, and training should be areas of focus for streamlining data collection, exchange, and utilization of trauma registry data for clinical practice, public health, and clinical and translational research. More specifically, automation of data entry (better systems integration and interoperability), improved training resources, and adequate staffing are critical areas for future improvement.

These challenges are not unique to the trauma domain and are encountered by registries across a variety of therapeutic areas.^24,25 The COVID-19 pandemic exposed several issues associated with the existing fragmentation across the US health care system.²⁶ This fragmentation continues to affect how clinical data registries operate, posing a number of challenges. Lack of interoperability has been considered a major rate-limiting factor for the “extraction, integration, and linkage of EHR data for registries,” resulting in the continued reliance on manual (including paper-based) methods.^1,27 In the last decade, federal regulations have been implemented (ie, the 21st Century Cures Act in 2016, final rule effective 2020)²⁸ and a common data exchange standard was selected (Health Level Seven [HL7] Fast Healthcare Interoperability Resources [FHIR] standard)²⁹ to promote interoperability of EHRs. However, many providers and health systems have yet to transition to fully interoperable, bidirectional data sharing beyond the minimum requirements of providing access of electronic health records to patients to conform with information-blocking regulations.^30–33 As such, interoperability and automated data collection remain elusive targets for most registries.^5,34,35

Through our ongoing collaboration with the ATR, we aim to address the gaps identified in this manuscript. Our future work seeks to enhance site capabilities for automated data reporting to improve EHR-to-registry interoperability. We aim to establish a best practice framework for the ATR and for individual trauma centers that can be leveraged by other registries and sites interested in pursuing electronic data sharing and automation. Additional research is needed to evaluate the resources and institutional investments necessary to support automation for EHR-to-registry interoperability, and to evaluate the effects of automation on site burden and data quality.

Conclusion

In this manuscript, we present the results from a mixed-methods, observational case study where we evaluated the data-related processes conducted at and challenges faced by level I–IV trauma centers submitting data to the ATR. These results illuminate existing data-related practices of sites participating in the ATR and the outcomes measures for process improvement efforts. The transferability of these results hinges on the shared structural and operational characteristics of registries. All registries rely on data collection (which may include extraction from multiple systems, such as EHRs) and site-level resources (registrars) to meet their objectives. Thus, attempts to develop integrated and interoperable solutions to streamline and improve data collection would benefit all registries.