Setting the foundation for a national collaborative learning health system in acute TBI rehabilitation: CARE4TBI Year 1 experience

Abstract

Introduction

A learning health system (LHS) approach is a collaborative model that continuously examines, evaluates, and re‐evaluates data eventually transforming it into knowledge. High quantity of high‐quality data are needed to establish this model. The purpose of this article is to describe the collaborative discovery process used to identify and standardize clinical data documented during daily multidisciplinary inpatient rehabilitation that would then allow access to these data to conduct comparative effectiveness research.

Methods

CARE4TBI is a prospective observational research study designed to capture clinical data within the standard inpatient rehabilitation documentation workflow at 15 TBI Model Systems Centers in the US. Three groups of stakeholders guided project development: therapy representative work group (TRWG) consisting of frontline therapists from occupational, physical, speech‐language, and recreational therapies; rehabilitation leader representative group (RLRG); and informatics and information technology team (IIT). Over a 12‐month period, the three work groups and research leadership team identified the therapeutic components captured within daily documentation throughout the duration of inpatient TBI rehabilitation.

Results

Data brainstorming among the groups created 98 distinct categories of data with each containing a range of data elements comprising a total of 850 discrete data elements. The free‐form data were sorted into three large categories and through review and discussion, reduced to two categories of prospective data collection—session‐level and therapy activity‐level data. Twelve session data elements were identified, and 54 therapy activities were identified, with each activity containing discrete sub‐categories for activity components, method of delivery, and equipment or supplies. A total of 561 distinct meaningful data elements were identified across the 54 activities.

Discussion

The CARE4TBI data discovery process demonstrated feasibility in identifying and capturing meaningful high quantity and high‐quality treatment data across multiple disciplines and rehabilitation sites, setting the foundation for a LHS coalition for acute traumatic brain injury rehabilitation.

1. INTRODUCTION

Evidence‐based practices use scientific methods to identify effective treatments and, in conjunction with clinical knowledge, apply best available evidence to patient care. ¹ Select fields of medicine, such as physical medicine and rehabilitation, struggle to establish scientific evidence for available treatments. ² Scientifically rigorous, randomized controlled trials (RCTs) of inpatient rehabilitation treatments and strategies are extremely difficult to design and implement due to varied methodologies associated with care and practice, practical and ethical constraints, lack of systematic definitions of treatments, and highly variable and complex treatment environments. ³ Large samples are required to detect effects among patients who present with conditions of different severities and unpredictable recovery trajectories. Given the high level of variability and complexity in rehabilitation, a learning health system (LHS) approach may serve as a beneficial lens through which to study rehabilitation. ⁴

Friedman, et al. ⁵ identify five components for a fully functional LHS: (1) high quantity of high‐quality data; (2) knowledge representative for all decision‐makers; (3) routine and continuous improvement processes; (4) infrastructure with common set of supportive services for iterative learning cycles; and (5) a culture of continuous learning. Central to a LHS is a learning cycle that continuously examines, evaluates, and re‐evaluates information and creates knowledge. Data are organized into information, transformed into knowledge, and knowledge is applied to performance to be evaluated, resulting in more data to begin the next cycle of learning. ⁵ , ⁶ At the core of a LHS is knowledge gleaned through data and experience. This model, applied to rehabilitation, requires identification of what is effective in inpatient rehabilitation, establishing the first component—high quantity of high‐quality data.

The purpose of this experience article is to describe the conceptual framework and discovery process used to identify the requisite rehabilitation treatment data. We describe the project, Comparing Treatment Approaches to Promote Inpatient Rehabilitation Effectiveness for TBI (CARE4TBI). CARE4TBI is a grassroot approach leveraging representation from various provider groups and decision‐makers from 16 medical rehabilitation centers. The collaborative process we used during Year 1 demonstrates feasibility and successfully sets the foundation for a future and fully functional LHS for acute TBI rehabilitation.

2. LEARNING HEALTH SYSTEM IN INPATIENT REHABILITATION

2.1. The challenge

Acute inpatient rehabilitation is delivered by professionals from multiple specialties. The TBI care team includes physicians, nurses, occupational therapists (OT), physical therapists (PT), speech‐language pathologists (SLP), therapeutic recreational therapists, psychologists, social workers, and case managers. During any 24‐h period, patients encounter multiple providers delivering different treatments of varying lengths of time and intensity. Treatments often target several impairments and functional limitations across stages of a patient's recovery, with interdependencies of impairments and limitations. Furthermore, providers may possess a myriad of therapeutic skills, treatment approaches, and care experiences. Collaborative care adds another level of complexity to researching treatment effectiveness.

Rehabilitation is commonly described as a “black box.” ³ The inability to pinpoint specific treatments for specific patients at specific times creates multiple unknowns (black box) about what precisely occurs during rehabilitation and throughout the entire rehabilitation process. Lack of specificity results in an inability to speak to what directly and causally produces recovery and restoration of function beyond natural recovery. ³ Large‐scale research studies have taken steps to begin to provide answers to what may contribute to better outcomes during and following rehabilitation. ⁷ , ⁸ , ⁹ , ¹⁰ , ¹¹ , ¹² , ¹³ These studies highlight the need for multiple types of data, including patient level (e.g., injury, impairment, function, social determinants, engagement), treatment level (e.g., dosing, duration, activity type, recovery phase, timing, level of assistance), and provider level (e.g., discipline, training, skill set, instruction method, setting, decision‐making), as well as outcomes (e.g., function, impairment, participation, disability). Ultimately, these layers of data would be used to define the individualized treatment plan that will optimize outcomes at the patient level.

Rehabilitation is governed by regulatory requirements impacting access to admission, length of stay, and continued stay, adding another layer of complexity to research. Many of these regulations are not based in evidence, yet shape who can and cannot access rehabilitation. For example, the “3‐Hour Rule,” as it is commonly referred to, requires a patient to actively participate in 3 h of therapy per day or 15 hours across a week. ¹⁴ However, there is no scientific evidence to support the claim a patient will recover more (or less) when provided with 3 h of therapy per day. ¹² , ¹⁵ In fact, some studies have shown that it is not the quantity of time in therapy but rather the content of therapy that is associated with better outcomes. ⁹ , ¹⁰ , ¹³ Understanding what produces the best possible outcomes for a given patient at a given point in their recovery would provide scientific evidence to challenge arbitrary regulations and support advocacy for changes in healthcare policies, which in turn could improve access to rehabilitation.

Justifying the need for high quantity of high‐quality data in rehabilitation is not difficult. As RCTs have substantial challenges, observational designs using detailed and multi‐layered data allow for robust inference. Advanced statistical methods in causal inference, ¹⁶ including those incorporating machine learning methods, ¹⁷ , ¹⁸ , ¹⁹ provide appropriate alternatives to RCTs but require data sets with careful data specification that aligns with key research questions. For example, level of effort is not consistently captured during treatment sessions but has been found to be a key component in predicting some outcomes. ¹² , ¹⁹ The collaborative and interactive environment of rehabilitation presents a challenge to isolating the effect of any single element due to interactions within and between all data layers, and it may be the power of rehabilitation lies within these interactions rather than in any single element. Understanding the mechanisms by which rehabilitation improves outcomes requires the ability to analyze effect modifiers while controlling for confounders. Finally, practices and practice standards evolve over time. The ability to capture naturalistically occurring practices, changes in practices, and adoptions of new treatments allows for ongoing evaluations of the impact of practice changes on outcomes. Research designed to meet these needs, accompanied by data that can address these questions, will provide rehabilitation with the data component necessary to create a fully functional LHS.

2.2. The context

Traumatic brain injury (TBI) is a leading cause of death and disability. ²⁰ There are an estimated 2.5 million TBI‐related emergency department visits per year in the United States, resulting in an estimated lifetime cost of $76.5 billion (2010 dollars). ²⁰ , ²¹ Individuals sustaining moderate‐to‐severe TBI may not return to pre‐injury physical or cognitive function and may require admission to acute inpatient rehabilitation for a period of days, weeks, or months. ²² Since 1987, the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR) has sponsored the TBI Model System (TBIMS) to conduct lifetime longitudinal studies on outcomes following inpatient rehabilitation for TBI. ²³ The more than 700 peer‐reviewed publications resulting from analyses of TBIMS data provide foundational knowledge for understanding acute and long‐term outcomes following TBI—foundational knowledge needed to begin the process of data discovery for treatment. ²⁴ , ²⁵ , ²⁶ While outcomes are routinely collected, we lack data on the input side of the treatment equation.

In 1998, the National Institutes of Health (NIH) convened a Consensus Conference and identified gaps in evidence for effective treatments in acute interdisciplinary inpatient rehabilitation for persons with TBI (pwTBI). ²⁷ , ²⁸ Subsequent systematic reviews through the Agency on Healthcare Research and Quality ²⁹ and the (then) Institute of Medicine ³⁰ concluded that there was a continued dearth of rigorous research on the effectiveness of rehabilitation practices. Since that time, new lines of research have made valuable in‐roads in identifying critical treatment components associated with better outcomes.

The first large‐scale study to investigate, collect, and analyze day‐to‐day treatment in TBI rehabilitation was the TBI Practice‐Based Evidence (TBI‐PBE) study. ³¹ TBI‐PBE demonstrated the feasibility of collecting day‐to‐day treatment data, albeit via manual data entry. TBI‐PBE provided cohort data on patient, treatment, and injury variables and allowed estimation of associations of patient and treatment measures to outcomes at discharge and 9 months post‐discharge. ⁷ Secondary analyses of the cohort dataset further identified treatment components contributing to better outcomes. ⁹ , ¹⁰ , ¹¹ , ¹² , ¹³ TBIMS, TBI‐PBE, and the multiple subsequent secondary analyses studies have made significant contributions to understanding TBI and acute TBI rehabilitation. However, the practice of rehabilitation has remained relatively unchanged. The relative stability of the environment is most evident through lack of change in regulatory guidelines and practice. ³² The limited impact on practice from the amassed knowledge is yet another reason to establish high quantity of high‐quality data as a foundation for a LHS in acute TBI rehabilitation. ³³ , ³⁴ , ³⁵

2.3. The case for a LHS in acute TBI rehabilitation

Past research created manually collected day‐to‐day treatment datasets; however, these practice‐based datasets were derived from cohort study designs and are therefore static in nature. While the TBIMS adds new information on pwTBI for each cycle of follow‐up interviews, no new large‐scale day‐to‐day treatment dataset has been created since TBI‐PBE data collection ended. No capacity exists to study, learn, or understand acute TBI rehabilitation treatments at scale, which in turn means advancing rehabilitation as an efficient and effective care delivery system is unlikely until a mechanism exists to provide continuous data collection. Furthermore, when effective treatments are identified, results can be more efficiently replicated and form the base of evidence to implement change across the industry, eliminating arbitrary regulations that currently govern inpatient rehabilitation. Evidence‐based guidance could improve access to inpatient rehabilitation and lead to personalized treatment plans for pwTBI.

3. COMPARING TREATMENT APPROACHES TO PROMOTE INPATIENT REHABILITATION EFFECTIVENESS FOR TBI (CARE4TBI)

CARE4TBI is a large‐scale research project funded by the NIH National Institute of Neurological Disorders and Stroke, involving 15 civilian TBIMS medical rehabilitation sites and one site from the Veteran's Health Administration Polytrauma System of Care (VA‐PSC) (see Table 1). Additional details for each participating site can be accessed at https://msktc.org/tbi/model-system-centers and https://www.polytrauma.va.gov/system-of-care. The VA‐PSC site participated in Year 1 as a knowledge contributor only and not as a site for eventual EMR transition because of anticipated nationwide EMR reconstruction for the entire VA system during the project time period. Of the 15 sites, 6 sites were utilizing the Cerner platform, and 9 sites utilized the Epic platform at the start of the project. The parent organizations of two Cerner sites were exploring transitioning EMR platforms (i.e., discontinuing Cerner and installing Epic). The representativeness of the TBIMS for typical TBI rehabilitation care has been demonstrated in prior research. ³⁶ , ³⁷ The long‐term goals of CARE4TBI are to compare effectiveness of rehabilitation approaches to improve function and community participation in pwTBI and identify patient and environmental factors that modify the effects of treatment on outcomes. By “rehabilitation approach,” we mean the conceptual framework underlying specific interventions, for example, whether the interventions are targeting functional limitations vs. impairments. CARE‐4‐TBI is prospective, naturalistic, and observational in design. The project intends to capture day‐to‐day treatment activity that occurs in inpatient rehabilitation focusing on the primary therapy disciplines—OT, PT, and SLP. The pragmatic and naturalistic design provides an opportunity for dynamic, sustainable data capture. An efficient and sustainable data capture system would provide the infrastructure to answer a multitude of research questions beyond the goals of the current project as it will include data at the patient, treatment, site, and coalition levels.

TABLE 1.

List of participating medical rehabilitation sites.

State	Project site	EMR platform
AL	University of Alabama at Birmingham Traumatic Brain Injury Model System	Cerner
CO	Rocky Mountain Regional Brain Injury System	Epic
FL	Tampa Polytrauma Rehabilitation Center at James A. Haley Veteran's Hospital
IN	Indiana Traumatic Brain Injury Model System	Cerner
MA	Spaulding‐Harvard Traumatic Brain Injury Model System	Epic
MI	Southeastern Michigan Traumatic Brain Injury System	Cerner
MN	Mayo Clinic Traumatic Brain Injury Model System	Epic
NJ	JFK Johnson Rehabilitation Institute Traumatic Brain Injury Model System	Epic
NJ	Northern New Jersey Traumatic Brain Injury System	Epic
NY	Rusk Rehabilitation Traumatic Brain Injury Model System	Epic
NY	New York Traumatic Brain Injury Model System	Epic
OH	Ohio Regional Traumatic Brain Injury Model System	Epic
PA	Moss Traumatic Brain Injury Model System	Cerner
TX	TIRR Memorial Herman/Baylor College of Medicine/UT Health Collaborative Traumatic Brain Injury Model Systems	Cerner
TX	North Texas Traumatic Brain Injury Model System	Epic
VA	Virginia Traumatic Brain Injury Model System	Cerner

4. METHODS

As a naturalistic, observational study design, the data capture system for CARE4TBI aspired to meet several requirements: (1) Minimal interference in daily clinical operations and preferably embedded as components of typical documentation workflows. Processes that would require therapists to complete data entry beyond their typical daily documentation requirements were viewed as undesirable and antiquated as it is considered a form of double documentation that interferes with typical daily workflow. (2) Data capture occurring at the time of care delivery, with details about treatment documented within time frames for documentation of care delivery, not as retrospective review of clinical notes. These requirements raised the option to capture necessary data directly from clinical documentation and completed daily by therapists. Clinical documentation in rehabilitation has moved to electronic platforms that can provide scripted, templated notes to ensure necessary content is documented and templates can be customized for individual provider workflow, facility‐specific quality assurance, and regulatory needs. Identifying a standardized and structured method to consistently capture relevant clinical data to meet clinical, regulatory, and research needs was necessary. ³⁸ , ³⁹ (3) Changes to clinical documentation to ensure data capture could not result in substantial increase to nonproductive (non‐clinical or unbillable) documentation time for therapists. Financial pressures across the healthcare industry demand quick, effective, and efficient care delivery. Time engaged in documentation is time away from patient care (nonproductive). (4) Changes to capturing data could not disrupt existing data systems that provide regulatory data to governing agencies or to quality and compliance personnel at any of the participating sites. To allow for as much of a naturalistic data capture as possible, these four requirements were used to guide work groups in all phases of data discovery—data identification, data standardization, and documentation design proposals.

4.1. Formation of a collaborative coalition

The large number of sites engaged in CARE4TBI necessitated identifying numerous stakeholders and decision‐makers to participate in data discovery and identification. Both overall project research aims and individual site clinical documentation requirements needed to be addressed. Governance of the project fell to the multiple principal investigators (MPIs; CLB, JB, EMH), Executive and Steering Committees, Consumer Advisory Group, Informatics and Information Technology Advisory Board, and CARE4TBI work groups (see Figure 1). Since CARE4TBI is foremost a research project, researchers were present in all groups and meetings. Specifically, the Executive Committee comprised the project MPIs, the lead co‐Investigator Informaticist, the lead co‐Investigator from the data center, the lead therapists in each of OT, PT, and SLP, content knowledge experts in the field of TBI research, and the Science Officer from NIH. The Executive Committee reviewed results from prior comparative effectiveness research ⁷ , ⁸ , ⁹ , ¹⁰ , ¹¹ , ¹² , ¹³ , ¹⁶ , ¹⁹ and identified the relevant data and associated response details, which were then forwarded to the work groups for incorporation. The Steering Committee comprised all Executive Committee participants plus the lead from each site (PIs). The lead therapists at committee meetings were specifically selected based on their engagement in rehabilitation research. The core coalition was composed of groups of individuals with content knowledge related to therapy documentation, therapy operations and regulations, informatics and EMR functionality, and persons with lived experience. Figure 1 depicts the organizational structure and communication loops implemented to guide the data discovery process. The site principal investigators were tasked with identifying individuals to represent their site within work groups.

FIGURE 1.

Organizational structure and lines of communication for work groups and grant governance. The triangle figure represents the four groups and the lines of communication between the four work groups with the various governance groups. Dotted lines depict the available lines of communication between any of the work groups and the governance groups.

The Therapy Representative Work Group (TRWG) was the first group assembled. Selection of representatives included the following qualifications: (1) frontline therapy provider, preferably with specialized experience in inpatient TBI rehabilitation; (2) knowledge of daily documentation requirements for inpatient setting; (3) ability to communicate effectively as a liaison with teams and workgroups; and (4) ability to mentor and coach during change initiatives. Each of the 16 sites selected a TRWG representative, resulting in a group comprised 4 OTs, 10 PTs, 3 SLPs, and 1 from therapeutic recreation. A few sites selected more than one representative as backup for the project.

Regardless of professional discipline, each TRWG member was expected to represent all disciplines from their site. Of all qualifications, communication was considered the most critical skill for representatives because each was expected to solicit input and provide feedback to all disciplines at their respective site. Their first task was to create a regular and predictable format to solicit input and communicate project progress. Communication format was determined by representative preference, but regularity was considered key to ensuring early buy‐in for any potential changes that would be required in their documentation system. The second task for the TRWG was to ensure familiarity with daily documentation of OT, PT, and SLP and the workflows of each discipline at their site. Familiarity with all disciplines would engender comfort and trust from on‐site teams for fair representation. Effective cross‐disciplinary communication within a site was demonstrated by a representative from the discipline of PT who conducted internal polling of OTs via email to solicit input on preferred components of various OT activities. For example, when the TRWG needed input on terminology to use to describe the components of bathing activities, the PT emailed the OTs at their site and requested their input. The PT collected their responses and noted their specific preference for inclusion of action terms along with body components (e.g., wash/rinse/dry[body part]) rather than a component listing of only body parts (e.g., face, abdomen, arms, etc.). The OTs further recommended consolidation of eight individual responses describing positioning for bathing, preferring two response options with free text. The feedback from this site's OTs were sent to the lead PI who then consolidated this site's OT feedback along with feedback from OTs from other sites.

The TRWG met weekly, with 12‐month goals to: (1) identify and select the relevant discrete data elements associated with daily treatment in each of OT, PT, and SLP; (2) provide operational definitions and standardized response sets for data elements; (3) propose standardized documentation design; (4) collaborate with rehabilitation leader representatives to ensure data complied with regulatory standards; (5) collaborate with rehabilitation leader and site‐specific teams to ensure design proposals fit with existing site workflows; (6) perform crosswalks between data elements identified with site‐specific existing documentation and data capture systems; and (7) identify site‐specific documentation change needs and submit requests to respective informatics teams. Each site was required to ensure the individual data elements contained in the final set of elements was either present in their existing documentation system or was identified as needing to be altered or built into their system.

The Rehabilitation Leader Representatives Group (RLRG) assembled clinical leaders from across the 16 sites. Qualifications for identifying site representatives included: (1) leadership position within the inpatient rehabilitation environment (e.g., senior clinical therapist, clinical manager); (2) working knowledge of regulatory guidelines and compliance requirements; and (3) ability to communicate effectively as a liaison, particularly with hospital administrations, informaticists, and information technology groups. Group selection resulted in representatives from OT, PT, and SLP, as well as social work/case management and psychology. The RLRG met monthly with 12‐month goals to: (1) ensure availability of TRWG members to attend project meetings; (2) provide opportunities for TRWG members to communicate with therapy staff at regular intervals; (3) assist TRWG members in resolving regulatory and compliance conflicts with proposed data capture procedures; and (4) assist TRWG members in communicating site‐specific documentation change management with informaticists and information technology representatives.

Within the RLRG, three of the individual leaders were selected to represent each of OT, PT, and SLP as project leaders for their discipline of practice. These three discipline leaders were identified to provide the theoretical background to classify data elements into higher level therapeutic approaches based on their respective discipline. Given each discipline is independently licensed or certified and independently governed by professional practice standards and guidelines, discrepancies and disagreements were anticipated due to differences in standards of practice and terminology. However, the communication loops created between TRWG and RLRG, and between the representative groups and each of their own site‐specific rehabilitation teams resulted in no need for resolution of disagreements between therapy disciplines.

The third critical group assembled was comprised of representatives from the fields of informatics and information technology. The Informatics and Information Technology (IIT) representatives were assembled as an advisory group to help guide data capture design through clinical documentation. A challenge within CARE4TBI was inclusion of sites with different EMR platforms that also included different vendor licensing agreements and site‐specific community documentation restrictions. The IIT Group facilitated the navigation and resolution of these challenges. The group also provided oversight and guidance for cross‐site data standardization and data integrity.

The Ohio Valley Advisory Council is our consumer advocacy group comprised of persons with lived experienced with TBI, persons with lived experience as a caregiver, advocates, and clinicians. The Council provides consumer input on research priorities and emerging issues in TBI for the Ohio Regional TBI Model System. The Council also reviews progress of active research projects and provides feedback and consumers' perspectives on relevance and meaningfulness. For CARE4TBI, selected Council representatives met quarterly to provide input on data elements. The group contributed to discussions on defining treatment dosing (e.g., how to capture “amount” of treatment), factors influencing treatment sessions (e.g., pain, mental health, language barriers), patient engagement, and minimally important differences to change in function.

Leadership of CARE4TBI was provided by a multiple‐PI model with support from investigators with specific areas of expertise in informatics and rehabilitation leadership (CB), biostatistics, observational study design and advanced causal inference methods (EH), and TBI Model Systems site and long‐term outcomes following TBI (JB). A leader in the field of health information technology assisted with leading the IIT Advisory Board (TH), and an expert in TBI stakeholder engagement (JC) facilitated the involvement of persons with lived experience with TBI. Governance of the entire project was managed through the Steering Committee while consumer input and community stakeholder feedback were managed through the Ohio Valley Advisory Council. Figure 1 depicts the lines and loops of communication and reporting structure.

4.2. Capturing meaningful practice as data

Initial meetings with the TRWG and RLRG consisted of open‐ended questions designed to brainstorm experiences in treating pwTBI in acute inpatient rehabilitation. The TRWG and RLRG brainstorming sessions were completed independent of each other to reduce possible social desirability bias and any bias due to power differentials between therapists and their supervisors. Separation provided a greater opportunity for freedom of expression on topics or issues potentially negatively impacting what or how therapy is conducted. The methods employed during this stage followed the initial open‐ended iterative steps of the Delphi process. ⁴⁰ Five questions were designed to solicit as much information as possible regarding factors that play a role in conducting therapy. These questions were:

1. What is important to know about the therapy session?

2. What is important to know about the specific session on any given day?

3. What happens before or during therapy that affects what you decide to do with patients?

4. What happens during the session that affects what you are doing with patients?

5. What happens during the session that affects what you plan to do in the next session?

To aid the brainstorming exercises, Figure 2 was developed to visually depict the intent of CARE4TBI (see Appendix  A in SDC for further details). Therapy was depicted as a complex interchange between therapist and patient, as well as interactions between interventions and activities designed to engage patients to promote improvement in performance. The therapy process is bidirectional in nature, with influence from both therapist and patient including impact of individual‐specific impairment(s) on task selection by therapist, patient engagement with the task, and response to treatment. This bidirectional relationship was an extended focus of questions #3 and #4, resulting in multiple discussions surrounding factors that could influence the session (e.g., cognition, communication, pain, etc.), activity components with varying challenges to a patient (e.g., internal versus external perturbations), level of assistance, level of performance, and context of the task. Several rounds of brainstorming were completed with the therapy and rehabilitation leader groups. The Delphi method was not utilized beyond the initial brainstorming. A primary reason for not continuing to use the Delphi method was the tight timeline of the project. All data elements, response sets, operational definitions, formatting, and design preferences for all three therapy disciplines needed to be completed within 12 months. A second reason was the researchers' judgment based on past experience that the open‐ended and unstructured nature would be more conducive to fostering discussions on complex therapy issues with full‐time clinical therapists who might otherwise feel constrained by a more formal or structured approach to data discussions. This resulted in performing all subsequent brainstorming iterations during virtual weekly and monthly group meetings. The methodology used to move TRWG members from broad‐focused brainstorming to meaningful discrete data element identification incorporated several rounds of iteration, feedback, summarization, and discussion for consensus building. ⁴⁰ , ⁴¹ In addition, down‐selection was guided by a set of a priori criteria that included prioritization on data necessary to fulfill regulatory requirements (e.g., time in session), consolidation of redundant terminology (e.g., walking versus ambulation), inconsistent rating systems (e.g., percent of effort versus descriptive rating of effort), and specifics aims of the CARE4TBI research project (e.g., factors influencing session). See Appendix  B in SDC for details.

FIGURE 2.

Visual representation of the dynamics during rehabilitation therapy sessions. The bidirectional arrows in the bottom box depict the fluidity of the therapist–patient interaction: Any focus within a given therapy session is influenced by patient status and their performance with a given task, the assessment skill and training of the therapist, and the therapist's selection of a specific task and the skill to apply.

5. RESULTS

The brainstorming exercises generated volumes of free‐form data, initially creating 98 distinct categories of data, each containing a range of data elements. For example, “time” was a distinct category of data deemed relevant to capture due to its regulatory requirement, but at least four discrete data elements were listed for time (total time, time of day, start time, end time). Over 850 discrete data elements were identified. The free‐form data were then tentatively categorized into three large categories based on relevance of identified discrete data elements that would fall within the categories depicted in Figure 2 by one of the PIs, with input from the three therapy leads. The three categories included characteristics of the session, characteristics of the patient, and characteristics of the specific therapy. This categorization scheme was then sent out to the TRWG with instructions to share with therapists at their site, solicit feedback, and send in feedback to the PI. The PI then collated feedback from all sites and reviewed results at the weekly meetings with the TRWG and the monthly meetings with the RLRG. New versions were sent back out to the TRWG to repeat the process of obtaining feedback from their sites. Through several reviews and iterations, the three categories were reduced to two categories of prospective data—session level and therapy activity level. For the few discrepancies that did emerge, discussions during meetings and allowing flexibility in documentation style resolved all issues. A total of 54 different therapy activities were identified. Within each therapy activity, discrete data elements were sub‐categorized into activity components (what was done), method of delivery (how it was done), and equipment or supplies (what was used). Duplication of discrete data was eliminated, synonymous terms were combined, fragmentation or reductionistic elements were consolidated, and operational definitions were constructed. The final tally of distinct data elements identified as meaningful to capture was approximately 561 across the 54 activities. A minimum of 12 discrete fields were also identified for any given session, and an average of 30 discrete fields were identified for most sessions. See Appendix  C in SDC for details.

Table 2 provides outlines of the discrete data elements. Session level data consist of data relevant to both the session and the patient. These data were identified as needing to be collected across all sessions with all patients. The therapy activity data, however, was designed to capture the unique features of each therapy session for each patient. Five session level data elements were considered “must‐have” for all sessions, and 5 therapy activity elements were considered “must‐have” for each activity. The entire data discovery process was completed in 1 year, achieving goals established for the project.

TABLE 2.

Sample listing of data elements and examples of select discrete data fields per data element.

Session level data elements	Therapy activity data elements
Total time Therapy discipline Others in attendance Environmental stimulation Location Patient effort and participation Purpose/target Skilled intervention(s) Factors influencing session Pain Vitals	Category (8)	Activity (54)	Component field(s)	Method of delivery field(s)	Adaptive equipment field(s)
	ADL	Toileting	Clothing management Perineal hygiene	Level of assistance Cueing/prompting	Ambulatory device Transfer device
	Mobility	Walking	Distance duration trials	Surface/terrain Level of assistance Cueing/prompting Compensatory strategy Situation	Orthosis/prosthesis/brace Ambulatory device Transfer device
	Cognition and language	Memory	Task employed Level of task difficulty	Level of assistance Cueing/prompting Compensatory strategy Situation	ADL items Smart devices Manual devices
	IADL	Home management	[multiple task selections]	Position Level of assistance Cueing/prompting Compensatory strategy Situation	Ambulatory device Transfer device

Abbreviations: ADL, activities of daily living (e.g., Bathing, Dressing, Eating); IADL, instrumental activities of daily living (e.g., Home Management, Medication Management, Driving Readiness).

6. DISCUSSION

Several lessons learned throughout this process can provide guidance for others seeking to create a system for capturing large quantities of high‐quality data across multiple clinical disciplines. First, the open‐ended questions used in the initial brainstorming sessions generally resulted in more questions from TRWG members than data discovery. To assist the process, therapists were asked to describe a therapy session from earlier in the day. These real life examples lead to other therapists asking for explanations about therapist–patient interactions (e.g., “How did you know the patient was disinterested?”), common complications faced during therapy delivery (e.g., “Did you note any triggers for the behavior?”), and dissection of narrative notes (e.g., “What does ‘some improvement’ mean?”). Storytelling provided familiarity from which therapists could draw upon to discuss relevant factors within the session and therefore identify data elements to capture. Second, identifying the data elements that were consistent within disciplines and across sites early in the discovery process allowed work groups to focus on data elements that were not consistent and to identify what would be needed to make them consistent. Third, clinical data are not always amenable to reduction into standardized or discrete fields because it is difficult to conceive of all possible responses or situations that may occur when human performance is the target of data capture. To that end, ensuring data fields are created that allow for uniqueness or outliers is necessary and usually requires that open, free‐text narrative fields remain within the documentation system to capture the possible uniqueness or outlier performances.

The clinical documentation system established by the CARE4TBI project shares some similarities with other rehabilitation data systems, though there are some important differences. For example, the data capture system developed for the TBI‐PBE project ³¹ was a manual data capture system from which we drew some of the data elements for the CARE4TBI system. However, during TBI‐PBE, therapists documented their daily therapy sessions in their documentation system and then filled in a manual data collection form. This resulted in their documenting daily activities twice, once for clinical purposes and a second time for research purposes. This form of “double documentation” adds additional work burden onto clinicians, can increase errors in data capture due to documentation fatigue, and adds significant human labor cost to research projects due to the need to monitor data capture and to manually enter data into databases. The documentation system developed by CARE4TBI completely eliminates the double documentation burden on therapists, eliminates the need to monitor data capture, and eliminates the need to manually enter data into a database. The advantage of the CARE4TBI electronic documentation system is that it satisfies the dual purpose of capturing treatment data for clinical and research purposes without adding work burden to therapists.

The CARE4TBI electronic documentation system is also different from the Rehabilitation Treatment Specification System (RTSS). ⁴² The RTSS is based on a theoretical approach that attempts to describe active ingredients while CARE4TBI documents observable processes used in therapy. The high degree of specificity for use of the RTSS also necessitates some amount of training to implement the system and thus changes how rehabilitation care is conceptualized and then delivered. CARE4TBI is designed to study the delivery of rehabilitation care in its current natural format, which requires no concerted effort for therapists to change their practice. The results of CARE4TBI may help to guide future RTSS efforts by identifying which treatments are associated with better outcomes, thus narrowing down the field of treatments to target with RTSS methods to decipher the active ingredients.

Despite the complexity of rehabilitation as a process approach to treatment and the varied dynamic human environment in which it takes place, the success of Year 1 of CARE4TBI exemplifies the feasibility of taking 15 different rehabilitation sites and multidisciplinary professional groups to gain consensus on what is relevant and meaningful to document clinically, providing the opportunity to capture data needed to naturalistically investigate rehabilitation. The unique organizational structure used to govern discovery was successful in achieving this goal within the expected time frame. Furthermore, the resulting data system is informed by experts from across multiple disciplines who engage in frontline treatment, by decision‐makers involved in day‐to‐day operations and quality performance, and by consumers of the treatment delivered during inpatient rehabilitation. Key data elements are targeted to be embedded into the clinical documentation environment. Embedding the data system into standard clinical workflow will eliminate the need for data collection forms and manual abstraction. The clinical data system will source data directly from point of care, thereby improving validity of the data, efficiency of capture, and reducing errors associated with manual abstraction. The latter features are key characteristics for sustainability of these efforts and expansion to additional health systems. Once incorporated into EMRs, data will be captured through standard documentation and extraction processes without added burden to frontline therapists or increased nonproductive time. This achievement provides a foundation for a future LHS coalition in acute rehabilitation.

7. LIMITATIONS

While our discovery milestone was achieved, it is not known if additional iterations would have improved the precision of the data elements. The limited time and constrained funding resources required a speedy timeline for discovery. The project needed to complete all aspects of data discovery, as well as identification, consolidation, standardization, and proposed formatting within the first year. Relevant data discovery may have been missed or inadequately explored due to these constraints. Also, and as mentioned above, not all clinical data are amenable to being captured in discrete data fields and necessitates continued need for narrative text fields. With advances in computer science applications (e.g., text mining) and in qualitative research (e.g., thematic analysis), narrative text field can eventually become a routine part of the data capture and analyses.

CONFLICT OF INTEREST STATEMENT

The authors have no conflicts of interest to declare.

Supporting information

Data S1. Supporting information.

Beaulieu CL, Bogner J, Swank C, et al. Setting the foundation for a national collaborative learning health system in acute TBI rehabilitation: CARE4TBI Year 1 experience. Learn Health Sys. 2025;9(2):e10454. doi: 10.1002/lrh2.10454