Generating consistent longitudinal real‐world data to support research: lessons from physical therapists

Researchers using real‐world data (RWD) hope to generate answers to clinical effectiveness research (CER) and patient‐centered outcomes research (PCOR) questions. However, reliability and validity of these results are dependent on data completeness and consistency. Because RWD are not generated with research as the primary goal, they suffer from incomplete and inconsistent documentation of routine clinical interventions. The two most common sources of RWD are clinician‐documented and health system use data stored in electronic health records (EHRs) and administrative data, respectively. Both sources of RWD are readily available within health systems or aggregated in regional databases, such as PCORNet or administrative claims data. EHR data quality, in particular, suffers from inconsistent data structure and documentation as well as fragmentation across time and settings. For example, prescription refills or physical therapy (PT) interventions are not systematically documented in the primary care physician's EHR. In rheumatology practices, performance on rheumatoid arthritis quality measures using the American College of Rheumatology's Rheumatology Informatics System for Effectiveness registry varies according to the specific EHR employed (1). In an era of chronic disease, the richness of existing data and the value to research driven by these data will be enhanced when systematic and comprehensive clinical documentation of interventions is included in the EHR across settings.

The EHR is the primary data source for real‐world CER and PCOR applications but can be a source of bias when clinical documentation is inconsistent, incomplete, and potentially biased (2). Missing EHR data result from clinician inconsistencies in what and when to document or when patients receive care across multiple health systems or from community‐based providers. Human decisions determine content and definition of the data elements (or not) in the EHR, hence contributing to incomplete intervention and outcome data (3). Thus, research using today's EHR, and its clinical data, risks validity because of two major factors: 1) inconsistent clinical intervention and outcome documentation in the course of care and 2) lack of integration of clinical documentation across time and place.

Following total knee (TKR) and hip replacement surgeries, PT providers are commonly not affiliated with the health system where the surgery was performed. Thus, their documentation does not reside in the patient's surgical EHR. Further, although PT office EHRs capture visit time and length, few PT EHRs capture the full content of the PT interventions (ie, specific PT components); their intensity, frequency, and progression; or the “dose” of PT. Thus, CER using real‐world evidence is stymied by the lack of complete, consistent PT data to explore best practices in PT care. This is particularly problematic because TKR is one of the most common and costly procedures in the United States today, and wide variation in PT practice after TKR is well documented (4, 5). More recently, the COVID‐19 epidemic introduced new peri‐TKR practice patterns that EHR notes are not prepared to evaluate. The incomplete data in today's RWD cannot generate best practice for content and dosage of PT interventions and changes in care patterns post TKR (6, 7).

Can the quality of RWD be improved to serve research and, ultimately, best practice? As proof of concept that clinicians can generate consistent and standardized clinical data to enhance data quality in the course of routine patient care, we collaborated with PT clinicians and experts to generate a web‐based comprehensive system to quantify the total dose of PT interventions with type of modality, quantity, intensity, and progressions over time (8). The system was designed to be implemented in outpatient PT clinics and capable of residing alongside or within a clinic's EHR. Our goal was to collect routine clinical data in a format useable by the general population of outpatient physical therapists treating patients post TKR and in a structure that would allow easy quantification and analysis across patients, therapists, and sites. Uniform and efficient documentation of real‐world PT practice following TKR is essential for the necessary comparative effectiveness research demanded by the unexplained practice variation that currently exists.

Using the list of interventions identified from retrospective chart reviews, we asked a small sample of clinicians to identify all interventions they used while treating patients post TKR and to add any unlisted interventions to the original list. This revised list was then sent to national experts in TKR rehabilitation for review and revision. Their revised list was sent finally to international experts for additional review and revision. This iterative process lasted approximately 1 year and resulted in a comprehensive menu of interventions. A web‐based Health Insurance Portability and Accountability Act–compliant data capture system was constructed to allow physical therapists to select their interventions from an all‐inclusive menu to minimize the use of open‐text contributions.

The PT capture system consisted of 143 interventions divided into eight treatment categories: strengthening, flexibility, aerobic exercises, balance, task‐specific activities, manual therapy, modalities, and patient education. Therapists documented their interventions using dropdown menus. Once a specific intervention was identified, additional dropdown menus appeared to describe dosage and intensity. More than 100 community‐based physical therapists were trained to use this PT web‐based capture system and its taxonomy when documenting PT care.

Over a period of 2 years, 83 physical therapists and PT assistants from eight practices located in three US states entered data for 161 patients post TKR with 2615 patient visits. No technical problems with the data capture system were reported, and physical therapists noted that data entry was quick and easy, typically taking less than 2 minutes. In 84% of the visits, all interventions were captured by the new taxonomy. The remaining 16% of visits captured interventions through text descriptions; 14% of these notes reported interventions included in the taxonomy, with 2% of visits including interventions not in the menus. These results demonstrate that routine PT interventions can be captured thoroughly in an efficient, systematic, and consistent manner across real‐world therapists and sites.

In addition to demonstrating the clinical feasibility of this data capture system, the RWD captured by our system during routine post‐TKR care confirm the wide variation in treatment content, dosage, and duration. We are analyzing these data to identify associations between practice factors and 6‐month functional outcomes. Future implementation of this PT intervention capture system has the potential to accelerate CER on PT interventions to determine best practice PT post TKR.

Recently, this PT intervention taxonomy was applied to an Agency for Healthcare Research and Quality–funded systematic review of pre‐ and post–total knee and hip rehabilitation practices (7). Application of this detailed taxonomy demonstrates that, as in clinical practice, there is the significant heterogeneity of rehabilitation interventions reported in the literature (Table 1). Specifically, application of the taxonomy to 83 studies demonstrated a lack of uniformity across interventions, with no consistent application of specific types of exercise, while, more importantly, providing little or no information about the dosage and intensity of the interventions. Without identifying and quantifying the components of the intervention, assessment of the comparative effectiveness of PT treatments is precluded.

Table 1.

Sample use of standardized documentation categories for literature review of postacute rehabilitation PT flexibility exercise interventions in total knee arthroplasty

Study	Arm	Flexibility	Ankle pumps	Bike (ROM)	Calf stretch	Hamstring stretch	Heel slides	Hip extensors stretch	Hip flexor stretch	Knee extension	Knee flexion	Standing TKE
Artz, 2017 (9)	Group‐based exercise
	Usual care
Bade, 2017 (10)	HI program rehab	1	1	1	1	1	1			1	1
	LI rehab	1	1	1	1	1	1			1	1
Bruun‐Olsen, 2013 (11)	Walking skill program	1			1	1					1
	Usual physio	1
Cai, 2018 (12)	CBT for kinesiophobia
	Standard care
Fransen, 2017 (13)	O group exercise	1			1	1				1	1
	Usual care
Heikkilä, 2017 (14)	Exercise	1		1	1	1			1
	Control
Kauppila, 2010 (15)	Multidisciplinary rehab	1
	Control	1
Lenguerrand, 2020 (16)	Group‐based O‐PT	1		1		1				1	1
	Usual care
Li, 2015 (17)	Education for daily PA
	Noneducation for daily PA
Liao, 2015 (18)	Functional rehab and BT	1	1			1				1	1
	Functional rehab	1	1			1				1	1
Liao, 2020 (19)	Elastic resistance exercise	1
	Standard care	1
Madsen, 2013 (20)	Group‐based rehab	1		1
	Individual‐based rehab
Minns Lowe, 2012 (21)	Home‐visit physio	1			1	1					1
	Usual physio
Monticone, 2013 (22)	Experimental	1		1
	Control	1		1
Moutzouri, 2018 (23)	FSET	1					1			1	1
	Functional exercise training	1	1				1			1	1
Piva, 2017 (24)	CBI	1		1	1	1				1	1
	Standard care	1		1	1	1				1	1
Schache, 2019 (25)	Standard rehab and HAT	1			1			1		1	1
	Standard rehab and general function exercise	1			1			1		1	1
Vuorenmaa, 2014 (26)	Intervention	1		1	1	1			1		1
	Control

Note: This table represents the first half of the studies assessing postacute rehabilitation interventions (part 1). All studies in this table assessed novel interventions hypothesized to improve effects compared with controls. 1 = presence of component.

Abbreviations: BT, balance training; CBI, comprehensive behavioral intervention; FSET, focal sensorimotor exercise training; HAT, hip abduction training; HI, high intensity; LI, low intensity; O, outpatient; O‐PT, outpatient physical therapy; PA, physical activity; physio, physiotherapy; PT, physical therapy; rehab, rehabilitation; ROM, range of motion; TKE, terminal knee extension.

This model highlights many important lessons for future use of RWD to support CER and PCOR. First, clinicians can collaborate to define a consistent, comprehensive library of interventions that capture uniform documentation of detailed components of clinical interventions across settings and over time. Second, clinicians in busy practices can complete these structured forms efficiently to improve the quality of future RWD. Last, researchers can use these data to answer important questions, to define best practices, and to monitor practice over time. In the future, EHRs can use application programming interfaces to merge data captured across EHRs to provide a single complete record of care. We believe that specialties throughout health care can use this model of collaboration to develop clinically relevant, efficient, and complete documentation systems that yield valid and consistent RWD for use in CER, PCOR, and quality improvement initiatives.

AUTHOR CONTRIBUTIONS

All authors were involved in drafting the article or revising it critically for important intellectual content, and all authors approved the final version to be published. Drs. Oatis and Franklin had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study conception and design

Oatis, Konnyu, Franklin.

Acquisition of data

Oatis, Franklin.

Analysis and interpretation of data

Oatis, Konnyu, Franklin.

Supporting information

Disclosure Form