Patient Effort in Traumatic Brain Injury Inpatient Rehabilitation: Course and Associations With Age, Brain Injury Severity, and Time Postinjury

Abstract

Objective

To describe patients' level of effort in occupational, physical, and speech therapy sessions during traumatic brain injury (TBI) inpatient rehabilitation and to evaluate how age, injury severity, cognitive impairment, and time are associated with effort.

Design

Prospective, multicenter, longitudinal cohort study.

Setting

Acute TBI rehabilitation programs.

Participants

Patients (N=1946) receiving 138,555 therapy sessions.

Interventions

Not applicable.

Main Outcome Measures

Effort in rehabilitation sessions rated on the Rehabilitation Intensity of Therapy Scale, FIM, Comprehensive Severity Index brain injury severity score, posttraumatic amnesia (PTA), and Agitated Behavior Scale (ABS).

Results

The Rehabilitation Intensity of Therapy Scale effort ratings in individual therapy sessions closely conformed to a normative distribution for all 3 disciplines. Mean Rehabilitation Intensity of Therapy Scale ratings for patients' therapy sessions were higher in the discharge week than in the admission week (P<.001). For patients who completed 2, 3, or 4 weeks of rehabilitation, differences in effort ratings (P<.001) were observed between 5 subgroups stratified by admission FIM cognitive scores and over time. In linear mixed-effects modeling, age and Comprehensive Severity Index brain injury severity score at admission, days from injury to rehabilitation admission, days from admission, and daily ratings of PTA and ABS score were predictors of level of effort (P<.0001).

Conclusions

Patients' level of effort can be observed and reliably rated in the TBI inpatient rehabilitation setting using the Rehabilitation Intensity of Therapy Scale. Patients who sustain TBI show varying levels of effort in rehabilitation therapy sessions, with effort tending to increase over the stay. PTA and agitated behavior are primary risk factors that substantially reduce patient effort in therapies.

Interdisciplinary traumatic brain injury (TBI) inpatient rehabilitation (IR) is a complex and multifaceted process. The individual and interactive contributions that patient, environment, and treatment factors make to outcomes remains understudied and not well understood.^1,2 Rehabilitation, as all health care fields, faces increasing pressure to deliver high-quality services that maximize outcomes and minimize length of stay (LOS). A key element in the quality service equation is providing patient-centered care that takes into account injury severity and patient strengths; the match of treatments to patients' deficits and desired outcomes; and, importantly, the level of patients' involvement in care. Rehabilitation clinicians often consider the level of patients' behavioral involvement in their therapies when evaluating treatment effectiveness and outcomes.³ Motivation for treatment is assumed to be a primary determinant of rehabilitation outcomes,⁴ but it is difficult to assess reliably because observers must infer patients' thoughts and intentions.⁵ Few empirical studies have examined patients' level of involvement in either TBI IR specifically or rehabilitation more broadly, and all have differed in the populations sampled, inclusion of persons with cognitive impairment, and nature and frequency of assessment.^6–11

Several constructs with overlapping conceptual definitions that focus on observable behavior have been used to describe patients' involvement in IR, including engagement, participation, and effort. Lequerica et al⁶ used the 15-item Rehabilitation Therapy Engagement Scale weekly during brain injury IR to assess a broad range of patient behaviors, including participation, cooperation, effort, persistence, frustration tolerance, responsiveness, and self-confidence. This sample, consisting primarily of patients with TBI, had engagement total scores that reflected slightly more maladaptive than adaptive behavior in their occupational therapy (OT) and physical therapy (PT).⁶ Pegg et al⁸ used the single-item, 7-point, Rehabilitation Intensity of Therapy Scale¹² to rate patients' level of effort in TBI IR. In an information provision clinical trial with patients who predominately sustained severe TBI and were at a Rancho level ≥6, patients' mean weekly ratings in speech therapy (ST) and PT generally indicated good level of effort.⁸

In a mixed diagnosis, non-TBI study, Kortte et al⁹ used the 5-item Hopkins Rehabilitation Engagement Rating Scale to assess attendance, attitude, participation, and need for prompting in patients receiving IR. Patients with spinal cord injury, hip/knee replacement, stroke, and amputation who had no more than mild cognitive impairments had their engagement for the entire rehabilitation stay rated once at discharge. Scores indicated they were nearly always engaged in their OT and PT sessions.⁹ Lenze et al¹⁰ developed the single-item, 6-point Pittsburgh Rehabilitation Participation Scale to assess patients' session completion, participation, effort, and interest in their IR. In this older adult sample with primarily orthopedic issues, debility, or stroke, patients were rated on average during 8.6 PT or OT sessions over a mean 12.6-day stay and had generally good to very good levels of participation.¹⁰ A modification of the Pittsburgh Rehabilitation Participation Scale was used in a spinal cord injury multicenter study to characterize patient effort in >260,000 IR sessions.¹¹ PT, OT, ST, and therapeutic recreation clinicians gave mostly excellent/engaged and very good/good/active effort ratings and very few fair/passive and poor ratings.¹¹

With regard to factors associated with quality of involvement in patients with TBI in IR, Lequerica et al⁷ found that Agitated Behavior Scale (ABS) scores, assessed once during the admission week, and duration of posttraumatic amnesia (PTA) explained significant variance in engagement ratings: more agitation and longer PTA were associated with less engagement. In mixed rehabilitation samples, patients' positive affect, denial of illness, negative affect, and depression ratings were correlated with engagement ratings,⁹ and their participation ratings significantly improved from IR admission to discharge.¹⁰

We found no multicenter or large sample studies that described the involvement of patients with TBI in their IR therapies. No study described patients' involvement in IR therapies on a daily basis or used longitudinal, multivariable modeling to evaluate prognostic indicators and their associations with level of patient involvement. The current investigation aimed to extend the research literature by using a prospective, 9-center, longitudinal cohort design to examine patients' daily level of effort during TBI IR OT, PT, and ST sessions. The 3 primary objectives were to (1) evaluate the psychometric properties of the Rehabilitation Intensity of Therapy Scale level of effort rating scale, including distribution of ratings; therapists' rating accuracy; test-retest and interrater stability; and concurrent validity with same session ratings of patients' inattention, low arousal, lack of initiation, and disinterest; (2) describe and assess the course of patients' level of effort in PT, OT, and ST sessions during their rehabilitation stay; and (3) model 6 patient factors (age, time from injury to IR admission, days of IR treatment, brain injury severity, agitation, PTA) to evaluate their effects on daily level of effort ratings.

Methods

Study population

As part of a prospective observational investigation, we studied a cohort aged ≥14 years with complicated mild, moderate, and severe TBI from 9 IR centers located throughout the United States.² Participants were enrolled from October 2008 to August 2011 and received interdisciplinary rehabilitation services that typically included physiatry, nursing, psychology/neuropsychology, case management, PT, OT, ST, and therapeutic recreation. Details of inclusion criteria, procedures, and enrollment rates have been published elsewhere.² Participants were excluded from the present analysis if they had a disorder of consciousness (defined as Rancho Level of Cognitive Functioning Scale score <4 or FIM cognitive subscale score ≤6) for most or all of their stay.¹³ Participants were followed daily from IR admission through discharge. The institutional review board of each site approved center participation; patients who sustained TBI or their authorized proxy provided informed consent.

Measures

The Rehabilitation Intensity of Therapy Scale, a single-item, 7-point, behaviorally anchored rating scale, was used to rate patients' level of effort during each IR PT, OT, and ST individual therapy session in which the patient was given a Rancho score ≥4. Effort is an observable behavioral construct, which is defined as “the use of physical or mental energy to do something”¹ and is not a synonym for motivation or malingering.¹⁴ Use of physical and mental energy within a rehabilitation therapy context is operationally defined as being attentive and engaged in goal-directed activity, including initiating activity, incorporating therapist feedback, and persevering when therapies become challenging (see appendix 1 for descriptions of all 7 anchored rating categories).¹² The scale has shown evidence of same day, between-observer consistency of ST and PT session ratings with intraclass correlation coefficients (ICCs) of .74.⁸ Therapists were trained in making Rehabilitation Intensity of Therapy Scale level of effort ratings and were tested for accuracy twice during the study using written vignettes. The clinicians had access to a complete description of the 7 level of effort categories when performing their session ratings. In addition, for each session therapists recorded whether any of 9 patient impairment or attitude factors (agitation, inattention, low arousal, pain, disinterest, lack of insight, medical complications, emotional problems, lack of initiation) interfered with carrying out the session.

Presence of PTA was based on psychologists' ratings on 1 of 2 equivalent standardized assessments: Orientation Log score >24^15,16 or Galveston Orientation and Amnesia Test score >75.^17,18 The ABS is a standardized, well-validated, 14-item ordinal measure used to serially assess the presence, severity, and change in agitation for patients who have sustained TBI.^19–21 Starting early in the admission week, nurses rated patients on the ABS daily during every shift. A bout of agitation was defined as having ≥6 four-hour shifts with an ABS score >21 out of 12 consecutive 4-hour shifts. Nurses discontinued ratings if the patient's total score was ≤21 for 3 consecutive days. If agitated behavior recurred, nurses resumed every shift ABS ratings, and the new bout was tracked.

The FIM,^22–24 a standardized, well-validated, and reliable measure of functional independence, was rated by the interdisciplinary team with items receiving input primarily from each patient's physical therapist, occupational therapist, speech therapist, and nurse. Preliminary analyses of the data revealed heterogeneity of cases in terms of almost all clinical variables. A number of stratification schemes were evaluated including case-mix groups as defined for IR patients with TBI. The FIM cognitive subscale score on IR admission, separated into 5 relatively homogenous subgroups (patients with scores of 5–6, 7–10, 11–15, 16–20, and ≥21), best differentiated outcomes (a complete description is available in the Horn et al² article of this supplement).

Trained data abstractors at each site, independent of the interdisciplinary treatment team, collected data from the IR medical record on age, sex, race, ethnicity, cause of injury, date of injury, dates of rehabilitation admission and discharge, and FIM motor and cognitive subscale admission and discharge scores. Abstractors also completed items for the Comprehensive Severity Index.² Brain injury severity factors were separated from other injuries, complications, and comorbidities to derive Comprehensive Severity Index brain injury severity scores. The Comprehensive Severity Index brain injury severity scores at admission were used in this analysis and included neurologic, radiologic, and vital signs items that were weighted and yielded a total score ranging from 0 to 162, with higher scores indicating greater severity.²

Efforts to minimize risk of bias

Uniform training, scale administration, and data collection procedures were used across the 9 centers. Standardized, reliable, and well-validated measures (eg, FIM, ABS, Galveston Orientation and Amnesia Test, Orientation Log) were used. Data abstracters received 3 days of formal training on data collection procedures and were required to complete data abstraction on 6 consecutive patient medical records with ≥95% accuracy (as compared with a criterion abstraction independently performed by a senior data abstracter at the data coordinating site) before being approved to collect data for additional patients. Independence of observers who provided ratings on the Rehabilitation Intensity of Therapy Scale (PT, OT, and ST), Comprehensive Severity Index brain injury severity (data abstracters coding information recorded primarily by physicians or copying values from laboratory reports), PTA (neuropsychologists), and ABS (nurses) reduced the risk of overestimating associations because of the same rater scoring all tests. Comprehensive data quality control standards, including daily checks for data completion of therapists' level of effort ratings, were used to minimize missing data. Prespecification of a parsimonious set of predictors of patient effort, as previously indicated, reduced the risk of overfitting the model to the sample data.^25,26

Statistical analyses

All analyses were conducted using SAS version 9.2,^a and figures were created using R statistics version 2.12.1.^b Level of effort rating distribution for each of the 3 disciplines, including skewness and kurtosis, was calculated, with scores closer to zero indicating a normative distribution. Level of effort scale test-retest stability was evaluated within the same discipline (almost always the same clinician) and within the same shift (either am or pm) on consecutive days. Level of effort scale interrater stability was evaluated with different disciplines rating patients' level of effort in consecutive sessions within the same day and shift (either am or pm). Test-retest and interrater stability were calculated using 1-way, random effects ICCs.^27,28 Variation of level of effort associated with therapists' report of the presence or absence of inattention, low arousal, lack of initiation, and disinterest was evaluated using t tests. Effect size was calculated with Cohen d using the pooled SD. For between-group comparisons, balancing the risk of type I and II error, a P value <.05 was deemed significant.

Summary statistics and graphs were used to examine patients' level of effort in PT, OT, and ST sessions during each week of IR. Patients (n=118) with ≤8-day IR LOS were included in the admission week only; that is, the same days were not used to calculate a mean discharge week score for these patients. Patients' number of weeks of treatment was calculated based on their LOS as follows: 1 to 8 days (1wk), 9 to 15 days (2wk), 16 to 22 days (3wk), and 23 to 29 days (4wk).

Linear mixed-effects regression modeling was used to examine the effects of age at injury, brain injury severity, functional cognition, and time on daily mean level of effort scores.²⁹ Data for age, injury date, IR admission date, admission Comprehensive Severity Index brain injury severity, admission FIM cognitive score, and presence of PTA were available for all patients. Missing ABS values within a bout of agitation (.74%) were statistically imputed using the observed ABS scores during the bout.³⁰ Per protocol, rating patients on the ABS was discontinued after resolution of an agitation bout; these unrecorded values were assigned a score of 14, indicating no agitation. There were approximately 3.3% of rehabilitation treatment days with unrecorded level of effort values that were treated as missing at random and imputation was not performed.

Statistical assumptions were tested. Rasch-transformed admission FIM cognitive scores were considered a predictor but were collinear with the Comprehensive Severity Index brain injury severity score, presence of PTA, and days from injury to IR admission and were not entered into the model. Days from injury to IR admission and number of days in IR were both found to be right skewed, and patients' scores were capped at 3SD above the mean to control for potential outlier effects. Age, Comprehensive Severity Index brain injury severity score, and days from injury to IR admission were centered at the sample mean to increase interpretability of the constant term and were entered as time invariant factors. Age was divided by 10 to allow unstandardized estimates to be interpreted by decade rather than year. Days from IR admission to treatment session date and presence of PTA were entered as time varying factors. ABS total score was recoded into 4 levels of ascending severity (no or subclinical agitation [≤21; used as the reference group], mild agitation [22–27], moderate agitation [28–34], severe agitation [≥35]) and was entered as a time varying factor.¹⁹ Intercept and days from IR admission were defined as random effects; all other predictors were fixed effects. Unstandardized estimates and P values reflect the significance and strength of the hypothesized predictors on level of effort ratings and therefore model performance. Balancing the risk of type I and type II error, a P value <.05 was deemed significant for predictors. The number of predicted values outside of the Rehabilitation Intensity of Therapy Scale level of effort 1 through 7 range was calculated. Given the very high ratio of cases to variables, internal validation techniques (eg, shrinkage) were not required.³¹

Sensitivity analyses³² were conducted on the initial model to (1) evaluate linearity and random effects assumptions; (2) examine whether potential interaction effects of the Comprehensive Severity Index brain injury severity with days from injury to IR admission and presence of PTA with ABS level were significant or changed model performance; and (3) rule out sex, race, and education effects in the Rehabilitation Intensity of Therapy Scale level of effort ratings. Local regression curves were used to evaluate linearity assumptions.³³ Akaike information criterion and Bayesian information criterion were used to assess whether changes in model complexity (eg, interactions evaluated in the sensitivity analyses) changed model fit.²⁵ When local regression curves revealed a nonlinear relation between days from IR admission and effort ratings, linear splines were created with the breakpoint/knot placed at 19 days. The initial model was rerun with the differing slopes for the 2 resulting segments of days from IR admission (1–19d, 20–90d) entered as separate predictors. The unstandardized estimates and performance of this revised model are reported in the results.

Results

Participants

A total of 1981 patients were enrolled in 9 U.S. centers participating in the practice-based evidence study. After excluding from analysis patients who had a disorder of consciousness for most or all of their stay (n=35), the final sample had 1946 participants. Patient demographics and injury, time from injury, and severity characteristics at admission are summarized in table 1. The sample was diverse with regard to age, sex, and race/ethnicity. Most patients were injured as a result of motor vehicle collisions (56%) or falls (31%), and the mean number of days from injury to IR admission was 26.1±31.2 days. Participants were admitted with high levels of brain injury severity and required assistance for most cognitive and motor functions. During their IR admission week, 64% were in PTA and 24% had clinical levels of agitation. Patients' mean IR LOS was 21.0±23.1 days.

Table 1.

Patient characteristics at IR admission (N = 1946)

Characteristic	Description
Age	44.1±21.4
Women	536 (27.5)
Race/ethnic group
White	1451 (74.6)
Black	309 (15.9)
Hispanic	128 (6.6)
Other	58 (3.0)
Cause of injury
Vehicular	1098 (56.4)
Fall	604 (31.0)
Violence	134 (6.9)
Miscellaneous	73 (3.8)
Days from injury to IR admission	26.1±31.2
Comprehensive Severity Index*
Brain injury severity score	46.0±23.1
Nonbrain injury severity score	17.9±15.0
PTA, present	1243 (63.9)
Agitation, present†	468 (24.1)
FIM
Motor score	32.4±17.0
Cognitive score	14.4±6.9

NOTE. Statistics reported are the sample size (%) or mean ± SD.

^*Patient characteristic data are measured or abstracted from the medical record based on the first 72 hours of IR admission.

^†Having at least 1 bout of agitation (≥6 four-hour shifts with an ABS score >21 out of 12 consecutive 4-hour shifts).

Psychometrics of the Rehabilitation Intensity of Therapy Scale level of effort ratings

The level of effort ratings in individual therapy sessions by OT (n=45,770), PT (n=50,383), and ST (n=42,402) all closely conformed to a normative distribution with minimal skewness (−.02 to −.11) and kurtosis (−.08 to −.12). High rates of accuracy (agreement with an expert-determined rating) were observed for ST (n=63 therapists, 98% correct), PT (n=99 therapists, 97% correct), and OT (n=109 therapists, 89% correct) at the June 2009 test. Rating accuracy remained high for ST (n=69 therapists, 91% correct), PT (n=118 therapists, 91% correct), and OT (n=115 therapists, 81% correct) at the March 2010 test. Test-retest stability for the single-item level of effort scale was excellent for all 3 disciplines during both morning and afternoon sessions, with ICCs ranging from .76 to .80 (table 2). Interrater stability for the level of effort scale was good for all 3 discipline pairings for both morning and evening therapy sessions (ICCs range, .53–.59) (see table 2). Presence of inattention, low arousal, lack of initiation, and disinterest in a therapy session were each found to be associated with lower same-session level of effort for all 3 disciplines (P<.001) (table 3). Effect sizes across disciplines were moderate to large for all 4 factors (low arousal [d=.97–1.07], inattention [d=.66–.70], lack of initiation [d=.63–.74], disinterest [d=.59–.84]), providing evidence of concurrent validity.

Table 2.

Stability of level of effort ratings (N = 1946)

Discipline	Shift	ICC	95% CI
Test-retest*
OT	AM	.79	.77–.81
PT	AM	.78	.76–.80
ST	AM	.79	.77–.81
OT	PM	.80	.78–.82
PT	PM	.76	.74–.78
ST	PM	.79	.77–.81
Interrater†
OT/ST	AM	.55	.52–.58
PT/OT	AM	.59	.56–.62
PT/ST	AM	.56	.53–.59
OT/ST	PM	.59	.56–.62
PT/OT	PM	.57	.54–.60
PT/ST	PM	.53	.49–.57

Abbreviation: ICC, intraclass correlation cefficient (1-way, random effects).

^*Same discipline rated patients' level of effort on consecutive days and within the same shift (either AM or PM).

^†Different disciplines rated patients' level of effort in consecutive sessions within the same day and shift (either AM or PM); the first-named discipline could be either the first or second of the 2 to treat the patient and rate level of effort.

Table 3.

Level of effort during a session by therapist report of presence of selected patient factors during same session

		Factor Present		Factor Not Present
		LOE		LOE
Factor	Discipline*	Mean±SD	95% CI	Mean±SD	95% CI	t Test †	Effect Size‡
Inattention	OT	3.21±1.18	3.18–3.24	4.08±1.26	4.07–4.10	52.91	.70
	PT	3.35±1.14	3.33–3.37	4.14±1.21	4.12–4.15	65.30	.66
	ST	3.11±1.18	3.09–3.14	3.96±1.27	3.94–3.97	56.60	.67
Low arousal	OT	2.67±1.18	2.61–2.72	4.02±1.27	4.01–4.03	45.33	1.07
	PT	2.88±1.17	2.84–2.91	4.04±1.20	4.03–4.05	61.14	.97
	ST	2.59±1.21	2.54–2.64	3.88±1.27	3.86–3.89	51.28	1.02
Lack of initiation	OT	3.12±1.18	3.08–3.16	4.06±1.27	4.04–4.07	48.63	.74
	PT	3.31±1.13	3.28–3.33	4.09±1.22	4.08–4.10	59.03	.65
	ST	3.08±1.17	3.05–3.12	3.89±1.29	3.88–3.90	43.93	.63
Disinterest	OT	2.94±1.14	2.89–3.00	4.01±1.27	4.00–4.02	38.54	.84
	PT	3.23±1.13	3.20–3.26	4.04±1.23	4.03–4.05	49.77	.66
	ST	3.10±1.17	3.05–3.14	3.85±1.29	3.84–3.86	32.31	.59

NOTE. Patient factors were rated as present or not present. Observations lack statistical independence because of multiple observations on each patient and made by each therapist.

Abbreviation: LOE, level of effort.

^*Number of observations were OT (45,770 sessions), PT (50,383 sessions), and ST (42,402 sessions).

^†All between-group comparisons were statistically significant at P<.0001.

^‡Effect size calculated as Cohen d using pooled sample SD.

Patients' level of effort in IR

The overall mean level of effort ratings for patients' entire stay were equivalent for the 3 disciplines. Mean ratings indicated average effort, and most patients' performance were rated from minimal to very good effort (table 4). The mean (across sessions) level of effort ratings show generally normative distributions and few outliers. The mean level of effort ratings for OT, PT, and ST were significantly higher in the discharge week than in the admission week (P<.001). Because the level of effort distributions were equivalent among the 3 therapy disciplines, level of effort ratings for PT, OT, and ST were combined and used in all remaining analyses.

Table 4.

Patients' mean level of effort during IR, by discipline

	LOE Over Stay	LOE Admission Week		LOE Discharge Week
Disipline	Mean±SD	Mean±SD	95% CI	Mean±SD	95% CI	t Test	P
OT	4.24±.95	3.76±1.15	3.71–3.81	4.64±1.03	4.60–4.69	37.90	<.001
PT	4.18±.91	3.71±1.09	3.66–3.75	4.58±.97	4.54–4.63	39.82	<.001
ST	4.09±.99	3.64±1.19	3.59–3.69	4.47±1.04	4.42–4.52	35.98	<.001
OT + PT + ST	4.17±.83	3.69±1.02	3.65–3.74	4.56±.85	4.52–4.60	45.84	<.001

NOTE. Patients with ≤8 day rehabilitation LOS (n=118) are considered to have a 1-week stay and are included in the admission week only.

Abbreviation: LOE, level of effort.

Plots of weekly mean level of effort ratings by level of FIM cognition at admission display a clear linear pattern of higher level of effort for each cognition group (fig 1). The size of the between-group differences was substantially reduced by IR discharge (see fig 1). Significant differences (P<.001) were observed within each FIM cognitive subgroup for successive weeks and between each of the FIM cognitive subgroups at weeks 1, 2, and 3, as is shown by nonoverlapping confidence intervals (CIs) (fig 2).

Fig 1.

Box-and-whisker plot of admission and discharge week mean level of effort rating for ST, OT, and PT sessions combined, stratified by admission FIM cognitive subscale score (N=1946). The distributions plotted represent the sample mean of patients' mean level of effort scores for all their sessions during the admission and discharge week, respectively. Patients (n=118) with ≤8 days of IR LOS were included in the admission week only. The lower and upper edges of the box represent the 25th and 75th percentiles, whereas the horizontal line in the middle of the box represents the 50th percentile (median). The dashed, vertical lines (whiskers) extending from the box indicate the range of data up to the 1.5 quartile range. Circles indicate values that extend further than 1.5 quartiles from the box. Abbreviations: Adm, admission; Cog, cognitive.

Fig 2.

Mean level of effort rating for ST, OT, and PT sessions combined for successive weeks of the stay by admission FIM cognitive subscale score category. Patients who completed at least 2, 3, or 4 weeks of IR within each of the 5 admission FIM cognitive subscale score (Adm FIM cog) groups were included in this graph. The values (different shaped dots) plotted on each line represent patients' mean level of effort scores for all their sessions during a given week. The vertical lines and error bars represent the upper and lower 95% CIs. Week is the week of IR in which patients' level of effort was rated. Abbreviation: CI, confidence interval.

Predictors of level of effort scores

A linear mixed-effects regression model produced an interpretable constant term for level of effort (estimate=3.66) that closely approximates the mean admission week level of effort of 3.69 for the sample (table 5). Age, Comprehensive Severity Index brain injury severity score at IR admission, days from injury to IR admission, days from IR admission to treatment date, and PTA and ABS level on the treatment date were all significant predictors of level of effort (P<.001). Presence of PTA reduced level of effort (in relation to the constant) by −.23 (95% CI, −.26 to −.20). Agitated behavior reduced level of effort by −.18 (95% CI, −.21 to −.15) for mild agitation, −.33 (95% CI, −.37 to −.29) for moderate agitation, and −.45 (95% CI, −.51 to −.39) for severe agitation. Days of IR treatment served as a proxy for recovery with each day of IR from days 1 to 19 increasing level of effort ratings by .056 (95% CI, −.054–.058) and each day from days 20 to 90 increasing level of effort by .020 (95% CI, .018–.022). Each 10 years of age >44 years reduced level of effort by −.06 (95% CI, −.08 to −.05); each 10 years of age <44 years increased level of effort by the same amount. Admission Comprehensive Severity Index brain injury severity and days from injury to IR admission had small yet statistically significant effects on level of effort.

Table 5.

Linear mixed-effects model for level of effort

Variable	Measure	Type (label)	Estimate*	SE	95% CI
Intercept	Constant	Time invariant	3.660	.021	3.619 to 3.701
Brain injury severity at IR admission	CSI brain injury score (centered at mean, 46)	Time invariant	−.024	.001	−.025 to −.023
Age at IR admission	Years (divided by 10; centered at mean, 4.4)	Time invariant	−.064	.007	−.077 to −.051
Days from injury to IR admission	No. of days (centered at mean, 26)†	Time invariant	−.006	.001	−.008 to −.004
Days from IR admission 1	No. of days from 1 to 19	Time varying	.056	.001	.054 to .058
Days from IR admission 2	No. of days from 20 to 90‡	Time varying	.020	.001	.018 to .022
PTA, present	OLOG score <25; GOAT score <75	Time varying	−.228	.015	−.257 to −.199
Agitated behavior	ABS	Time varying
	ABS total score, 14–21	(None)	(Reference)
	ABS total score, 22–27	(Mild)	−.179	.014	−.206 to −.152
	ABS total score, 28–34	(Moderate)	−.333	.021	−.374 to −.292
	ABS total score, ≥35	(Severe)	−.449	.031	−.510 to −.388

NOTE. Model is based on 1946 patients who had mean level of effort ratings computed for 38,206 days of inpatient rehabilitation.

Abbreviations: CSI, Comprehensive Severity Index; GOAT, Galveston Orientation and Amnesia Test; OLOG, Orientation Log.

^*All model variables were statistically significant at P<.0001.

^†To control for outliers, number of days from injury was capped at 3SD above the mean (mean ± SD, 26±31; capped at 119d).

^‡To control for outliers, number of days of IR was capped at 3SD above the mean (mean ± SD, 21±23; capped at 90d).

Sensitivity analyses indicated that there were no significant effects or improvements in model fit when including interactions between Comprehensive Severity Index brain injury severity score and number of days from injury to IR admission or between PTA presence and ABS level. No significant effects were found for sex, race, and education level.

Discussion

In the United States there is a growing emphasis on providing patient-centered care, optimizing service utilization, and improving treatment effectiveness and efficiency.^34,35 In rehabilitation, understanding how patient factors impact treatment delivery and ultimately outcomes is a key to providing patient-centered care.³⁶ Level of effort in therapies is considered an important mediator of therapeutic effectiveness; however, few studies have described patients' effort in TBI IR or identified factors that affect level of effort. This prospective, 9-center, longitudinal cohort study described a U.S. sample of 1946 patients in TBI IR and their level of effort in 138,555 OT, PT, and ST sessions. Our study met quality standards for longitudinal observational studies including enrolling a large, diverse sample; using well-validated measures or validating novel ones; minimizing missing data and loss to follow-up; having independent ratings on primary variables; and using a linear mixed-effects regression model. Evidence that our full sample is similar in composition to the population of persons receiving rehabilitation for TBI in the United States suggests that our findings are likely to generalize to most U.S. TBI IR settings.²

Our study provides evidence that patients' level of effort can be observed and reliably rated in the TBI IR setting. The Rehabilitation Intensity of Therapy Scale level of effort rating system provides therapists with 7 observable levels of patient goal-directed activity that take into account their initiation of, attention to, and sustained activity; responsiveness to feedback; and perseverance in sessions. The single-item scale is easy to complete in a time-limited clinical environment and demonstrates solid psychometric properties. OT, PT, and ST level of effort ratings closely conformed to a normative distribution. Therapists demonstrated high levels of accuracy in rating patients' level of effort in vignette-based case studies; high levels of consecutive day, test-retest rating consistency within the same discipline and time of day; and good interrater agreement between different disciplines' same day ratings in consecutive sessions.

Our study provides substantial evidence that physiological factors have large effects on patients' level of effort. Therapist ratings of the presence of impaired arousal, attention, and initiation influencing sessions were significantly related to lower level of effort ratings. In mixed-effects regression model analyses, higher levels of admission brain injury severity and daily ratings of PTA and agitation severity were all strong predictors of lower level of effort. These findings are consistent with the recent TBI IR study by Lequerica et al⁷ in which patients' longer duration of confusion and higher agitation scores were predictors of lower engagement in therapies.

Our inclusion of patients with significant cognitive impairments and their stratification into 5 subgroups based on admission FIM cognitive scores demonstrated the wide variability in patients' mean level of effort in therapies (see fig 2). The inclusion of patients with admission FIM cognitive scores ≤10 likely explains the lower mean patient level of effort observed in our study compared with 4 studies—2 with mixed diagnosis IR populations,^9,10 a spinal cord injury sample,¹¹ and a TBI IR clinical trial⁸—that excluded persons with significant cognitive impairments and/or PTA. Our study also provides substantial evidence that patients' level of effort in TBI IR improves over time. Although a significant amount of improvement is attributable to resolution of agitation and PTA, our mixed-effects regression model results show that patients experienced small serial improvement in effort above and beyond resolution of these secondary conditions. This improvement may reflect natural recovery independent of PTA and agitation and/or resolution of other factors that may interfere with optimal effort (eg, pain, lack of familiarity with the surroundings and treatment staff).

The modifiable risk factors for suboptimal level of effort found in our mixed-effects regression model and our univariate analysis may aid in the identification or development of interventions to improve patient effort in TBI IR. For example, our findings suggest that the large effect of agitation on patients' level of effort underscores the necessity not only to resolve agitation but to do so without attenuating arousal, which also appears to be strongly related to effort. Our findings may also inform clinicians, families, and patients regarding typical levels of effort in IR, physiological factors affecting effort, and an estimated time course for improvement. For example, therapists may keep a record of level of effort that could be automatically calculated as a 3- to 5-session rolling average (either between or within disciplines) by an electronic health record function and displayed on a recovery dashboard to set proper expectations and monitor change.

Study limitations

A number of study limitations should be noted. Patient effort/engagement/participation in rehabilitation therapies remains a somewhat amorphous construct. Use of a single item to describe patient behavior in any and all treatment activities during a session typically lasting from 30 minutes to hours is subject to measurement error because of patient variability in effort over the course of a session. Insufficient training in using the Rehabilitation Intensity of Therapy Scale and therapist inattention to the behavioral anchors when providing ratings may also contribute to measurement error. However, our findings of high therapist accuracy in rating case vignettes and good test-retest and interrater consistency provide evidence that although interrater variability may have been present using the single-item Rehabilitation Intensity of Therapy Scale level of effort scale, it was not a significant issue. The design of this study did not allow for the presence of ≥2 independent level of effort raters during the same session, which limits our ability to differentiate between factors that may explain different level of effort scores from therapist to therapist and day to day.

Research directions

Research on a wide array of patient, environment, and treatment factors that potentially affect patient level of effort was beyond the scope of this study, but it is required. Future research should identify and examine the differential effects of a broader set of antecedent patient factors on effort in TBI IR, including visual and auditory impairments; depression, anxiety, and other psychological conditions; pain; and self-efficacy. Other factors worthy of exploration include environmental and process factors in the IR unit such as levels of noise and distractions in the treatment room; family encouragement or interference; amount and quality of sleep; time of day and duration of therapies; presence of other patients and staff as is common in group therapies; scheduled breaks in the daily program; and patient-therapist working alliance, including patient-centered goal setting and selection of activities. The effects of treatment on patients' level of effort, including use of nutritional supplements, sedating and stimulating medications, and delivery of patient education and psychotherapeutic interventions, should be examined. Finally, future research should investigate whether level of effort impacts the benefit that patients obtain from all or specific types of treatments and specific and global IR outcomes.

Conclusions

Our study provides evidence that patients' level of effort can be observed and reliably rated in the TBI IR setting using the single-item Rehabilitation Intensity of Therapy Scale effort rating criteria. Patients who sustain TBI show a wide range of effort levels in their IR therapies. The level typically improves over the course of their stay. Presence of PTA and agitated behavior are primary factors that substantially reduce patient effort.

List of abbreviations

ABS

Agitated Behavior Scale

CI

confidence interval

ICC

intraclass correlation coefficient

IR

inpatient rehabilitation

LOS

length of stay

OT

occupational therapy

PT

physical therapy

PTA

posttraumatic amnesia

ST

speech therapy

TBI

traumatic brain injury

Appendix 1.

Rehabilitation Intensity of Therapy Scale: level of effort ratings and behavioral anchors

Effort Rating	Behavioral Anchor
7 (superior)	Patient sustains full attention and goal-directed activity throughout the entire therapy session. The patient consistently initiates activity; seeks performance feedback and/or self-monitors performance; adjusts activity based on feedback; and requests more challenging activities. The patient perseveres with therapy tasks, even when activities are extremely physically or mentally challenging.
6 (very good)	Patient sustains full attention and goal-directed activity throughout the entire therapy session. The patient sometimes initiates activity, may seek performance feedback, and adjusts activity based on feedback. The patient perseveres with therapy tasks that are physically or mentally challenging without encouragement or prompting.
5 (above average)	Patient sustains full attention and goal-directed activity during most of the therapy session. The patient rarely initiates activity or seeks performance feedback but consistently adjusts activity when performance feedback is provided. The patient perseveres with therapy tasks that are physically or mentally challenging with some encouragement or prompting.
4 (average)	Patient is generally attentive, follows instructions, and works toward goals during the therapy session. The patient relies on the therapist to direct all tasks. The patient does not seek feedback but sometimes adjusts activity when performance feedback is provided. The patient requires prompting and/or encouragement to continue with therapy tasks that are physically or mentally challenging.
3 (below average)	Patient is inconsistently attentive and may require repetition of instructions and/or redirection toward therapy session goals. The patient is generally unresponsive to performance feedback and rarely adjusts activity when feedback is provided. The patient gives up easily when therapy tasks become physically or mentally challenging.
2 (minimal)	Patient is inconsistently attentive and requires frequent repetition of instructions and/or redirection toward therapy session goals. The patient may refuse to comply with the therapist's instructions and/or requests and may end the session early. The patient does not attempt therapy tasks that are physically or mentally challenging.
1 (absence of effort)	Patient is rarely attentive and is engaged in virtually no goal-oriented activity. The patient either refuses or is unable to comply with the therapist's instructions and/or requests, which may lead to early termination of the session.