New Approach to Study the Contents and Outcomes of Spinal Cord Injury Rehabilitation: The SCIRehab Project

Abstract

Background/Objective:

Limited research evidence is available to show the effectiveness of the many specific interventions provided in spinal cord injury (SCI) rehabilitation; what is available typically focuses on effects of the full rehabilitation package but not specific therapy interventions, medical procedures, patient education, or counseling. Given the problems of conducting randomized controlled trials (RCTs) in rehabilitation, practice-based evidence (PBE) research has been suggested as an alternative methodology for identifying which rehabilitation interventions are associated most strongly with positive outcomes, after controlling for patient differences. Using the PBE research methodology, the SCIRehab project attempts to “open the black box” of acute SCI rehabilitation, provide detailed information on treatments delivered by all rehabilitation disciplines, and contribute to outcomes-based guidelines for clinical decision-making.

Methods:

The SCIRehab project includes 1,500 patients with acute SCI, consecutively admitted to 1 of 6 US inpatient rehabilitation facilities. Details of the rehabilitation process are captured by clinicians from multiple disciplines documenting their interventions in handheld personal digital assistants after sessions with their patients. Outcome data are abstracted from medical records (clinical outcomes data) and obtained from patient interviews at 6 and 12 months after injury. Extensive patient, injury, and other treatment characteristics are abstracted from medical records. SCIRehab is the first research project to collect detailed information on individual interventions offered by the full rehabilitation team.

Results:

SCIRehab is the first research project to collect detailed information on individual interventions offered by the full rehabilitation team. These findings are presented in a series of 9 articles.

Conclusions:

To date, SCIRehab's major contribution is a system for categorizing specific contributions of each discipline and a technology for documenting that detail. After data collection is complete, future manuscripts will relate those process elements to outcomes. The SCIRehab Project is an important step toward establishing outcomes-based guidelines for SCI rehabilitation.

INTRODUCTION

Inpatient rehabilitation for spinal cord injury (SCI), like all of medical rehabilitation, has been studied largely as an undifferentiated black box (1,2). Research has examined effects of the whole “rehabilitation package” (3) but has not addressed which specific therapy interventions, medical procedures, patient education and counseling approaches, or other activities are offered to whom, when, or whether they are effective when offered in various combinations or sequences for specific types of patients and impairments. The SCIRehab project described in this and the other articles in this issue attempts to “open the black box” of acute rehabilitation for individuals with SCI, provide detailed information on the treatments delivered by all SCI rehabilitation disciplines, and contribute to evidence-based guidelines for clinical practice. This paper provides an overview of the reasons for and methods of the study. The second article in this series by Gassaway et al (4) describes how representatives of rehabilitation disciplines at 6 centers developed a taxonomy for categorizing rehabilitation interventions and a documentation system suitable for capturing detailed treatment information prospectively and provides information on documentation elements that are common to the information collected by most disciplines. The remaining papers in the series describe the interventional taxonomies developed by 7 disciplines and the unique elements of their documentation; they also place the information collected on SCI rehabilitation against the background of issues in the disciplines involved (5–11).

Effectiveness of SCI Rehabilitation

Each year in the United States, approximately 11,000 people survive initial trauma and are hospitalized with a new SCI (12,13). It often results in a life-threatening condition that includes varying degrees of motor paralysis and sensory loss and impairment of bowel, bladder, sexual, and other physiologic functions. Because of its impact on multiple systems, SCI has been classified among the 5 most expensive hospital diagnoses (14). SCI onset is life-changing, and a lengthy program of SCI rehabilitation is needed to train patients to use their remaining abilities and support “adjustment” to a changed body and life situation.

The first writings on SCI treatment and rehabilitation were marked by extreme pessimism, calling SCI “an ailment not to be treated” (15,16). Today, not only has survival improved dramatically, but life expectancies of individuals with less severe SCI are approaching those of the general population (17). The literature provides clear evidence of the success of modern medical rehabilitation for SCI but also highlights that much more research into the nature, quality, and effectiveness of inpatient rehabilitation is needed (18).

Application of the criteria of the evidence-based medicine movement has exposed the fact that little high-quality evidence, produced using RCTs or other rigorous research designs, exists to show the effectiveness of rehabilitation or even specific rehabilitation treatments; the highest level of evidence for much of rehabilitation continues to be only “expert opinion” (19). New technology and treatment interventions are being implemented without an adequate understanding of their effectiveness and appropriate timing. This is true for SCI even more so than for other rehabilitation areas. A recent review of the state of the science of SCI rehabilitation is revealing: only 5 “Level 1” (RCTs) treatment studies involving participants with SCI were identified, whereas there were 15 focused on traumatic brain injury and 12 on burn rehabilitation (18,20,21). A few high-quality studies investigating the effects of specific drugs, (eg, steroids) or of processes (eg, body weight–supported treadmill training) (22) yielded controversial or unexpected findings that raise questions about whether such large and expensive studies should be performed without supportive Level 2 evidence obtained through preliminary cohort and case-control studies. Practice-based evidence (PBE) studies as described here can serve to develop this supportive evidence.

SCI inpatient rehabilitation sometimes has been studied as an undifferentiated program, using observational research designs to identify beneficial characteristics. Such an approach allows for comparisons between patients who received rehabilitation and those who did not, between those who received it early vs late, between people who received intensive treatment and those whose program was less intense, or between those who had longer rehabilitation length of stay vs individuals with shorter stays. Several studies concluded that longer stays in rehabilitation facilities were associated with increased functional gains, but variations in improvement rates were seen in different impairment groups (23,24). A number of studies have found that early SCI rehabilitation is beneficial (25–29). Studies of inpatient rehabilitation found that initial functional status is associated with differences in rate of functional improvement (30) and length of stay (31), although an Australian study found large variability in discharge outcomes within groups defined by admission functional status (32).

The number of studies of this type is small. However, there are large numbers of studies investigating the relationship between patient characteristics and outcomes. The association of outcomes after SCI rehabilitation with personal and injury characteristics, including age (26,27,33,34), sex (35–37), ethnicity and cause of SCI (31,38–40), education level (41–43), traumatic vs nontraumatic cause of spinal cord dysfunction (43–49), degree and level of neurologic impairment (42,50–59), presence of traumatic brain injury (60), and preinjury alcohol problems (61), has been studied extensively; the findings sometimes have been mixed or contradictory. Many of these studies made only minimal or no distinctions in the component parts of inpatient rehabilitation (eg, distinguishing only long/short stay or high/low intensity treatment), and they did not report on specific components and details of multidisciplinary therapeutic rehabilitation interventions. An exception was the study by Heinemann et al (3), in which total time spent in physical therapy (PT) or occupational therapy (OT) per day was not found to be associated with improved functional outcome (3), but even here, only the time in therapy was studied and not the content of therapy.

Although there is a substantial amount of literature on rehabilitation after SCI, this summary highlights gaps that the SCIRehab study will address. The most dramatic limitation of the undifferentiated therapy approach, typified even by the study of Heinemann et al (3), is that it does not address which treatments are offered, in what combination or sequence, for whom, in what settings, and when. Tracking time spent (eg, length of stay in general or hours in PT and OT) without identifying the specifics of which treatments were used is an inadequate approach to identifying important sources of variation in rehabilitation outcomes. This conclusion is supported by the results of the Post-Stroke Rehabilitation Outcomes Project, which found that total time spent in PT and OT per day did not explain variations in outcomes, but time spent on specific PT and OT activities and interventions did (62).

Because the evidence base for specific SCI rehabilitation interventions is generally weak, and clinical guidelines in SCI rehabilitation are few and cover only a small part of all that rehabilitation specialists do (63), and are not immediately and universally adhered to on publication (64), acute inpatient SCI rehabilitation practices remain highly variable from site to site and perhaps even from clinician to clinician within a site. Our lack of understanding of how recovery, functional independence, and psychosocial outcomes are influenced by the complex interplay of multi-disciplinary rehabilitation care and patient characteristics (e.g., comorbidities, injury severity, and demographics) has allowed cost-containment measures (e.g., shortened lengths of stay) to be implemented without adequate attention to their effects on outcomes. A more complete and systematic understanding of what treatment factors and processes lead to better outcomes, and for which patients, would allow development of more cost-effective and efficient SCI rehabilitation care. The information needed is broader than what can be gained from comparing two treatment protocols in an RCT. We need to “open the black box” of SCI rehabilitation and identify presumed sources of variance in outcomes. Information gleaned in this way can be used to design randomized controlled trials, guide clinical pathways development, or stimulate development of new and innovative treatment approaches.

Evaluating Effectiveness of Rehabilitation

If the “effectiveness” of acute SCI rehabilitation as an undifferentiated whole is to be studied using the strongest research design, there seems to be no alternative to using a randomized design. However, rehabilitation is now a national standard of care, and randomization would require withholding treatment, which is not ethically feasible. There are other salient reasons why randomized designs, specifically the RCT, are not optimal for studying something as complex and individualized as rehabilitation (65). For instance, unlike an active drug and a placebo, rehabilitation treatments are difficult to blind, certainly to the clinician delivering them. RCTs tend to be expensive because of (a) the need to find patients qualified by all inclusion and exclusion criteria, willing to be in the research, and willing to be assigned to either treatment arm; (b) the requirement to train all participating clinicians in a very precise protocol; and (c) in most instances, the need to pay for the treatment out of the research budget because it is not “usual” treatment that can be charged to the patients or their third parties. Costs in the tens of thousands of dollars per enrolled subject are not uncommon; the Clinical Antipsychotic Trials of Intervention Effectiveness schizophrenia trial followed 1,493 subjects for $64 million (65); SCIRehab is expected to follow the same number of cases at a cost of $5.8 million. The major reasons for the cost difference is that no extensive searches for narrowly defined categories of patients are needed and that observation is simply added to ongoing care processes.

The quantity of distinct interventions used by a full SCI rehabilitation team is large. RCTs are severely limited in the number of interventions that they can test at any one time; they examine an intervention in isolation from other interventions to detect the unique contribution to outcomes of one or, at best, a few variables. In showing an intervention's efficacy (impact under optimal circumstances), they cannot address its effectiveness (impact under real world conditions.) For that reason, Tunis et al (66) called for new research methodologies to fill the gaps in the supply of information needed by clinical and health policy decision makers. They described what they called the next phase in the evolution of clinical trials, namely pragmatic or practical clinical trials (PCTs) “for which the hypothesis and study design are developed specifically to answer the questions faced by decision makers” (66).

PBE methodology uses detailed descriptive data on rehabilitation practices to examine relationships among patient characteristics, the content of therapy, and their effects on rehabilitation outcomes (65,67,68). In contrast to the interventional nature of the RCT and PCT, PBE is an observational approach that does not disrupt treatment. It offers a naturalistic view of rehabilitation treatment by examining what happens in the care process, without attempts to alter the treatment regimen to evaluate the efficacy of a particular intervention. PBE methodology is characterized by the following:

• Providers are involved in the design and implementation of the research as well as in the analysis and reporting of data; they play a role in such issues as specifying study questions, defining variables, collecting detailed data, guiding data analysis by statisticians, and co-writing reports.

• Comprehensive data are collected from administrative and medical records and (in settings where the nature of treatments is reflected insufficiently in routinely completed medical and other records) from point-of-care (POC) documentation: special forms used to record systematically and in considerable detail the treatments administered each session, shift, or day.

• Detailed data on interventions allow researchers to focus on specific types of care rendered and to perform analyses of outcomes consistent with current knowledge and with insights offered by clinical participants.

• Inclusion and exclusion criteria are minimized so as to enhance the generalizability of findings.

• No constraints are placed on clinicians to follow a specific protocol; in fact, variety in treatments used for the same patient problem is desirable because it allows exploration of the effects of treatment variations on outcomes.

• Comprehensive analyses focus on multivariate associations between treatments and outcomes, controlling for baseline patient differences such as severity of illness.

• Large numbers of patients (typically more than 1,000) are included from multiple centers that use different treatment philosophies and programs, which makes possible capitalizing in the analysis of the effects of interventions on outcomes on variations in treatments.

• Patient differences are controlled statistically in evaluating the relative impacts of treatments.

• Details of patient characteristics, including illness severity and functional status measures, possibly at multiple times, are analyzed.

• Prospectively specified hypotheses, large numbers of subjects, internal replication of analyses (for subgroups of patients, for related outcome measures, for parallel treatment measures, etc), and checking of findings against clinical expertise prevent the testing of multiple hypotheses in the same dataset from turning into a “fishing expedition.”

PBE advances beyond the RCT and PCT approaches by using a broad sample of patients and assessing the full range of treatments. PBE is well suited for the rehabilitation paradigm of multiple rehabilitation practitioners providing individualized services concurrently. It is likely that the interaction of interventions influences rehabilitation outcomes. Relatively small, nonsignificant effects of a single intervention may be magnified when used in combination with other interventions (69). On the other hand, interventions that seem effective in isolation may be antagonistic or less effective when provided together. In addition, the effectiveness of combinations of interventions is likely to vary across patients.

PBE methodology overcomes many of the limitations of simple observational studies by the manner in which it creates a comprehensive database of patient characteristics, disease-specific, physiological severity of illness measures, therapy interventions, medical and nursing procedures, patient education and counseling, and detailed outcome information. PBE research methodology as applied in rehabilitation isolates specific components of rehabilitation interventions and determines how, and to what degree, each component is associated with outcomes, after controlling for individual patient differences (70–73). Details of the SCIRehab study, which uses PBE methodology, are provided below.

SCIREHAB PROJECT

The SCIRehab Project, a 5-year research effort funded by the National Institute on Disability and Rehabilitation Research (NIDRR), is examining outcomes attained during initial rehabilitation and in the first year after injury. The following outcomes are being examined: neurologic recovery; functional independence; discharge to home; medical complications and rehospitalizations in the first year after injury; and return to productive activity, extent of societal participation, and perceived quality of life reported at the first anniversary of injury. It links patient characteristics and treatments to these outcomes. The research is ongoing, with patient enrollment scheduled to end in early 2010, and the last 12-month postinjury follow-up interview to be completed a year later.

Hypotheses

Figure 1 shows the general scheme for hypothesis testing. Directional hypotheses are posited consistent with the research cited previously. The hypotheses listed provide broad “signposts.” Teams of clinicians and researchers are formulating, before the data are available, more specific hypotheses linked to their prior decisions as to what information to collect.

• Hypothesis 1: Individual patient differences in severity of spinal injury explain variation in outcomes after acute rehabilitation.

• Hypothesis 2: Individual patient differences in demographic characteristics and severity of illness (complications and comorbidities) explain variation in outcomes after acute rehabilitation.

• Hypothesis 3: Controlling for patient characteristics, particular and identifiable medical procedures and therapy interventions are associated with better outcomes.

• Hypothesis 4: Specific interactions of levels of impairment with treatment activities are associated with better outcomes, controlling for patient characteristics.

Collaborating Facilities

The Rocky Mountain Regional Spinal Injury System at Craig Hospital leads the SCIRehab collaborative research partnership of 6 SCI rehabilitation facilities, including Carolinas Rehabilitation, Charlotte, NC; Mt. Sinai Medical Center, New York, NY; National Rehabilitation Hospital, Washington, DC; Rehabilitation Institute of Chicago, Chicago, IL; and Shepherd Center, Atlanta, GA. These 6 centers are not a probability sample of SCI rehabilitation facilities in the United States, but provide geographic and patient-level variations, which may affect outcomes. It also is assumed that significant practice variation exists among participating centers, contributing to the variance that PBE methodology capitalizes on. The Institute for Clinical Outcomes Research, Salt Lake City, UT, serves as the data and analytical center, and MobileDataforce, Boise, ID, offers programming and software support for the handheld computer devices on which POC documentation is entered.

Five of the 6 participating SCI centers are model SCI systems of care funded individually by NIDRR. As such, they compile standardized information on injury-through-community discharge (reported on Form I) and follow-up status at regular intervals after discharge (Form II) on select patients and submit it to the National Spinal Cord Injury Statistical Center Database, Birmingham, AL. Because of more restrictive inclusion/exclusion criteria, SCI cases contributed to the National SCI Database by the study centers are a subset of the SCIRehab patients. However, Form I and Form II for the first anniversary are completed for all SCIRehab subjects at all centers to provide key patient and injury characteristics and outcomes, as well as to maintain a basis of comparison with the participants in the National SCI Database.

Patient Enrollment

Each center received institutional review board approval for this study and obtained informed consent from each patient (or their parent/guardian). Patients who are older than 12 years of age with new traumatic SCI are enrolled consecutively. Each facility began enrolling patients with SCI during the fall 2007 with an enrollment goal of 1,500 patients across the 6 centers.

Study Data

PBE methodology focuses on the processes of care a patient receives; it addresses interventions and patient management strategies. Much of this information comes from routine documentation through postdischarge chart review. The SCIRehab project team cautioned that many relevant details of interventions used in (SCI) rehabilitation are not documented adequately; medical records tend to focus on elements that are required by third-party payers and accreditors. The SCI clinicians agreed that the need to capture details of what rehabilitation specialists do on a daily basis is essential, and thus, POC documentation was incorporated as it had been in the earlier stroke and joint replacement rehabilitation PBE studies (68,74). An innovative point of departure from the two earlier rehabilitation PBE studies is that, rather than using paper forms, SCIRehab uses specialized software (PointSync Pro version 2.0; MobileDataforce, Boise, ID) on electronic handheld personal digital assistants (Hewlett Packard iPAQ hx2490b; Hewlett-Packard, Palo Alto, CA). The paper by Gassaway et al (4) in this issue describes the electronic data capture applications and methods, along with the process that was used to develop the discipline-specific POC documentation (4). Specifics for each discipline's POC are described in the remaining papers in this series (5–11).

Study data also are obtained from the following sources:

• Medical record information is abstracted by specially trained medical record technicians. This includes data on treatments by nursing (except for care management and patient education, which are documented on a POC instrument because information routinely in the record was deemed to be insufficient) (10); respiratory therapy and physiatry; secondary impairments and complications during rehabilitation; and the results of formal assessments by all rehabilitation disciplines. One major component of the information abstracted is the data needed for a SCI-specific version of the Comprehensive Severity Index (CSI), a software application that produces disease-specific physiologic severity of illness scores (75–79). CSI is an age- and disease-specific measure of physiologic and psychosocial complexity, based on more than 2,100 signs, symptoms, and physical findings.

• Model SCI System Form I and Form II data, which describe the patient from before injury through rehabilitation discharge (Form I) and from discharge through the first anniversary of injury, including neurologic, functional, social, vocational, and psychologic outcomes at the anniversary (Form II).

• Separate 6- and 12-month postinjury interviews are used to supplement Form II data with additional detail, especially on outpatient rehabilitation and health services received that might explain first anniversary status.

• Data on all medications received by the patient while in inpatient rehabilitation are provided by the electronic databases of the pharmacy departments of the participating hospitals, including name, dose, and route administered, for every day of the rehabilitation stay.

• Information on physician attendance (attending physiatrists and consultants) is obtained from billing and medical records.

• Profile information is obtained about each clinician participating in POC data collection. Clinical training, SCI expertise, and current practice patterns (full-time vs part-time SCI services) are among the information collected.

Database Management

All data are submitted to the Institute for Clinical Outcomes Research (ICOR) for quality control and integration. Patients, clinicians, and facilities are identified by study identification number. The magnitude of the project is indicated by an expected 600 clinicians at 6 centers documenting 300,000 sessions with 1,500 patients. The clinician and patient study identification numbers, as well as the service dates for POC and medication, make it possible to determine who received what specific service or medication when and from what clinician who had what background. The dataset is exported for analysis with SAS statistical software (SAS statistical software, v 9.1.3; SAS Institute, Cary, NC).

Data Analysis

When complete data are available, analysis will be a collaborative effort between (lead) clinicians and researchers at the 6 SCI centers and ICOR statisticians and scientists. Following standard PBE methodology, the overarching hypotheses specified above give direction and are developed further by clinicians, who will specify what analyses are to be done and how data are to be interpreted.

DISCUSSION

The largest research database on SCI in the world is the National SCI Statistical Center Database, with more than 25,000 Form I records (each representing a person incurring a SCI since 1972 and receiving treatment in 1 of the 26 centers that have contributed data) and more than 150,000 Form II records with follow-up data (80). However, detail as to what specific rehabilitation treatments were administered by various disciplines is lacking. Other research projects, such as the RCT to evaluate body weight–supported treadmill training (22), have collected treatment information on select patients, but only with respect to those narrowly defined interventions of interest. The same activity is happening in ongoing research by, for example, the NeuroRecovery Network funded by the Christopher and Dana Reeves Foundation (81). The SCIRehab Project is the first study to address care processes and outcomes of inpatient SCI rehabilitation on a comprehensive scale.

SCIRehab is assembling a detailed database that will provide the opportunity to examine the complex interplay of patient and process factors and their impact on outcomes. The study's primary contribution to date is the development of a comprehensive SCI rehabilitation treatment taxonomy or, more properly, a set of 7 discipline-specific classifications of key elements of SCI treatments. These taxonomies serve as the basis for a integrated personal digital assistant documentation system at the POC that will provide the first detailed examination of the SCI rehabilitation process. Except for recently published work by van Langeveld et al (82,83), this is the first attempt to describe systematically and comprehensively what therapists and nurses do for and with their patients with SCI to make possible “life after rehabilitation.” van Langeveld et al (82,83) limited their focus to occupational, physical, and “sports” therapy interventions targeting mobility and activities of daily living. The SCIRehab taxonomies reflect activities by 7 disciplines focused on a multitude of outcomes.

PBE-type studies are no panacea for our lack of knowledge as to what are the best ways of treating persons with new onset SCI. As suggested above, they can be used as a precursor for RCT or PCT studies, producing the Level 2 evidence needed before investing major resources in a randomized trial. It also has been proposed that the unanswered questions remaining, once a clinical trial shows that a particular intervention is effective, are best handled using PBE designs. For instance, a PBE study may be used to answer questions about efficacy, ways of packaging treatments, and differences among subgroups of patients in their ability to benefit from the treatment. PBE “focuses on actionable findings that can be implemented to improve effectiveness of care” (65). PBE studies and RCTs are not alternatives, but designs optimized for different purposes that are best used in tandem to answer important questions as to what works in health care and rehabilitation.

In this role, PBE studies have certain limitations. There is the problem of determining a priori what information is crucial to document on treatments (or the active ingredient that brings about the desired outcomes).

CONCLUSION

Research is needed to show that the taxonomic distinctions the lead therapists made are indeed useful—that is, differentiate between therapies with unequal impacts on outcomes. A similar problem is that of the nature of information on patients needed to control for patient differences statistically while drawing conclusions about treatment effects. If an important variable is not measured, it cannot be used as a statistical control and potentially becomes a confounder. Last, incomplete documentation always is a potential problem. In prospective studies such as RCTs, funds are used to pay the salaries of data collectors—clinicians or researchers who are blinded to treatment conditions collect information on outcomes. PBE studies tend, instead, to rely on documentation that clinicians create as part of clinical services. It is no secret that, even in electronic medical record systems, these data may be far from complete or may systematically differ in quality and quantity from one clinician to the next. In SCIRehab, POC documentation, which has been the major addition to PBE methodology in rehabilitation studies, is completed by clinicians in addition to their standard medical record entries. To date, completion percentages have been high; the investigators and the lead clinicians owe their colleagues a debt of gratitude.

Figure 1. Conceptual framework of hypothesis testing.