Clinical Taxonomy Development and Application in Spinal Cord Injury Research: The SCIRehab Project

Abstract

Background/Objective:

Applying practice-based evidence research methodology to spinal cord injury (SCI) rehabilitation requires taxonomy (typology or classification) of rehabilitation interventions provided by every discipline contributing to SCI rehabilitation. The rehabilitation field currently lacks such taxonomy.

Methods:

SCIRehab project researchers and clinicians representing 7 rehabilitation disciplines from 6 US inpatient SCI rehabilitation facilities worked in discipline groups during 2 face-to-face meetings and weekly discipline-specific teleconferences for 9 months to identify key contributions of each discipline to SCI rehabilitation and to develop a classification of treatment interventions used by each discipline. These clinician groups were charged with designing documentation systems that collected enough details to describe treatment adequately while not imposing an unrealistic data collection burden on clinicians. Completed documentation systems were programmed onto handheld personal digital assistants (PDAs) to facilitate data entry by clinicians at the point of care.

Results:

Seven discipline-specific SCI rehabilitation taxonomies were developed that describe and quantify intervention activities (major categories of treatment offered by the discipline) and the activity-specific details (variables deemed important to fully describe the interventional process). Much treatment information is unique to each discipline; some is common across disciplines.

Conclusions:

The taxonomies provide a format with which clinicians document actual interventions performed with or for patients. The SCIRehab project has developed the first comprehensive multidisciplinary taxonomy for describing the details of the SCI rehabilitation process and designed a PDA-based documentation system based on that taxonomy that allows clinicians to describe the specifics of their interactions with their patients.

INTRODUCTION

The SCIRehab Project is a 5-year, multicenter study recording and analyzing the details of the spinal cord injury (SCI) inpatient rehabilitation process and the first-year outcomes achieved for 1,500 people with SCI. The first paper on SCIRehab in this issue provides the rationale for selecting practice-based evidence (PBE) methodology as the most appropriate to determine which specific interventions or combination of treatment elements in the rehabilitation process are associated with positive neurologic, medical, functional, social, and quality-of-life outcomes, after controlling for patient differences (1). Whiteneck et al (1) also provides an overview of the project design, outlines the overarching hypotheses, and enumerates the study's many data sources.

More than any other research methodology, PBE requires detailed documentation of each patient encounter or intervention by every clinician in the disciplines contributing to SCI rehabilitation. This PBE approach to rehabilitation research can be characterized as a form of “participatory action research” (2,3), a bottom-up approach that is critically dependent on the participation of clinicians engaged in the care process. SCIRehab embraced this concept and assembled a large project team with expertise in each clinical area of rehabilitation. Each of the 6 participating SCI centers identified at least 1 lead clinician in each of 9 disciplines as well as an expert in research design (site co-investigator). Thus, the Project Clinical Team comprises approximately 60 clinicians (physicians, nurses, therapists, psychologists, case managers, respiratory therapists, dietitians) and 20 researchers. In addition, some facilities engaged previous patients and/or staff members with SCI to represent the consumer perspective.

The SCIRehab project clinical team realized that many relevant details of rehabilitation interventions are not recorded adequately by rehabilitation specialists in typical documentation, which tend to focus on those elements that are required to be reported by third-party payers and regulators. Thus, like their colleagues in earlier PBE rehabilitation research projects in stroke and joint replacement (4,5), the SCIRehab clinicians endeavored to develop and implement supplemental documentation to be completed at the point of care (POC) to identify and quantify details of what rehabilitation specialists do on a daily basis.

This paper aims to describe why the investigators in the SCIRehab project (1) saw a need to develop POC documentation, including a classification of treatments delivered, for 7 disciplines that play a role in SCI rehabilitation. It depicts the process by which POC documentation was developed, as well as those documentation elements that are common to multiple disciplines. (Other papers in this series describe in detail the treatment classification for each discipline.) Last, it describes how the POC documentation was implemented in an electronic system for data collection, as well as what is being done to assess the quality of the data contributed by clinicians at the participating centers. The discussion section addresses strengths and weaknesses of the approaches selected and the value of the POC documents for future research and clinical activities.

Need for SCI Rehabilitation Taxonomy

Describing the various processes that are part of SCI rehabilitation (all forms of therapy, education, treatment, and counseling) requires a taxonomy of rehabilitation interventions. A taxonomy (also termed typology or classification) “group[s] phenomena or observations into categories that are objective, mutually exclusive, and useful in scientific inquiry” (6). It reflects the conceptual structure of the domain of interest (here: SCI rehabilitation) and represents the way we organize our knowledge of treatment activities (7). Currently, the rehabilitation field lacks a taxonomy of interventions. The Medical Subject Headings (MeSH) of the National Library of Medicine offers a typology of rehabilitation therapeutic procedures and techniques that is so coarse that it is of limited value; the classification of interventions used in, for example, the Cumulative Index to Nursing & Allied Health Literature (CINAHL) (8), is not sophisticated either.

A well-developed and validated taxonomy or typology of medical rehabilitation interventions, focused on the “active ingredients” hypothesized to carry treatment effects, would go far to advance the field (6). It would offer a mechanism for identifying each of the various treatments, procedures, practices, services, and approaches that are used by rehabilitation professionals. Ideally, identification of treatments would be based on those characteristics of interventions that are relevant to outcomes, both theoretically and practically. Characterization should be followed by quantification, which is a necessary step toward linking interventions to patient outcomes (9).

DeJong et al (9) describe two approaches to developing intervention taxonomies. In a “bottom up,” inductive approach, expert opinion, possibly combined with empirical data on the associations between specific treatment elements, is used to distinguish activities that are assumed to have a differential impact on outcomes. This is the approach that was used in creating the well-known Nursing Interventions Classification (8). A “top-down,” deductive approach would start with a well-developed and validated treatment theory (or a set of midrange theories) (6) and use expert opinion to identify those treatments that fit in this theory—that is, interventions that are stipulated as a necessary and potentially sufficient treatment for the disablement experienced by 1 or more categories of patients (6,9).

To date, efforts to develop rehabilitation intervention taxonomies all have been of the bottom-up variety, presumably because rehabilitation is lacking well-developed treatment theories. This includes classifications to describe interventions provided to patients with stroke (4,10–12) and joint replacement (5). In SCI, van Langeveld et al (13,14) has published a bottom-up taxonomy developed for occupational therapy (OT), physical therapy (PT), and “sports therapy” treatments.

Lack of a formal classification of (SCI) rehabilitation interventions has not prevented therapists from recording elements of their treatments. They make entries in medical records and other clinical files using “free-form” language to describe treatments delivered; however, these notes tend to focus more on the why of treatment (assessment findings) than on the what and how (detailing of treatment sequences). Most healthcare facilities maintain additional administrative record systems, in which yet other information is collected. Often derived from encounter forms, these records form the basis of billing; therapists may enter diagnoses (justifying treatment) and units of treatment delivered. These forms may offer rudimentary information on what treatment was delivered but are insufficient to provide the level of process detail required for PBE analysis. Furthermore, there is limited consistency in how these documentation systems are used among clinicians within a center. The fact that nearly every center has its own documentation forms and methods constitutes another barrier. This was the situation faced by the researchers and clinicians who are involved in the SCIRehab study of inpatient SCI rehabilitation and its effects: no existing record system contained the quality, quantity, and detail on treatment that was seen as necessary to achieve the objectives of the project.

INTERVENTION TAXONOMY

Development

Each of the 6 SCIRehab centers identified a clinical leader for each discipline, an experienced full-time clinician with interest in research who volunteered to coordinate project activities among colleagues. Nine teams (nursing, OT, PT, speech-language pathology [SLP], therapeutic recreation [TR], psychology, social work/case management, respiratory therapy, and physiatry), each comprised of a lead clinical expert from each of the 6 centers, worked separately and together for 98 months using 2 face-to-face meetings and weekly teleconferences to identify each discipline's contributions to rehabilitation and the key elements of each intervention. First, each group reviewed the discipline-specific documentation used in the stroke and joint replacement PBE studies (4,5). Then they asked what interventions do we provide that can be assumed to have significant impact on outcomes for patients with SCI. In addition to impairments, activity limitations, and adjustment outcomes, clinicians also considered a broader range of outcome indicators such as participation in household, family life, community, and society. Ongoing discussion took into account the centrality of various interventions to the practice of each discipline, their presumed impact on outcomes (despite the fact that relatively limited treatment time may be given to the intervention activity), and especially the ability of the average clinician to describe the intervention correctly in a few minutes time using a set of codes (each reflecting a category in the classification) detailing how it was delivered. Included among these identified intervention activities were practices that were never or only infrequently used at certain centers, as was determined during early discussions. For example, OTs in only 1 facility work extensively on bowel and bladder management; some centers are beginning to use gaming systems (eg, the Nintendo Wii; Nintendo of America, Inc, Redmond, WA) for rehabilitation. These (relatively) unique treatment approaches are included in the documentation forms used by all centers, because PBE methodology capitalizes on variations in treatments that are not related to patient needs or other patient characteristics (1).

Next, with the exception of the respiratory therapists and physiatrists, the teams organized these contributions into an ad hoc taxonomy and designed POC documentation (similar to an encounter form) to make systematic documentation of treatment sessions possible. The respiratory therapy and physiatry groups determined that the information they enter into the medical record and other records (eg, CPT4 billing records) is sufficiently detailed that the salient research questions and hypotheses they could formulate could be answered without further data collection.

Each of the other groups decided that its discipline spends a great deal of time in treatment sessions that encompass a wide array of interventions that are not adequately documented in the medical record. A delicate balance was to be found between, on the one hand, collecting enough details to document treatment adequately and, on the other, not imposing a data collection burden that was incompatible with a busy clinical schedule. Their fellow clinicians needed to agree that the supplemental documentation to be completed for the project would not be overly burdensome and detract from traditional care processes. The researchers realized that excessive detail in supplemental documentation coupled with a lack of documentation time significantly increases the risk of a decrease in data completeness and quality. Thus, clinicians from the 6 centers discussed at length which intervention activities were to be incorporated in the POC record and what level of detail about each activity should be included to result in a clinically meaningful data set that would be reasonable to collect.

In selecting intervention activities, the clinicians came to realize that what is practiced in one center often is different from the routine in other centers and that terminology differs from center to center and from clinician to clinician. Therefore, clear definitions of all terms were developed. In addition to developing the content, each rehabilitation discipline decided on the frequency with which clinicians are to complete the POC documentation (after each treatment session or cumulatively for the day/shift).

Each discipline began the POC development process with a rudimentary data capture form that evolved through approximately 20 iterations into a “final” version that was standardized for all centers. The resulting documentation taxonomies (systems of classifying and categorizing interventions) include the intervention detail needed to identify and quantify each discipline's contribution to SCI rehabilitation. The intervention documentation provides a format with which clinicians document actual interventions performed. It does not suggest treatment strategies or impose changes to routine practice.

Content

The PT, OT, TR, and SLP teams followed similar documentation strategies to capture therapy session details: general session information (date/time of the therapy session, identity of center and the therapist), intervention activities and activity-specific details performed in the session, and session-level variables, such as co-treatment, group vs individual approach, and patient and family participation. Also recorded were factors that limit session activities and a few measures of patient achievement/improvement.

The psychology and social work/case management teams took a slightly different approach that was more conducive to recording patient interactions that typically occur multiple times over the course of a day rather than in scheduled sessions. For each clinical day, these clinicians indicate intervention activities or topic/content areas addressed, with whom, and for how long. Nursing did not attempt to duplicate the rich information on their activities that is already documented in traditional source documents such as nursing care narrative notes, flow sheets, or Kardex, from which it can be abstracted. Instead, they adopted supplementary documentation to include the nature and intensity of education and care management activities only, which are not included in any detail in regular documentation.

At the core of each discipline's POC documentation are the discipline-specific taxonomies, which are described in detail in the subsequent 7 articles in this section (15–21). The primary building blocks of each discipline-specific taxonomy are the intervention activities (major categories of treatment offered by the discipline) and the activity-specific details (variables deemed important to fully describe the interventional process). Each discipline has a unique set of activities; OT identifies 26 treatment activities, and psychology focuses on 6 areas of intervention. The discipline groups developed the level of detail associated with each activity as they thought necessary to describe work done during a treatment session. Some activities have extensive subdetails; other activities have very few. At the time of development, clinicians realized that the documentation system would be implemented in a menu-driven software program that could allow for detailed documentation when appropriate, but these details would be transparent when not applicable to a treatment session. Thus, treatment details could be included without being burdensome to the documentation process, as it would be if using paper forms. For example, if a PT works on gait with a patient, the therapist documents details about the surface on which gait work was done, adaptive equipment used, need for joint stabilization, distance ambulated and number of attempts at each distance, and type of body weight support that was provided. However, if the focus in another therapy session is a simpler intervention such as range of motion (ROM), only 2 descriptors (body part involved and type of ROM) are required for documentation. None of the gait details (or details associated with the other 20 PT activities) is presented for documentation once ROM is selected on the main menu. Details about each activity in each discipline's documentation systems are described in the remaining papers in the series.

Taxonomy development focused on describing details of interventions performed with or on behalf of patients with SCI. However, clinical selection of specific activities and the intensity with which a specific activity is performed often are dependent on a patient's functional status and the need for assistance from a therapist. Therefore, taxonomy details were enhanced with additional descriptors that focus on assistance needs, patient direction of care, and family involvement. Although these factors can be interpreted as measures of progress, and thus interim outcomes, clinicians felt strongly that they also help to describe and justify activity selection.

To establish a high level of consistency across disciplines, several aspects of POC documentation are shared across the disciplines; these are described below.

Individual and Group Therapy.

Therapy sessions are classified as either individual or group sessions. Group work is an important component of therapy programs; the amount of group work varies from center to center. Some groups can be quite large; others contain only a few patients. Completing detailed documentation for each participant in large groups can be burdensome to clinicians, so each discipline devised two options for recording group therapy: “Group as Individual” or “Group as Group.” Information included in Group as Individual documentation is the same as documented for an individual session. This option is used if therapists consider it important to document patient-specific information (eg, level of assistance, family involvement) for the individual patients in the group rather than just the intervention activities performed with the group as a whole. Group as Group documentation allows quick entry of the same intervention activity information for all patients in the group but does not include any patient-specific information, other than the identity of the group members.

Co-Treatment With Other Disciplines.

An important component of some therapeutic sessions is co-treatment where clinicians from 2 (sometimes more) disciplines work together with a single patient (or group of patients) on specific patient goals appropriate for their discipline. Billing guidelines sometimes restrict multiple disciplines from billing for overlapping activities; however, clinicians think joint treatment is at times beneficial to the patient, and thus, it is captured in the POC documentation. Each participating therapist completes his or her own documentation for the session, and marks co-treatment with a specific other discipline to indicate collaboration between clinicians.

Time as a Measure of Intervention Dose.

All disciplines chose to quantify the dose of their interventions, or how much of each intervention activity they engaged in with each patient, by tracking time spent on each activity. Time had been the method used to quantify interventions in previous rehabilitation PBE studies, and the lead clinicians chose to maintain that approach in the SCIRehab study, because it seemed the only practical way to measure the quantity of intervention delivered. Time, without doubt, is related to dose of intervention, but for specific treatments, there may be measures that better reflect the amount of active ingredient provided (eg, the number of repetitions offered in passive range of motion). However, counting repetitions or otherwise keeping track of dose was not considered feasible; “time” was maintained as the easiest common denominator for dosage in all types of treatment.

The SCIRehab POC documentation captures the number of minutes spent on each activity. Documenting time allotments for specific details within each activity was considered but rejected for most activity types, because it was burdensome for therapists to record small segments of time. Stopwatch measurement is not used; all entries are estimates rounded to 5-minute blocks of time. Clinicians opted for 5-minute increments because this time break reflects how they typically think of treatment time (5, 10, 15, 20, 30 minutes, etc). Blocks are documented for each intervention activity individually, so that the total number of documented minutes should equal approximately the length of the session.

Level of Assistance Needed in Performing Tasks.

The results of patient assessments generally are well documented in clinical records, whether in clinical notes or using formal measures such as the Functional Independence Measure (FIM) (22), which is completed at admission and discharge, and in many rehabilitation facilities weekly. SCIRehab will abstract this information during the chart review process that occurs after the patient is discharged and use it as a measure of patient deficits at admission to SCI rehabilitation. Although the primary analyses planned for SCIRehab will relate the type and extent of interventions used during SCI rehabilitation to outcomes at 1-year after injury, controlling for baseline deficits and functional assistance needs, therapists wanted to be able to document day-to-day rehabilitation progress as a way to explain the selection and amount of time spent on various intervention activities with patients. Decreasing needs for assistance within an activity can be the trigger to advance to a more complex activity, during which assistance needs may increase. Therefore, a level of assistance assessment was added to the PT, OT, SLP, and TR activity documentation.

Each center's therapists are trained to score the FIM, which is used to describe a patient's functional status at admission and discharge (and in some centers at intermediate points), set rehabilitation goals, and monitor functional gains (23). However, a major concern of the SCIRehab clinicians was that progress within a therapy intervention activity may not be reflected in a change in the FIM score. For example, a person with tetraplegia may improve significantly in a component of the bed transfer FIM item, such as rolling. This improvement constitutes decreased assistance needs for rolling and may contribute to a patient's level of independence. However, it may not be reflected in the FIM item score, which is based on the entire bed transfer that starts and ends in the supine position. The SCIRehab project clinicians wanted to provide a more detailed description of functional status than the FIM instrument could offer. Thus, in the POC documentation, a FIM-like rating of independence can be attached to a patient's performance of only a subset of the skills that are subsumed under the FIM's definition of a task. Another notable alteration to the FIM instrument was distinguishing “total assistance of more than one person” from “total assistance of one person” to demonstrate decreased burden of care.

Patient-Directed Care.

Intervention activity selection also can be guided by advancement in the amount of care a patient is able to direct. If there is an expectation for the patient to direct his or her care, the amount of care the patient directs (all, >50%, ≤50%, or none) is documented by the clinician for each session.

Patient Participation.

Rehabilitation professionals often cite patient motivation as a key determinant of successful rehabilitation outcomes. Anecdotally, patients who readily engage in various aspects of rehabilitation and show enthusiasm to work hard and maximize their potential enjoy better outcomes than patients who lack similar qualities. A large body of literature (24–30) has conceptualized a variety of patient factors (eg, apathy, low self-efficacy, negative personality factors) as predictive of poor outcomes in rehabilitation. Research also examined the concept of motivation and its relationship to various factors, primarily social variables within the rehabilitation setting and in the patient's larger support network (31–33). Because of the difficulty inherent in measuring a largely subjective construct such as motivation, Lenz et al (31) suggested an alternative way to measure patient engagement in the rehabilitation setting. These researchers recommended measurement of “participation,” which they defined as a cluster of observable behaviors, as a surrogate for the concept of motivation. They developed the Pittsburgh Rehabilitation Participation Scale (PRPS) to measure patient participation in PT and OT sessions in an inpatient rehabilitation setting. The instrument was shown to be a reliable measure of patient participation in a sample of primarily elderly rehabilitation inpatients with a variety of admitting diagnoses and was predictive of rehabilitation outcome (as measured by FIM score change) (31). Although this measure was designed for use by PT and OT, other SCIRehab clinical groups (SLP, TR, psychologists, and nurses) thought it offered a promising conceptual framework for describing patient participation. However, slight modifications to optimize the PRPS for each discipline's circumstances were seen as necessary. The adaptation of this tool by each of these disciplines is described in the discipline-specific articles (15–21).

Family/Caregiver Involvement.

Because the education of families and training of caregivers is an integral part of SCI rehabilitation, each discipline involved in the SCIRehab project incorporated a description of family or caregiver involvement in their POC documentation. PT, OT, SLP, and TR document whether the family only observes the session, receives verbal instruction, shows skills independently or requires assistance, or is not present for the session. Psychologists, social workers/case managers, and nurses indicate whether the family is a recipient of education or counseling (15–21).

Missed Therapy and Factors That Impact Sessions.

Scheduled therapy sessions can be altered (from what the therapist is planning based on achievements in the previous session and/or new assessments) in several ways. The most common deviations from a planned session include a decrease in duration of the session or a change in content of the session as dictated by the patient's condition. SCIRehab therapists thought it important to capture some of this information. They realized that outcomes are brought about only by treatment delivered and not by sessions or content skipped. However, the literature has very limited information on how various factors interfere with planned treatments, and the POC documentation seemed to be a feasible mechanism to collect this type of data for SCI rehabilitation.

If the patient misses some or all of a scheduled therapy session, SCIRehab clinicians indicate the number of minutes missed and the reason for missing therapy: refused; patient not available/ready; equipment or therapist not available/ready; and/or patient medical complications.

Content (goals and/or activities used) of a therapy session may be altered depending on the patient's condition. For example, if respiratory complications confine a patient to bed, PT gait activities planned for the day may be replaced with range of motion and stretching exercises in bed. “Factors Impacting Session” includes medical complications (eg, pain, spasticity, orthostasis, and respiratory status), behavior and psychosocial issues, and surgical precautions/limitations. Therapists select as necessary 1 or more factors that impact a session's activities.

Implementation

Electronic Data Capture.

The volume of anticipated treatment documentation at the point of care necessitated the use of an electronic data capture method. After a 2-month trial period during which clinicians tested and modified paper POC forms for a subset of patients, each discipline's “final” POC document was programmed into a modular custom application (PointSync Pro version 2.0; MobileDataforce, Boise, ID) for use on handheld personal digital assistant (PDA; PDA hx2490b; Hewlett Packard, Palo Alto, CA). The costs of this equipment (documentation programming, purchase of PDAs, and training of staff in the use of the equipment) compared with an approach using paper forms are offset by a number of advantages. Separate data entry costs (and potential data entry errors) are eliminated. In addition, drop-down menus make it possible to capture levels of detail that would not fit onto a 1- or 2-page paper form. Last, “tool tips” that define each term used, as well as data quality-checking capacities, can be built into the system, which assist clinicians to minimize errors while recording their treatments.

The POC documentation system at each site includes a separate application for each discipline residing on the clinician PDAs, a study coordinator application located on a personal computer, and a large database residing on a local server. They are connected on a network where cradles allow the transfer of data between the PDAs and the database. The study coordinator application manages the entry of patient data at the time of subject enrollment (patient admission) and assigns subject ID numbers. It also provides rudimentary reporting capabilities, allowing the local study coordinators to check easily which specific clinicians and disciplines have or have not submitted documentation for specific patient(s) and date(s).

The local database holds the patient information table, as well as all clinical data collected at the point of care. The PDA establishes 2-way synchronization with the local database when cradled. Each day, data are pushed from the database to populate the mobile application with patient names and application updates, and data are pulled from the PDAs and sent to the local database.

All POC clinical data residing in the local database are sent to the central project database, which is housed at the SCIRehab data coordinating center. These data do not include any patient identifying information other than the assigned subject number. Encryption and password protection are provided at critical points throughout the system to secure patient data as regulated by the Health Insurance Portability and Accountability Act.

Documentation Training.

A total of 250 PDAs were provided to the 6 centers combined. Clinicians received 1 PDA for individual use or to share with others, depending on their patient caseload. The Project Team conducted on-site training at each center. The first center was trained in mid-August 2007 and the last at the end of September 2007. Every clinician in each discipline group at each center attended a 1.5-hour training session during which PDA functioning and content were described and demonstrated. (This was in addition to content discussions that lead clinicians or local coordinators had with staff in preparation for the Project Team visit.) These sessions were supported with and supplemented by a training manual that included step-by-step instructions for how to enter data (depicted with screen shots), definitions for terms used, and paper copies of the intervention taxonomies. Clinicians practiced using the PDAs the rest of the first training day; they returned for a follow-up 1-hour session the next day to reinforce training issues and have questions answered. Project patient enrollment began with patients first admitted to the center after training was complete.

Initially, completing PDA documentation required quite some time to explore the activity selection fields, especially for PT and OT, whose taxonomies include 22 and 26 activities, respectively. Once clinicians became familiar with the details associated with each activity, time required to document a session consumed less than 1 minute for most sessions; sessions that include multiple activities (eg, OT activities of daily living sessions that include grooming, dressing, bathing, and toileting) take more time because each activity is documented separately.

Lead clinicians at each site continued weekly discipline-specific teleconferences with the Project Team to support clinicians in the field and make consistent cross-center decisions about questions arising during the first few weeks of data collection. If clinicians did not know how to include a specific intervention in the treatment taxonomy, lead clinicians discussed the issues and decided on how to adapt the system to incorporate the activity. For example, PTs wondered how to document sessions that focus on cognitive or visual retraining work while doing grooming. Clinicians felt that even when they work on this type of activity, they still focus on PT goals, such as balance, transfers, rolling, etc, and thus, would document this work in the appropriate categories. When therapists worked on transfers into airline seats, they realized that airline seat needed to be added to the transfer surface list. In addition, identified programming “bugs” were resolved by the application development team. Each discipline's revised application software was updated and “locked-down” for the remainder of the study period.

Documentation Data Validity.

Face validity was built into the clinical taxonomies as they were developed and are being used by practicing clinicians. Clinicians came to concur with the taxonomy content and its organization by discussing findings from the pilot testing they performed and then agreeing to add, edit, or delete items (content validity). Feedback from other disciplines and from the separate group of rehabilitation researchers helped assure that no important interventions were omitted and that each taxonomic tree and each intervention's subclassifications were as logical as clinical expertise and existing research information could assure. Predictive validity will be assessed when taxonomy-differentiated interventions are incorporated into project analyses and are shown to be associated with specific outcomes, while controlling for patient differences.

Documentation Data Reliability and Completeness.

Clinical leaders in each discipline group developed written case scenarios that describe typical clinician–patient interactions in that discipline and then agreed on the appropriate codes (PDA menu selections) to document the described session. Reliability testing requires each clinician in each discipline to read such a discipline-specific case scenario and enter the codes characterizing the session into the PDA. The degree of correspondence between the codes agreed on by the clinical leaders and the codes entered by individual clinicians constitutes a rough indicator of the clinician's knowledge and application of the system. The first reliability testing occurred about 1 month after initial training. PDA data for the case scenario were sent to the project database and results were compiled for each discipline. Reliability testing was repeated after the second month of data collection (using a different scenario) and then quarterly for the remainder of the data collection period. The goal of the reliability testing process is 2-fold. First, the “test score” tracks how clinicians perform on each reliability scenario and over time. Second, and most important to clinicians, the reliability case scenarios serve as continuing education opportunities in which all taxonomy concepts and details of documentation are reinforced periodically.

In feedback reports shared with the clinicians, the reliability test results are separated into sections that cover the major documentation areas. For example, PT reliability reports contain information about session level variables (date/time), intervention activities performed during the session (bed mobility, transfers, gait, etc), the associated activity-specific details, family involvement, patient variables (participation, reason for missed therapy, factors that limit session activities), and interim progress measures (assistance needs, amount of care the patient directs). In the reliability reports, frequency distributions are provided for each entry field so that lead clinicians and other study staff can see the number of correct and incorrect responses entered for each field. A summary of entries for each clinician is provided so that the lead clinician in each discipline at each center can follow-up with individual colleagues, as needed, to improve understanding of the taxonomy elements and other concepts. The results of the initial reliability assessments indicated that, across all clinicians in all disciplines at all centers, an overall 85.1% exact agreement was achieved with “gold standard” documentation established by the lead clinicians. Higher reliability was reported for intervention activities (88.9% exact agreement) and for session-level variables (95.1% exact agreement).

To ensure completeness of session documentation, the project coordinator at each site compares PDA entries with billing forms for each discipline as soon as both are available. Therapy sessions that are billed but not documented in the PDA are reported to the lead clinician in the relevant discipline so that documentation can be completed retroactively, if possible.

DISCUSSION

The primary contribution of the SCIRehab Project to the field of SCI rehabilitation to date is the development of a new comprehensive SCI rehabilitation treatment taxonomy integrated into an electronic POC documentation system, which will provide the first detailed examination of the SCI rehabilitation process. This comprehensive database will offer the opportunity to examine the complex interplay of patient and process factors and their impact on outcomes for patients with SCI.

The decision to use the PBE approach necessitated a standardized documentation process to identify and quantify types of rehabilitation intervention activities used at the participating centers. Routine clinical documentation typically does not include details to quantify therapeutic intervention intensity (eg, what specific intervention activities/topics are addressed and how much of each is delivered). Overall session time alone does not provide this quantification; thus, capturing the detail of each discipline's sessions required use of supplemental POC documentation.

This PBE approach to rehabilitation research can be characterized as a form of “participatory action research” (2,3)—a bottom-up approach that not just values, but is critically dependent on, the participation of clinicians engaged in the care-providing process. (The SCIRehab project also obtained input from individuals with SCI; they made valuable contributions to capturing postdischarge mentoring activities, as well as the outcomes that needed to be included in follow-up data collection.) SCIRehab garnered extraordinary contributions of clinical expertise and time to develop new intervention taxonomies and to collect an unprecedented level of intervention detail at the point of care. The physicians, therapists, psychologists, social workers/case managers, and nurses involved in the project (both the clinical leaders and their front-line colleagues) realize that better understanding of the details of everyday practice (obtained from data, not just based on individual impressions, or expert consensus) and the association of these details with patient outcomes can make great contributions to better outcomes for patients with SCI and better training for clinicians. The level of detail about SCI rehabilitation that became a part of each discipline's intervention taxonomy has never been documented before and provides tremendous potential to discover treatments that are best for specific patient types.

PBE methodology, as implemented in the SCIRehab Study, uses clinicians to develop a comprehensive bottom-up treatment taxonomy based on their practical experience to describe what they do. It is unclear whether a more top-down theory-driven taxonomy would better differentiate more effective from less effective interventions. The whole issue of differentiation, classification, and quantification of behavioral and other treatments is new in healthcare in general and in medical rehabilitation specifically. Some areas may have gone further in developing theories underlying treatments (eg, cognitive rehabilitation for executive dysfunction), but even in these specialty areas, intervention taxonomies do not exist.

Although PBE is designed to examine the natural variation in treatment occurring in everyday practice without introducing any new interventions, the very introduction of a systematic taxonomy and a new documentation system may have an impact on practice. Clinicians who are provided an PDA may alter treatment to “try an option” on the documentation activity list that they may not have used previously or may add to a treatment plan so that each “field” in the POC can be filled in with session data. Such decisions are not prohibited in a PBE study (as opposed to a randomized clinical trial), whereas in principle this behavior would reduce between-therapist variance in treatments used, it is very unlikely that it happens frequently enough to impact the outcomes of study analyses.

As much as supplemental POC intervention documentation using handheld devices such as PDAs provides an unprecedented level of detail about rehabilitation interventions, it too has intrinsic limitations. The menu-driven PDA programs, requesting only data relevant to a particular intervention activity, hide the complexity of a data structure that can not be duplicated in paper and pencil forms in case of a computer failure. A review of the treatment classification offered in any one of the relevant articles in this series (15–21) will make clear that even the simplest classification is too complex to fit on a standard sheet of paper and leave space for the various nontaxonomy elements the clinicians collect. The very fact that a PDA allows far more information to be collected than paper forms may have encouraged clinicians to specify more data than will prove useful. Supplemental documentation added to traditional charting increases the documentation burden of front-line staff and allotted documentation time may not be sufficient to ensure complete documentation of both. SCIRehab local coordinators monitor the quality and quantity of POC documentation to keep missing or poor information to the minimum compatible with clinical responsibilities, which always come first.

Electronic issues must be addressed frequently by busy clinical staff. Learning to use the technology was the first challenge for clinicians at each site. Clinicians were trained to use the PDA before data collection; however, differing levels of preexisting technology knowledge played a role in the ease of which they were able to learn and adapt quickly to inputting data efficiently into the handhelds. Transfer of data through a synching process is seamless and consumes only seconds when it works as designed. However, as with any electronic device, the PDAs need to be rebooted occasionally or to have software applications reinstalled. As often happens with new technology, problems with storage and retrieval of real-time data initially were difficult and created user dissatisfaction. This was especially true for centers or time periods where clinicians had a large caseload of study patients.

Although the electronic data capture and the synchronization system makes quick assessments of reliability feasible using written case scenarios, there is no guarantee that success on the written scenarios adequately portrays a clinician's ability to input data from actual patient treatment sessions. Identification of intervention activities may be fairly straightforward, but assessment of assistance needs, patient participation, and specific intervention details can be less concrete and more difficult to describe in a written scenario.

CONCLUSION

Development of new discipline-specific clinical treatment taxonomies and the implementation of an innovative PDA application to more fully document the SCI rehabilitation process are but the first phases of the SCIRehab Project. It will be in later phases that the true value of the taxonomies and documentation will become evident. If the taxonomies differentiate activities that are more strongly associated with positive outcomes after SCI from other activities that seem to have a minimal relationship to outcomes, the taxonomies will have described the rehabilitation process in a way that facilitates changes in practice to improve care.

This article provides general background for the 7 articles that follow, which describe discipline-specific taxonomy development and implementation issues. This set of articles details the POC data collection protocols for the SCIRehab Project. However, more importantly, these articles describe the first attempt to develop and implement a comprehensive taxonomy of SCI rehabilitation involving all disciplines represented on the rehabilitation team. Although developed for a specific research project in SCI, the taxonomies and documentation tools may serve as a basis for the next generation of taxonomies and documentation designed for much broader clinical use.