Feasibility and Validity of the Coma Recovery Scale-Revised For Accelerated Standardized Testing: A practical assessment tool for detecting consciousness in the intensive care unit

Abstract

We developed and validated an abbreviated version of the Coma Recovery Scale-Revised, the CRSR-FAST (CRS-R For Accelerated Standardized Testing), to detect conscious awareness in patients with severe traumatic brain injury (TBI) in the intensive care unit. In 45 consecutively enrolled patients, CRSR-FAST mean[SD] administration time was approximately one third of the full-length CRS-R (6.5 [3.3] vs 20.1 [7.2] minutes, p<0.0001). Concurrent validity (simple kappa=0.68), test-retest (Mak’s ρ=0.76), and inter-rater (Mak’s ρ=0.91) reliability were substantial. Sensitivity, specificity, and accuracy for detecting consciousness were 81%, 89%, and 84%, respectively. The CRSR-FAST facilitates serial assessment of consciousness, which is essential for diagnostic and prognostic accuracy.

INTRODUCTION

Bedside examination plays a key role in clinical management of patients with disorders of consciousness (DoC) in the intensive care unit (ICU) and, in addition to serving as the “gold standard” for diagnostic assessment, is the primary means of determining intensity of care, detecting complications, monitoring rate of recovery, establishing prognosis, and planning discharge disposition.¹ However, studies of diagnostic accuracy in patients with DoC conducted in both acute and post-acute settings consistently indicate that 30–40% of those judged to be unconscious on bedside examination actually retain some degree of conscious awareness.² Failure to detect signs of conscious awareness may inappropriately influence clinical decision-making, lead to premature withdrawal of life-sustaining therapy and limit access to medical and rehabilitation services.

The Glasgow Coma Scale (GCS)³ and the Full Outline of UnResponsiveness (FOUR)⁴ score are brief and widely used assessment scales developed to detect changes in level of consciousness in acutely injured patients. However, neither scale was designed to quantify level of consciousness or differentiate the minimally conscious state (MCS) from the vegetative state/unresponsiveness wakefulness syndrome (VS/UWS), a critical diagnostic distinction for predicting subsequent outcome.¹ The Coma Recovery Scale-Revised (CRS-R)^{5, 6} enables this distinction, has the best performance characteristics among 13 DoC assessment scales,⁷ is recommended by current American⁸ and European⁹ clinical practice guidelines, and is a National Institute of Neurological Disorders and Stroke (NINDS) Stroke Common Data Element.¹⁰ CRS-R administration time, however, can extend up to 30 minutes, limiting its use in the ICU as patients often cannot tolerate being off of sedation for extended periods, and clinicians face significant time pressure. The Simplified Evaluation of CONsciousness DisorderS (SECONDS)¹¹ is a brief instrument developed to assess level of consciousness but is not validated in the ICU. To address these limitations, we developed the Coma Recovery Scale-Revised For Accelerated Standardized Testing (CRSR-FAST, clinicaltrials.gov NCT03549572), an abbreviated version of the full-length CRS-R. We tested the feasibility, concurrent validity, reliability, diagnostic sensitivity, and specificity of the CRSR-FAST in the ICU.

METHODS

Participants

We consecutively enrolled patients with acute traumatic brain injury (TBI) admitted to an ICU at a level 1 trauma center between August 2018 and December 2022. Patients were at least 18 years old, English-speaking, had a total GCS score ≤8 on at least 1 assessment within the first 48 hours of injury, were not consistently following commands (per clinical chart review conducted by a study team member not involved with data acquisition), and were ≤ 3 weeks post-injury. Of 776 patients screened, 127 met inclusion criteria and 56 were enrolled (see Supplementary Figure 1, Table 1). Eleven patients did not complete at least two study assessments, excluding them from further analyses. Legally-authorized representatives provided informed consent, consistent with the local Institutional Review Board-approved protocol.

Table 1:

Patient Demographics and Clinical Characteristics

	Included in Analysis (n=45)	Eligible but Not Enrolled or Did Not Complete at Least 2 Assessments (n=71)
Age, years, mean (SD)	44 (20)	55 (21)*
Sex, n (%) male	30 (67%)	46 (72%)
Mechanism of injury, n
Motor vehicle collision	26	29
Fall	16	32
Gunshot	3	1
Unknown	0	9
Total GCS Score in the ED
Min range	3/3T - 10	3/3T - 15
Max range	3/3T - 14	3/3T - 15
Days from injury to first study assessment, mean (SD)	8.3 (5.1)	NA
CRS-R Diagnosis, n
Coma/VS-UWS	18
MCS-	17
MCS+	9
eMCS	1
CRS-R Total score, mean (SD)	6.9 (5.2)

^*In addition to the 45 patients included in the analysis, 71 patients with TBI met all inclusion criteria at the time of screening but became ineligible prior to consent (n=24), refused participation (n=21), were positive for COVID (n=6), did not have a surrogate who could be reached (n=9), or consented but did not complete at least 2 assessments (n=11). Patients in the cohort that was eligible but not included in the study were older ( ttest p <0.005) but otherwise similar to the patients who were included. CRS-R: Coma Recovery Scale-Revised; ED: Emergency Department; eMCS: emergence from the minimally conscious state; GCS: Glasgow Coma Scale; SD: Standard Deviation; MCS: minimally conscious state without (MCS-) or with (MCS+) evidence of language function; VS-UWS: Vegetative State-Unresponsive Wakefulness Syndrome

Measures

The CRS-R consists of six subscales that are designed to detect behaviors associated with different levels of consciousness. Further details are available online.¹² The CRSR-FAST assesses only those CRS-R behaviors that differentiate conscious (MCS or emerged from MCS [eMCS]) from unconscious (coma or VS/UWS) patients. Multidisciplinary ICU clinicians, representing physicians, therapists and nurses involved in neurocriticial care and rehabilitation, participated in focus groups and surveys to establish a maximum acceptable duration of 10 minutes for an ICU assessment and to reach at least 80% consensus on the final core behaviors included in the CRSR-FAST: command-following, automatic motor responses, visual pursuit/fixation, localization to noxious stimulation, and intelligible speech (Figure 1 and Supplementary Materials provide the scale and administration manual).

Figure 1 :

The five core items of the CRSR-FAST are administered in order until a scoreable response is elicited or, for specified items, observed to occur spontaneously and meet the scoring criteria. The final rating is “conscious” if at least one of the core items is observed, or “not conscious” if none of these items are observed. There are three supplementary items, two of which can be used by the examiner to detect emergence from the minimally conscious state. Resting posture is recorded as the position of the extremities at rest, which can influence motor responses to noxious stimulation. Contextual factors that may confound assessment findings such as psychoactive medications, recent anesthesia, and intubation are recorded. Test Completion Codes are entered to establish the validity of the assessment and to record reasons for confounding or failure to complete the examination. The CRSR-FAST Administration and Scoring Manual is provided in the Supplementary Materials.

Medical complications and other factors confounding the examination are documented to convey the conditions under which the CRSR-FAST is administered, and Test Completion Codes are used to establish the validity of the assessment. To minimize administration time, the CRSR-FAST includes stop rules that are triggered when a behavior indicating consciousness is observed or elicited. CRSR-FAST results provide a binary diagnostic rating (i.e., conscious versus not conscious).

Procedure

Three trained examiners completed 4 study examinations (1 CRS-R and 3 CRSR-FAST assessments) over a maximum period of 48 hours. Examiners were masked to the clinical diagnosis of the patient and to the results of the other examinations (except for test-retest assessment). Test administration order and the assignment of raters to test condition (i.e., reliability or validity) was pseudo-randomized to prevent order effects. To optimize examination, all assessments were performed during periods when continuous infusion of sedative agents were held. Examiners recorded data on case forms that were transcribed into a REDCap¹³ database. Methodological details, the CRSR-FAST Administration Manual, and the CRSR-FAST scoring form are provided in Supplementary Materials.

Data Analysis

Examiners documented the start and end time of each assessment to determine the feasibility of the CRSR-FAST (goal: mean ≤10 minutes). We tested concurrent validity by comparing CRS-R and CRS-FAST diagnostic ratings (conscious [MCS or eMCS] vs. unconscious [coma or VS/UWS]), using the simple Kappa coefficient¹⁴ and CRSR-FAST test-retest and inter-rater reliability using Mak’s ρ (Statistical Analysis System [SAS v9.4]).¹⁵ We established an a priori threshold of ≥ 0.60 to indicate substantial validity and reliability.¹⁶

RESULTS

The 45 participants had the following characteristics: mean (SD) age = 44 (20.1) years, 67% male, full-length CRS-R total score = 6.9 (5.2), CRS-R range = 1–22, see Table 1 for sample characteristics and Supplementary Figure 2 for CRS-R score distributions. Participants completing the study protocol were younger than those who were eligible but not enrolled or enrolled but did not complete the protocol (Table 1, Supplementary Figure 1). The administration time for the CRSR-FAST was on average 32% of the time required for the full-length CRS-R (CRSR-FAST mean [SD] = 6.5 [3.3] minutes, CRS-R = 20.1 [7.2] minutes; including the 1-minute observation period, p<0.0001). Of 134 CRSR-FAST administrations, 81% required less than 10 minutes. All full-length CRS-R administrations required 10 minutes or more (see Supplementary Figure 3). Simple kappa for concurrent validity, Mak’s ρ for test-retest reliability and interrater reliability were 0.68, 0.76 and 0.91, respectively (Table 2), indicating substantial agreement in ratings.The sensitivity, specificity, and accuracy of the CRSR-FAST for detecting consciousness were 81%, 89%, and 84%, respectively (Table 3). Diagnostic disagreements between the CRS-R and CRSR-FAST, test validity, and the results of alternate methods for calculating validity are detailed in Supplementary Materials.

Table 2:

Psychometric properties of the CRSR-FAST

Concurrent Validity
	CRS-R		Simple kappa (SE)
CRSR-FAST	Conscious	Not Conscious	0.68 (0.10)
Conscious	22	2
Not Conscious	5	16
Test-retest Reliability
	CRSR-FAST		Mak’s ρ (SE)
CRSR-FAST	Conscious	Not Conscious	0.76 (0.10)
Conscious	21	1
Not Conscious	4	16
Inter-rater Reliability
	CRSR-FAST		Mak’s ρ (SE)
CRSR-FAST	Conscious	Not Conscious	0.91 (0.06)
Conscious	23	1
Not Conscious	1	20

CRS-R: Coma Recovery Scale Revised; CRSR-FAST: Coma Recovery Scale Revised For Accelerated Standardized Testing; SE Standard Error

Table 3:

CRSR-FAST Accuracy and Precision

Measure	Formula	Value	SE*
True positive rate $\left(TPR\right)$, Sensitivity	$\frac{TP}{TP+FN}$	0.81	0.07
True negative rate $\left(TNR\right)$, Specificity	$\frac{TN}{TN+FP}$	0.89	0.08
Accuracy	$\frac{TP+TN}{TP+TN+FP+FN}$	0.84	0.05
Positive predictive value $\left(PPV\right)$	$\frac{TP}{TP+FP}$	0.92	0.06
Negative predictive value $\left(NPV\right)$	$\frac{TN}{TN+FN}$	0.76	0.09
False positive rate $\left(FPR\right)$	$\frac{FP}{FP+TN}$	0.11	0.08
False discovery rate $\left(FDR\right)$	$\frac{FP}{FP+TP}$	0.08	0.06

^*Bootstrap method, 1000 samples

TP, True Positive: number of times both CRS-R full and CRSR-FAST ratings indicate “conscious”

TN, True Negative: number of times both CRS-R full and CRSR-FAST ratings indicate “not conscious”

FP, False Positive: number of times CRS-R full rating is “not conscious” and CRSR-Fast rating is “conscious”

FN, False Negative: number of times CRS-R full rating is “conscious” and CRSR-Fast rating is “not conscious”

SE, Standard Error

DISCUSSION

The reemergence of behaviors signaling consciousness marks a pivotal milestone in the early phase of recovery from severe TBI. An accurate and efficient method of detecting recovery of consciousness during the acute phase is essential as this milestone is a strong predictor of outcome and often influences decisions regarding intensity of treatment.¹ The results of this study indicate that the CRSR-FAST is a feasible, valid, reliable, and accurate method of detecting consciousness in patients with acute TBI in the ICU. The CRSR-FAST offers clinicians a standardized assessment tool that can be used in the acute setting to capture subtle but clinically important behavioral signs of consciousness that might otherwise be missed on standard bedside examination. The short duration of the CRSR-FAST facilitates serial assessment, which is more likely to capture fluctuating consciousness. These features may also improve diagnostic precision and patient selection and classification procedures in TBI clinical trials.

Five participants were deemed conscious on the full CRS-R but not conscious on the CRSR-FAST examination that was conducted to determine concurrent validity (see Supplementary Tables 6 and 12). In four of these five participants, behavioral evidence of consciousness was detected on one or both of the other two CRSR-FAST assessments. For the fifth participant, the full-length CRS-R detected only one subtle sign of consciousness (i.e., visual fixation) not observed on the CRSR-FAST, likely reflecting the difficulty of capturing optimal performance in a single assessment. These findings, coupled with evidence supporting the need for serial assessment to detect consciousness when it’s present,¹⁷ reinforce the notion that behavioral fluctuation is a hallmark feature of patients with severe brain injury ^{8, 17–21} and a likely contributor to the scoring discrepancies between the CRS-R and CRSR-FAST.

The CRSR-FAST is set apart from other behavioral measures used in the ICU by its ability to differentiate between MCS and VS/UWS, systematic approach to documenting factors that may confound the examination (e.g., sedation, peripheral pathologies), and comprehensive administration and scoring manual. CRSR-FAST administration requires approximately 6.5 minutes, which includes the 1-minute baseline observation. A scale of this duration can be administered repeatedly over the course of the day and, for patients receiving continuous sedation, provides an opportunity to acquire a standardized behavioral assessment during periods of sedation weaning. In the ICU setting, where patients are critically ill and multiple procedures may need to be carried out during short windows of opportunity, a rapid, repeatable, and reliable determination of consciousness may inform clinical decisions that result from an unexpected decline, improvement, or a response to treatment, and provide prognostic information. On the other hand, an assessment that requires 20 minutes or more is infeasible to administer during short sedation lifts and cannot be conducted serially. However, it is important to acknowledge that despite these advantages, given the significant level of prognostic uncertainty associated with recovery from TBI, the absence of early behavioral signs of consciousness during the acute period should not drive decisions regarding withdrawal of life-sustaining therapy.

Optimal clinical management and disposition planning across the continuum of recovery require a common frame of reference for clinical data sharing. The CRSR-FAST provides a standardized metric for longitudinal assessment that bridges the communication divide between acute care and rehabilitation clinicians and may facilitate caregiver education across the recovery continuum. The diagnostic and prognostic accuracy of the CRSR-FAST relative to other brief assessment measures, generalizability to non-TBI etiologies, potential application in low-resource environments, and scaling properties (i.e., ordinal vs. interval) should be assessed in future studies.

Summary for Social Media If Published:

1. Twitter Handle: @YelenaBodien

2. What is the current knowledge on the topic? There are no behavioral assessments designed and validated for differentiating patients in the intensive care unit who are conscious from those who are not conscious

3. What question did this study address? Is the Coma Recovery Scale-Revised For Accelerated Standardized Testing (CRSR-FAST), a tool that we designed for detecting consciousness after severe brain injury, valid, reliable, and feasible to administer in the intensive care unit?

4. What does this study add to our knowledge? The CRS-R FAST is a valid, practical, and short assessment of consciousness that can be used in the intensive care unit. An administration and scoring manual ensure standardized use of the measure.

5. How might this potentially impact on the practice of neurology? The CRSR-FAST provides a standardized metric for longitudinal assessment that bridges the communication divide between acute care and rehabilitation clinicians and may facilitate caregiver education across the recovery continuum. Serial assessment of consciousness with the CRSR-FAST may increase diagnostic and prognostic accuracy after severe brain injury.

Abbreviations

CRS-R

Coma Recovery Scale Revised

CRSR-FAST

Coma Recovery Scale Revised For Accelerated Standardized Testing

DoC

Disorders of Consciousness

GCS

Glasgow Coma Scale

ICU

Intensive Care Unit

MCS

Minimally Conscious State

eMCS

Emerged from Minimally Conscious State

FOUR

Full Outline of UnResponsiveness

SE

Standard Error

SECONDS

Simplified Evaluation of CONsciousness disorders

TBI

Traumatic Brain Injury

UWS

Unresponsive Wakefulness Syndrome

VS

Vegetative State