Limits of the Glasgow Coma Scale When Assessing for Sepsis in Allogeneic Hematopoietic Cell Transplant Recipients

Abstract

Background:

The well-documented association between acute mental status changes and sepsis development and progression makes acute mental status an attractive factor for sepsis screening tools. However, the usefulness of acute mental status within these criteria is limited to the frequency and accuracy of its capture. The Glasgow Coma Scale (GCS) score—the acute mental status indicator in many clinical sepsis criteria—is infrequently captured among allogeneic hematopoietic cell transplant recipients with suspected infections, and its ability to serve as an indicator of acute mental status among this high-risk population is unknown.

Objective:

We evaluated the GCS score as an indicator of acute mental status during the 24 hours after suspected infection on set among allogeneic hematopoietic cell transplant recipients.

Methods:

Using data from the first 100 days posttransplant for patients transplanted at a single center between September 2010 and July 2017, we evaluated the GCS score as an indicator of documented acute mental status during the 24 hours after suspected infection onset. From all inpatients with suspected infections, we randomly selected a cohort based on previously published estimates of GCS score frequency among hematopoietic cell transplant recipients with suspected infections and performed chart review to ascertain documentation of clinical acute mental status within the 24 hours after suspected infection onset.

Results:

A total of 773 patients had ≥1 suspected infections and experienced 1,655 suspected infections during follow-up— 625 of which had an accompanying GCS score. Among the randomly selected cohort of 100 persons with suspected infection, 28 were accompanied with documented acute mental status, including 18 without a recorded GCS. In relation to documented acute mental status, the GCS had moderate to high sensitivity and high specificity.

Discussion:

These data indicate that, among allogeneic hematopoietic cell transplant recipients with suspected infections, the GCS scores are infrequently collected and have a moderate sensitivity. If sepsis screening tools inclusive of acute mental status changesare to beused, nursing teams needto increase measurementof GCS scores amonghigh sepsisrisk patients or identify a standard alternative indicator.

After transplantation, allogeneic hematopoietic stem cell transplant (HCT) recipients are at increased risk of sepsis and sepsis-related complications (Kumar et al., 2015). Because rapid initiation of broad-spectrum antibiotics has been shown to reduce the risk of mortality among patients with sepsis, early and accurate detection is crucial (Seymour et al., 2017). However, sepsis is challenging to diagnose, and in the absence of a diagnostic test, clinical scores are recommended for sepsis screening among patients with suspected infections (Singer et al., 2016). Such screening tools include factors known to be associated with sepsis in general patient populations (Giamarellos-Bourboulis et al., 2012; Royal College of Physicians [RCP], 2012, 2017; Singer et al., 2016).

Altered mental status (AMS), a commonly recognized sign of sepsis, is a factor in multiple sepsis screening scores, including the Sequential Organ Failure Assessment, the Acute Physiologic Assessment and Chronic Health Evaluation II, and the National Early Warning Scores (NEWS/NEWS2; Giamarellos-Bourboulis et al., 2012; RCP, 2012, 2017; Singer et al., 2016). Within such scores, AMS is most commonly assessed using the Glasgow Coma Scale (GCS) score, a 15-component (four visual, five verbal, and six motor) score built to assess consciousness among trauma patients.

Since its development in 1974, the GCS has been shown to be a good indicator of acute brain damage in trauma patients and associated with 30-day mortality (a common sepsis end point) among numerous patient populations (Knox et al., 2014; Reith et al., 2016; Safatli et al., 2016; Teasdale & Jennett, 1974; Wang et al., 2014). However, evidence of an association between GCS score and short-term mortality among patients with infections is mixed, and if GCS is a reliable indicator of cognition and awareness (including AMS) and thus a reliable factor in sepsis screening tools among high-risk, immunocompromised patients, such as allogeneic HCT recipients with infections, is unknown (Baršić et al., 1996; Gaini et al., 2019; Seymour et al., 2019).

What has become increasingly recognized is that despite wide use of GCS for nursing assessment in the intensive care, trauma, and neurological units, GCS assessment is infrequently used in acute care oncology and medical/surgical units (Lind et al., 2021). This infrequent capture likely limits its validity as an indicator of AMS in allogeneic HCT recipients with sepsis. Here, we set out to assess if GCS scores collected around the time of infection onset aligned with documentation of AMS in the medical charts and if the absence of a GCS score meant an absence of documented AMS among allogeneic HCT recipients.

METHODS

Study Design and Population

We conducted a retrospective study among a sample of previously collected adult allogeneic HCT recipients with inpatient suspected infections (SIs) who were transplanted between September 2010 and July 2017 at the Fred Hutchinson Cancer Research Center/Seattle Cancer Care Alliance. We assessed data from the first 100 days following transplantation utilizing established clinical databases and electronic medical record (EMR) chart review. In alignment with Seymour et al. (2016), we defined SI as the addition of a new antibiotic and bodily fluid culture within a specific time epoch (Seymour et al., 2016). The study was approved by the Fred Hutchinson Cancer Research Center Institutional Review Board.

Study Cohort and Chart Review

Using data from the 24 hours after SI onset (culture collection), we randomly selected a cohort of 30 inpatient SIs with and 70 inpatient SIs without GCS scores. We selected this 70/30 split based on previously published data showing that 26% of SIs were accompanied with GCS scores among a similar cohort (Lind et al., 2021). For the 30 SIs with an accompanying GCS score, the lowest GCS score recorded during the 24 hours after SI onset was retained.

For each SI, we reviewed EMRs for documentation of AMS within the 24 hours after SI onset. We defined documented AMS as the presence of at least one of the following: clinical note listing mental status change, symptoms observed during neurological general assessment, or lack of orientation/confusion documented in nursing orientation assessment. Clinical notes were examined using the EMR chart search tool and supplemented by a focused review of neurological general and orientation-level assessments. Examples of mental status notes we considered indicative of documented AMS were “mental status continued to decline to point that [patient] became unresponsive” and “A/O x4, has been intermittently confused.” The specific list of accepted clinical notes and included health states were developed by a group of nurses and clinicians who perform and chart mental health evaluations regularly. To ensure the consistency of our review, the extraction was performed by limited personnel trained by our group of nurses. Data were collected in REDCap (Research Electronic Data Capture) and analyzed in R 3.6.2 (R Core Team, 2013).

Antibiotic, Culture, andGCS Center Standard Practices

A detailed description of our center’s antibiotic prophylaxis and culture collection practices is published elsewhere (Lind et al., 2021). Briefly, levofloxacin was used as first-line neutropenic prophylaxis. Although center-specific guidelines recommended the collection of cultures in patients with fevers and patients receiving high-dose steroids, all cultures were collected at the discretion of the treating team (Lind et al., 2021). GCS assessment is neither a standard at our center nor is it open by default in our center’s EMR.

Statistical Analysis

We examined GCS score as our primary indicator of AMS and, in agreement with the quick Sequential Organ Failure Assessment score, we defined an abnormal GCS score as a score below 15 (Seymour et al., 2016). We estimated the value of nursing collected GCS as an indicator for AMS among allogeneic HCT recipients with SIs using descriptive and predictive value metrics. The agreement between GCS score and documentation of AMS was summarized by count and percentages, and we estimated the sensitivity, specificity, and positive (PPV) and negative (NPV) predictive values of GCS (<15) in relation to documented AMS (Seymour et al., 2016). We compared the median number of mental status evaluations among individuals with and without recorded GCS scores and tested for an association between GCS score and presence of AMS using Fisher’s exact test. To see if alternative AMS indicators would lead to better AMS capture, we summarized the measurement frequency of the Alert, Voice, Pain, Unresponsive (AVPU) score using counts and percentages.

RESULTS

A total of 1,170 adult allogeneic HCT recipients received a transplant between September 1, 2010, and July 31, 2017, of which 773 (66%) experienced at least one inpatient SI. Of the 1,655 inpatient SIs experienced by the recipients, 625 (37.8%) were accompanied with a GCS score, and 80 (4.8%) were accompanied with an AVPU score. From the 625 SIs with accompanying GCS scores and 1,030 without GCS scores, we randomly selected 30 and 70 SIs, respectively (Figure 1).

FIGURE 1.

Flowchart of hematopoietic cell transplant (HCT) recipient transplanted between September 2010 and July 2017 and their suspected infections. We randomly selected 30 suspected infections with accompanying Glasgow Coma Scale (GCS) scores and 70 without GCS scores for evaluation. A total of six altered mental status events were properly captured with a GCS score of <15 (the commonly used threshold). AMS = altered mental status. This figure is available in color online (www.nursingresearch.com).

The selected SIs came from 98 individuals, most of whom were male (63.3%) and Caucasian (83.5%). Patients with recorded GCS scores tended to be younger (median age with GCS = 49.5 vs. median age without GCS = 55.2 years) and more likely to have received stem cells from an unrelated donor (75.9% with GCS vs. 65.2% without GCS) than patients without recorded scores (Table 1).

TABLE 1.

Demographic and Transplant Characteristics of Examined Allogeneic Hematopoietic Cell Transplant Recipients With Suspected Infections

	Full sample (N = 98)	With GCS (n = 29)	Without GCS (n = 69)
Age at transplant (median, IQR)	53.99 (39.92, 62.60)	49.52 (41.56, 60.63)	55.20 (37.45, 63.40)
Male (count, %)	62 (63.3)	19 (65.5)	43 (62.3)
Race (count, %)
American Indian or Alaskan Native	1 (1.0)	0 (0.0)	1 (1.4)
Asian	10 (10.2)	3 (10.3)	7 (10.1)
Black or African American	3 (3.1)	2 (6.9)	1 (1.4)
Native Hawaiian or other Pacific Islander	1 (1.0)	0 (0.0)	1 (1.4)
Unknown	1 (1.0)	0 (0.0)	1 (1.4)
White	82 (83.7)	24 (82.8)	58 (84.1)
Donor cell type (count, %)
Bone marrow	10 (10.2)	4 (13.8)	6 (8.7)
Bone marrow, peripheral blood stem cell	1 (1.0)	0 (0.0)	1 (1.4)
Umbilical cord blood	15 (15.3)	5 (17.2)	10 (14.5)
Peripheral blood stem cell	72 (73.5)	20 (69.0)	52 (75.4)
Unrelated donor (count, %)	67 (68.4)	22 (75.9)	45 (65.2)

Note. GCS = Glasgow Coma Scale; IQR = interquartile range.

Twenty-eight of the 100 SIs (28.0%) were accompanied with documented AMS in the EMR; more than half (18, 64.3%) were without a recorded GCS score. Among SIs with GCS scores (n = 30), 6 of the 10 with documented AMS had a GCS score of <15 (consistent with AMS) and 18 of the 20 without documented AMS had a GCS score of 15 (indicating no AMS; Figure 1). This corresponds to a sensitivity of 60.0% (95% CI [26.2, 87.8%]), a specificity of 90.0% (CI [68.3, 98.8]), a PPV of 75.0% (CI [34.9, 96.8]), and an NPV of 81.8% (CI [59.7, 94.8]; Table 2). The GCS scores for the two patients without documented AMS but GCS scores of <15 were 3 and 9 (Figure 2).

TABLE 2.

Predictive Value of Glasgow Coma Scale (GCS) Among Allogeneic Hematopoietic Cell Transplant Recipients With Suspected Infections

	Point estimate	95% Confidence interval
Predictive accuracy
Sensitivitya	60.0%	[26.2, 87.8%]
Specificitya	90.0%	[68.3, 98.8%]
Positive predictive value	75.0%	[34.9, 96.8%]
Negative predictive value	81.8%	[59.7, 94.8%]
Odds ratio
Association between recorded GCS and documented AMSa	12.0	[1.5, 165.2]
Association between documented AMS presence of GCS measurementb	1.4	[0.5, 4.0]

Note. AMS = altered mental status.

^aAmong suspected infections with recorded GCS scores.

^bAmong all suspected infections.

FIGURE 2.

Stacked bar chart of Glasgow Coma Scale (GCS) score counts by presence or absence of documented altered mental status (AMS) among allogeneic hematopoietic cell transplant recipients with suspected events and accompanying GCS scores. A GCS score of 15 was the most commonly observed score regardless of documentation of AMS. This figure is available in color online (www.nursingresearch.com).

Among SIs with GCS scores, patients with GCS score of <15 were more likely to have documented AMS than patients with a score of 15 (odds ratio = 12.0, p = .007). However, among all 100 examined SIs, patients with GCS scores were no more likely than patients without GCS scores to have documented AMS (odds ratio = 1.4, p = .472).

DISCUSSION

In this retrospective review, we examined the validity of GCS as an indicator of clinical AMS among allogeneic HCT recipients with SIs. Despite being an important factor in many sepsis screening scores, we found that GCS was infrequently captured and that a score of <15 missed 40% of documented AMS events (sensitivity of 60%). Although we observed GCS to have high specificity, PPV, and NPV, we found that the absence of a GCS score was not indicative of an absence of documented AMS. In fact, only 35.7% of patients with EMR documented AMS had a GCS score.

Though the infrequency of recorded GCS scores was not surprising given that GCS assessment is not required nor open in the EMR by default at our center, the absence of a GCS score in over 60% of patients with clinically documented AMS greatly reduces the validity of sepsis screening tools inclusive of GCS. However, this may be less concerning in current practice because GCS documentation is unlikely a driver of treatment decisions, it poses significant problems for the accurate use, evaluation, standardization, and development of sepsis screening tools.

We posit that enhanced collection of GCS score by nursing staff caring for allogeneic HCT recipients and similarly high-risk patients is needed for reliable use of many sepsis screening tools. Our estimates suggest that increased GCS recording may lead to adequate capture of AMS (sensitivity may be as high as 88%) while not leading to excessive false capture (specificity = 90%, PPV = 75%). However, GCS scores cannot be automatically calculated from vital signs, and scoring requires a lengthy bedside nursing assessment.

The use of other existing or new scores poses potential alternatives to increased GCS capture. For example, NEWS and NEWS2 include the AVPU, a four-component score that can be quickly and easily assessed from the bedside. However, the infrequency of AVPU capture (<5%) among our population prevented our assessment of it as an indicator of AMS (RCP, 2012, 2017; Romanelli & Farrell, 2019). New scores have been shown to increase AMS capture and pose additional alternative routes to increased GCS capture (Shalabi et al., 2018).

Our study had several strengths. First, it was conducted at a large transplant center. Second, using intensive chart review, we were able to evaluate GCS as a predictor of AMS among a novel population, allogeneic HCT recipients. However, our study had limitations. First, it was conducted in a single center and may reflect center-specific GCS collection practices. Second, we had a limited sample size (n = 100). Finally, we relied on clinical medical records, which may not fully reflect true mental status changes within our population.

Conclusion

We found GCS, as it was collected within our center, to be a poor indicator of documented AMS. Not only did it havea moderate sensitivity among patients with recorded scores, but its absence was also not reflective of an absence of AMS. If AMS continues to be used in sepsis screening tools, more frequent capture of GCS (or alternative score) by nurses treating high-risk patients is needed.