Adherence to Guidelines in Adult Patients with Traumatic Brain Injury: A Living Systematic Review

Abstract

Guidelines aim to improve the quality of medical care and reduce treatment variation. The extent to which guidelines are adhered to in the field of traumatic brain injury (TBI) is unknown. The objectives of this systematic review were to (1) quantify adherence to guidelines in adult patients with TBI, (2) examine factors influencing adherence, and (3) study associations of adherence to clinical guidelines and outcome. We searched EMBASE, MEDLINE, Cochrane Central, PubMed, Web of Science, PsycINFO, SCOPUS, CINAHL, and grey literature in October 2014. We included studies of evidence-based (inter)national guidelines that examined the acute treatment of adult patients with TBI. Methodological quality was assessed using the Research Triangle Institute item bank and Quality in Prognostic Studies Risk of Bias Assessment Instrument. Twenty-two retrospective and prospective observational cohort studies, reported in 25 publications, were included, describing adherence to 13 guideline recommendations. Guideline adherence varied considerably between studies (range 18–100%) and was higher in guideline recommendations based on strong evidence compared with those based on lower evidence, and lower in recommendations of relatively more invasive procedures such as craniotomy. A number of patient-related factors, including age, Glasgow Coma Scale, and intracranial pathology, were associated with greater guideline adherence. Guideline adherence to Brain Trauma Foundation guidelines seemed to be associated with lower mortality. Guideline adherence in TBI is suboptimal, and wide variation exists between studies. Guideline adherence may be improved through the development of strong evidence for guidelines. Further research specifying hospital and management characteristics that explain variation in guideline adherence is warranted.

Editor's Note: This article is published as a Living Systematic Review. All Living Systematic Reviews will be updated at approximately three-six month intervals, with these updates published as supplementary material in the online version of the Journal of Neurotrauma (see Update 4).

Introduction

Traumatic brain injury (TBI) is a major public health concern affecting approximately 150–300 per 100,000 persons annually in Europe.¹ The World Health Organization has predicted that TBI will be one of the leading causes of death and disability worldwide by the year 2020.²

The care for patients with TBI is often complex and multidisciplinary. Guidelines, protocols, and care pathways have been developed to improve quality of care, to reduce variation in practice, and to ensure that evidence-based care is optimally implemented.³

A 2013 systematic review⁴ found that the use of protocols in the management of severe TBI in the intensive care unit (ICU) led to improved patient outcomes. The findings, however, were based on observational studies that did not report on adherence rates. Without an understanding of adherence rates, the improved outcomes stated in the review cannot be directly attributed to the use of protocols.

Guideline adherence can be defined as the proportion of patients treated according to a guideline recommendation, which often represents evidence-based or best practice care. Previous studies have found that guideline adherence in medicine is generally low^5–7 and varies widely across centers,^7,8 medical condition,⁹ types of guideline,^10,11 and time period.^8,10 As a result, many patients do not receive evidence-based care, while others receive unnecessary care that may even be harmful.⁵ To date, no systematic review of the literature about guideline adherence in TBI has been conducted.

The aim of this systematic review was to provide a comprehensive overview of professionals' adherence to guidelines in adult patients with TBI. The objectives were threefold:

• 1.To quantify adherence to guidelines in adult patients with TBI.
• 2.To explore factors influencing adherence to TBI guidelines in those studies reporting on adherence.
• 3.To examine the association between adherence to guidelines and outcome in patients with TBI in those studies reporting on adherence.

Methods

This review was conducted and reported in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement.¹² Details of the protocol for this systematic review were registered on PROSPERO (registration number CRD42014012863) and can be accessed at http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42014012863.

This review is being prepared as a “living systematic review” as part of the CENTER-TBI project¹³ (www.center-tbi.eu). A living systematic review is a high quality, up-to-date, online summary of health research that is updated as new research becomes available.¹⁴ This means that the searches will be rerun frequently and new studies will be incorporated into the review, with revisions to recommendations as appropriate. We will seek to publish regular updates.

Information sources

A comprehensive literature search was conducted on October 22, 2014. Search strategies were developed in consultation with search experts using a combination of subheadings and text words (Supplement A; see online supplementary material at ftp.liebertpub.com). The databases EMBASE, MEDLINE (via Ovid SP), Cochrane Central, PubMed as supplied by publisher, Web of Science, PsycINFO, SCOPUS, and CINAHL were searched. In addition, grey literature was examined via Google Scholar, opengrey.eu, and dissertation databases (openthesis.org, dissertation.com). Reference lists and citation indices of the included articles and relevant reviews were inspected to identify additional relevant citations. All selected studies were downloaded to the reference management database Endnote X5¹⁵ and duplicates were removed. We restricted the search to original articles published in English. There was no date restriction.

Inclusion and exclusion criteria and study selection

We used the following inclusion and exclusion criteria to select studies:

Study designs: We included retrospective and prospective cohort studies, cross-sectional studies, time series, and controlled clinical trials. Reviews, qualitative studies, case reports, and editorials were excluded.

Participants: Studies were included if they were conducted in adult patients with suspected or confirmed TBI. Studies including a mixed population (e.g., all trauma patients) were only included if they presented their results for patients with TBI separately. Studies solely about children were excluded because other factors, such as radiation, might play a role in guideline adherence in this group. If studies presented results for children and adults separately, only the information on adults was extracted.

Guidelines: Evidence-based international and national clinical TBI guidelines were included. Evidence-based guidelines were defined as guidelines for which evidence was found in quantitative research. We included studies analyzing adherence to a complete guideline or protocol as well as studies analyzing adherence to one or more single guideline recommendations. Local and regional guidelines, and guidelines based on expert opinion were excluded. Studies were further excluded if they assessed adherence to guidelines not published or implemented during the study period.

Adherence: Adherence or compliance was conceptualized as the percentage of patients who were treated according to a guideline, a subset of guidelines, or an individual recommendation of a guideline. This definition was chosen to enable comparison of adherence to different guidelines or guideline recommendations. Studies using self-reported adherence were excluded because of the risk of overestimation.¹⁶

Setting: Studies were included if they examined the acute curative care of patients with TBI, in the pre-hospital setting, emergency department (ED), hospital ward care, and ICU.

The first review author (MC) screened all titles and abstracts and deleted obviously irrelevant citations. After the initial selection, two independent reviewers (MC and ACS) screened the remaining citations on title and abstract and obtained those selected in full text. Results were compared, and any disagreement was resolved by discussion or consulting a third author (SP). The search process was documented according to the PRISMA flowchart.¹²

Data collection and assessment of methodological quality

Two reviewers (MC and ACS) independently extracted data and assessed the risk of bias of included studies. Any discrepancies were resolved by discussion or consulting a third author (SP).

A data extraction form was developed based on the Effective Practice and Organisation of Care Cochrane Review Group (EPOC) data collection checklist,¹⁷ and the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement.¹⁸ In addition, topic-relevant criteria about guidelines, adherence, and influencing factors were extracted. Guideline recommendations were classified as “strong” or “weak/moderate” recommendations. Strong recommendations were defined as being based on good quality randomized controlled trials (RCTs). Weak or moderate recommendations were defined as being based on moderate- or poor-quality RCTs, cohort studies, case control studies, or case series.

We developed three risk of bias forms to rate the risk of bias in quantifying adherence (Objective 1), exploring factors influencing adherence (Objective 2), and examining the association between adherence and outcome (Objective 3). Risk of bias forms were based on items from the Research Triangle Institute (RTI) Item Bank for observational studies^{19, 20} (Objectives 1 and 3) and the Quality in Prognostic Studies (QUIPS) risk of bias tool²¹ (Objective 2). The risk of bias was assessed for each of the three objectives separately because different risks are relevant in the three objectives. Moreover, it was possible that studies assessing more than one review objective had a low risk of bias for one objective but a high risk for another.

Risk of bias items were subdivided into six categories for every objective: selection bias/confounding, performance bias, attrition bias, detection bias, reporting bias, and information bias^19,22 (see Supplement B; see online supplementary material at ftp.liebertpub.com). For every category, individual items were scored as high, low, or unclear risk of bias.

If at least one item in a bias category was scored as high, the risk of bias within this category was scored as moderate risk. If at least 50% of the items in a bias category were scored as high, the risk of bias category was scored as high risk. Every study received a total risk of bias score for every objective that was equal to the highest score obtained in all risk of bias criteria.

Risk of bias was presented with a table stratified by objective. Attrition and detection bias were not reported for Objective 1 because these were considered irrelevant for the percentage adherence obtained. We accounted for risk of bias by narratively describing studies with a low (none of the criteria was rated as high risk of bias) and moderate (<50% of the criteria was rated as high risk of bias) risk of bias separately for the three objectives.

To enhance interrater reliability, data extraction and risk of bias forms were pilot-tested on three studies that were likely to be included in the review. Interrater reliability was assessed by calculating concordance rates between the two independent reviewers in data screening, data extraction and risk of bias assessment.

Data synthesis

Because of heterogeneity in settings, guidelines, populations, statistical methods, and outcomes, meta-analytic techniques were not used. Instead, we conducted a narrative synthesis of results stratified by objective.

For every guideline recommendation that was examined in at least two studies, mean guideline adherence was calculated by adding up the total number of patients treated according to the guideline recommendation and subsequently dividing them by the total number of patients eligible for the guideline. In addition, the percentage adherence was presented separately for strong and moderate/weak recommendations. We also compared the differences in percentage adherence for relatively more invasive (e.g., intracranial pressure monitoring and intracranial operation) and less invasive (e.g., computed tomography [CT] scanning and antiseizure prophylaxis) procedures separately. A total percentage adherence was not calculated, because there was considerable variation in guidelines and patient severity.

An overview of factors influencing adherence was conducted. We examined whether associations between predictive factors and adherence were positively or negatively directed and whether they were statistically significant (p < 0.05). In addition, we conducted an overview of the association between adherence and outcome and reported whether associations were positively or negatively directed and statistically significant.

All eligible studies were used for Objective 1. Those that also reported factors influencing adherence and/or outcome were further analyzed for Objective 2 and/or Objective 3. There were no further specific inclusion criteria for these objectives. All results are presented before and after the exclusion of studies that were judged as high risk of bias.

Treatment of studies with multiple publications

Multiple publications refer to the situation where more than one article has been written based on the same dataset.²³ Multiple publications assessing the same guideline in an overlapping time period and setting were dealt with by extracting information from the study that could be used for the most study objectives. If the number of objectives was similar across studies with multiple publications, the article that included the largest number of patients was chosen. Articles from the same dataset that assessed different guidelines or that were conducted during a different study period or in a different setting were analyzed separately.

Results

Study selection

A total of 1903 citations were identified through the extensive search strategy (Fig. 1). After removing duplicates, 912 were screened on citation and 518 obviously irrelevant records (determined on title) were removed. We screened 394 citations on title and abstract and excluded 310. We obtained 84 citations in full text of which 62 were excluded. Three additional citations were found via reference lists and citation indices. For an overview of related studies excluded at the full text stage, see Supplement C; see online supplementary material at ftp.liebertpub.com.

FIG. 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flowchart of the selection process. Reasons for exclusion full text: Study design: the study was no prospective or retrospective cohort study, randomized controlled trial, clinical trial, cross-sectional study, or time series; Guideline: the study did not describe a guideline, the guideline was local or not evidence-based, the guideline was not implemented or disseminated before the study period; Adherence: the study did not measure adherence per patient, adherence was self-reported; traumatic brain injury (TBI): the study was not about patients with TBI; Setting: the study was not conducted during the hospital and pre-hospital setting; Language: the study was not published in English; Solely about children: the study did not include adults. Adapted from: Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6: e1000097.

The concordance rates between the two independent reviewers were generally high in screening of title and abstract (91%), screening of full text (81%), and data extraction (93%).

Study characteristics

We included 22 studies, reported in 25 publications (Table 1). Three articles were removed from the analyses because of multiple publications.^10,24,25 Two more studies were based on the same dataset,^26,27 but the study describing the least number of objectives²⁶ was still included for extracting the amount of adherence to another guideline recommendation.

Table 1.

Characteristics of the Included Studies (n = 22)

Study ID	Objective	Study design & setting	Patients	Guideline and topic	Strength of recommendation*	Adherence operationalization	Adherence (% (n adherent /n total))
Alali (2013)	1,2,3	Retrospective cohort (US, Canada) in 155 centers	Severe TBI, age ≥16	BTF (2007) – ICP monitoring	M/W	ICP monitor inserted	18% (1874/10628)
Andriessen (2011)	1A	Same dataset as Biersteker (2012)	Severe TBI, age ≥16	BTF (2007) – Pre-hospital intubation	M/W	Pre-hospital intubation performed	69% (234/339)
Biersteker (2012)	1,2,3	Prospective cohort (The Netherlands) in 5 centers	Severe TBI, age ≥16, intracranial pathology or 2/3 criteria: age >40, ED motor score ≤3 or systolic blood pressure <90 mmHg	BTF (2007) – ICP monitoring	M/W	ICP monitor inserted	46% (123/265)
Bulger (2002)	1,3	Retrospective cohort (US) in 33 centers	Severe TBI, multitrauma, age ≥18. ICP monitoring: abnormal head CT	BTF (1995) – Pre-hospital intubation and ICP monitoring	M/W	Pre-hospital intubation performed	43% (79/182)
						ICP monitor inserted	58% (105/182)
Fakhry (2004)	1,3	Prospective cohort with historical control group (US)	Severe TBI, age >14	BTF (1995) – ICU management of severe TBI patientsB	M/W	Following ICU protocol***	76% (466/611)
Farahvar (2012) Gerber (2013)	1,2,3	Retrospective analysis of prospectively collected database (US) in 22 centers	Severe TBI, intracranial pathology or 2/3 criteria: age >40, hypotension or GCS motor score ≤3; ICP lowering treatment on first 2 days	BTF (2000) – ICP monitoring	M/W	ICP monitor inserted	83% (1084/1307)
Franschman (2012) Franschman (2009)	1	Retrospective cohort (The Netherlands) in 3 centers	Severe, CT scan confirmed TBI, age >10	BTF (2000) – prehospital intubation	M/W	Pre-hospital intubation performed	88% (NR/372)
Griesdale (2010)	1,2,3	Retrospective cohort (Canada)	Severe TBI, intracranial pathology	BTF (2000) – ICP monitoring	M/W	ICP monitoring with EVD inserted	61% (98/161)C
Harr (2011) Heskestad (2012)	1,2	Retrospective cohort (Norway)	ICD-10 diagnosis head injury, age ≥15 years	Scandinavian guidelines (2000) – CT scanning & hospital admission	M/W	CT scanning and hospital admission according to algorithm	61% (520/860)
Härtl (2006)	1,2,3	Same dataset as Farahvar (2012)	Severe TBI	BTF (2000) – Direct transport	M/W	Direct transfer to trauma center	77% (864/1118)
Haydon (2013)	1	Retrospective cohort (UK)	Head injury, age ≥16, received a CT scan	NICE CG 56 (2007) – CT scanning	S	Documentation of ≥1 CT scan requirements	84% (129/153)
						Performing CT scan ≤1 h of request for all but three of indications	86% (93/108)
						Performing CT scan ≤8 h in three other risk factors	100% (21/21)
Heskestad (2008)	1,2	Prospective cohort (Norway)	ICD-10 diagnosis head injury	Scandinavian guidelines (2000) – CT scanning & hospital admission	M/W	CT scanning and hospital admission according to algorithm	51% (259/508)
Mauritz (2008)	1,2,3	Prospective cohort in 13 tertiary care centers (Austria, Slovakia, Bosnia, and Macedonia)	Severe TBI	BTF (1995) – Pre-hospital intubation, direct transport, steroid use	M/W	Following BTF guidelines for:
					M/W	Pre-hospital intubation	58% (673/1172)
					M/W	Direct transfer	72 (534/746)
					S	Steroids not used	83% (468/564)
Mooney (2011)	1	Retrospective cohort (UK) in 2 centers	Head injury	NICE CG56 (2007) – CT scanning	S	CT performed according to criteria	97 (741/762)
Prowse (2009)	1	Retrospective cohort (UK)	Isolated head injury	NICE CG56 (2007) – CT scanning	S	NICE criteria reported in patients who had a CT scan performed	70% (23/33)
Ravindran (2007)	1	Retrospective cohort (UK)	Head injury	NICE CG4 (2003) – CT scanning	S	NICE criteria reported in patients who had a CT scan performed out of hours	100% (27/27)
Rognas (2013)	1	Prospective cohort (Denmark)	Severe TBI	BTF (2007) and Scandinavian Guidelines on pre-hospital management of TBI (2008) – pre-hospital intubation	M/W	Pre-hospital intubation performed	93% (50/54)
Rusnak (2007)	1,3	Prospective cohort (Austria) in 5 centers	Severe TBI	BTF (1995) – Various recommendations		Following BTF guidelines (see Rusnak (2007) Table 2) for:
					M/W	Resuscitation of BP & O2	79% (217/274)
					M/W	Indications for ICP monitoring	68% (283/415)
					M/W	Hyperventilation	92% (363/393)
					M/W	Barbiturates	83% (269/326)
					S	Steroids	89% (362/409)
					M/W	Antiseizure prophylaxis	89% (360/407)
Shafi (2014)	1,2,3	Retrospective cohort (US) in 11 centers	Severe TBI patients, age <99, intracranial pathology	BTF (2007) – various recommendations		Following BTF guidelines (see Shafi (2014) Table 1):
					M/W	Endotracheal intubation	92% (1890/2056)
					M/W	Resuscitation	75% (48/64)
					M/W	ICP monitoring	52% (818/1569)
					M/W	ICP directed therapy	76% (742/978)
Shafi (2014b)	1	Retrospective cohort (US) in 5 hospitals	Severe TBI, age ≥16	BTF – various recommendations	M/W	Pre-hospital intubation performed	94% (468/497)
					M/W	Intracranial pressure monitoring in trauma patients with a GCS ≤8 and intracranial bleed on head CT and endotracheal intubation	39% (100/257)
					M/W	Craniotomy in patients with GCS ≤8 and intracranial bleed on head CT	20% (66/326)
Shravat (2006)	1	Retrospective cohort (UK)	Head injury	NICE CG56 (2007) – CT scanning	S	Whether CT had been requested within existing NICE criteria	100% (472/472)
Talving (2013)	1,2,3	Prospective cohort (US)	Severe TBI, age >18, meeting BTF criteria for ICP monitoring	BTF (2007) – ICP monitoring	M/W	ICP monitor inserted	47% (101/216)

^*S = strong recommendation, the guideline recommendation was based on good quality randomized controlled trials; M/W = strong/weak recommendation, the guideline recommendation was based on lower level evidence.

^A = Because of multiple publications, only the amount of intubation adherence is assessed from Andriessen. For ICP monitoring, see Biersteker (2012).

^B = See appendix Fakhry (2004) for the ICU protocol.

^C = Authors stated that 98 of 171 patients got an ICP monitor placed. They also stated that 10 of the patients who got no ICP monitor placed did not had an indication. We therefore recalculated the percentage adherence without those 10 patients.

TBI, traumatic brain injury; NR, not reported; AIS, Abbreviated Injury Scale; BTF, Brain Trauma Foundation; ICP, intracranial pressure; LOS, length of stay; GCS, Glasgow Coma Scale; ED, emergency department; GOSE, Glasgow Outcome Scale Extended; US, United States of America; CT, computed tomography; ICU, intensive care unit; RLAS = Rancho Los Amigos Scale; ISS, Injury Severity Score; EVD, external ventricular drainage; ICD, International Classification of Diseases; HI, head injury; NICE, National Institute of Health and Clinical Excellence; HIC, high income country; UMIC, upper middle income country; LMIC, lower middle income country; BP, blood pressure; O₂, oxygen; RBC, red blood cell; SBP, systolic blood pressure; CPP, cerebral perfusion pressure.

Definition: severe TBI = GCS <9.

All included studies used an observational cohort design with 14 being retrospective^28–41 and 8 being prospective.^{26,27,42–47} Twelve studies described multicenter studies^{26–31,34,36,40,41,44,46} with a median of eight (range 2–155) hospitals included. All studies were conducted in North America (n = 9) or Europe (n = 13) and were published between 2002 and 2014. Six of the included studies^{33,40,41,43,44,46} examined adherence to more than one guideline recommendation (mean number of guideline recommendations in studies describing more than one guideline recommendation: 3.6; range 2–6). The sample size in the included studies ranged from n = 27³⁸ to n = 10,628²⁸ patients.

Adherence to a total of 13 guideline recommendations was assessed, including those from the Brain Trauma Foundation (BTF),⁴⁸ National Institute of Health and Clinical Excellence (NICE),⁴⁹ and Scandinavian guidelines for the initial management of minimal, mild, and moderate head injury.⁵⁰ The most frequently studied guideline recommendation was the BTF guideline for incracranial pressure (ICP) monitoring (n = 9). Other guidelines that were studied in more than one study were the NICE guidelines for CT scanning (n = 5), the BTF guidelines for pre-hospital intubation (n = 7), transport (n = 2), steroids (n = 2), and resuscitation (n = 2), and the Scandinavian guidelines for CT scanning and hospital admission (n = 2).

Six studies were performed during ICU admission, seven during an emergency department (ED) visit, and three during the pre-hospital phase. The remainder (six studies) reported on a combination of these settings. The majority of studies reported on guideline recommendations that were judged as weak/moderate. Only seven studies included strong recommendations. The majority of studies were funded by government organizations. One study²⁹ was funded by the BTF.

Methodological quality

Overall, the methodological quality of studies was good, with the majority of studies judged at low risk of bias in most domains (Table 2). For studies measuring the amount of adherence to guidelines (Objective 1, n = 22), 19 had an overall low risk of bias. The remainder (n = 3)^34,36,38 received a high risk of bias score, because of high scores on selection bias/confounding.

Table 2.

Risk of Bias Assessment

	Selection bias/confounding			Performance ias			Attrition bias		Detection bias		Reporting bias			Information bias			Highest score OB1	Highest score OB2	Highest score OB3
Study ID	OB1	OB2	OB3	OB1	OB2	OB3	OB2	OB3	OB2	OB3	OB1	OB2	OB3	OB1	OB2	OB3
Alali (2013)	L	M	L	L	L	M	L	L	L	L	L	L	L	L	L	L	L	M	M
Andriessen (2011)	L	-	-	L	-	-	-	-	-	-	L	-	-	L	-	-	L	-	-
Biersteker (2012)	L	L	L	L	L	M	L	L	L	L	L	L	L	L	L	L	L	L	M
Bulger (2002)	L	-	L	L	L	M	-	L	-	L	L	-	L	L	-	L	L	-	M
Fakhry (2004)	L	-	H	L	-	H	-	L	-	L	L	-	L	L	-	L	L	-	H
Farahvar (2012)	L	M	L	L	L	M	L	L	L	L	L	L	L	L	L	L	L	M	M
Franschman (2012)	L	-	-	L	-	-	-	-	-	-	L	-	-	L	-	-	L	-	-
Griesdale (2010)	L	M	M	L	L	M	L	L	L	L	L	L	L	L	L	L	L	M	M
Harr (2011)	L	M	-	L	L	-	L	-	L	-	L	L	-	L	L	-	L	M	-
Härtl (2006)	H	H	L	L	L	M	L	L	L	L	L	L	L	L	L	L	H	H	M
Haydon (2013)	L	-	-	L	-	-	-	-	-	-	L	-	-	L	-	-	L	-	-
Heskestad (2008)	L	H	-	L	L	-	L	-	L	-	L	L	-	L	L	-	L	H	-
Mauritz (2008)	L	H	L	L	L	M	L	L	L	L	L	L	L	L	L	M	L	H	M
Mooney (2011)	H	-	-	L	-	-	-	-	-	-	L	-	-	L	-	-	H	-	-
Prowse (2009)	L	-	-	L	-	-	-	-	-	-	L	-	-	L	-	-	L	-	-
Ravindran (2007)	H	-	-	L	-	-	-	-	-	-	L	-	-	L	-	-	H	-	-
Rognas (2013)	L	-	-	L	-	-	-	-	-	-	L	-	-	L	-	-	L	-	-
Rusnak (2007)	L	-	L	L	-	M	-	H	-	L	L	-	L	L	-	M	L	-	H
Shafi (2014)	L	L	L	L	L	M	L	L	L	L	L	L	L	L	L	M	L	L	M
Shafi-b (2014)	L	-	-	L	-	-	-	-	-	-	L	-	-	L	-	-	L	-	-
Shravat (2006)	L	-	-	L	-	-	-	-	-	-	L	-	-	L	-	-	L	-	-
Talving (2013)	L	L	L	L	L	M	L	L	L	L	L	L	L	L	L	L	L	L	M

Table represents the risk of bias for the three objectives.

OB1, objective 1 (assessing the amount of adherence); OB2, objective 2 (assessing factors influencing adherence); OB3, objective 3 (assessing the association between adherence and outcome); L, low risk of bias; M, moderate risk of bias; H, high risk of bias.

For studies exploring factors influencing adherence to guidelines (Objective 2, n = 10) three and four studies received a low and moderate overall risk of bias score, respectively. Three studies^34,43,44 were judged as being at high risk of bias because of selection bias/confounding.

None of the studies examining the association between adherence to guidelines and outcome (Objective 3, n = 11) had an overall low risk of bias. Nine studies received a moderate risk of bias score and two studies^42,46 a high risk of bias score. This was because of selection bias/confounding, performance bias, and information bias. None of the studies sufficiently isolated the impact of the guideline studied from concurrent interventions. In addition, some studies used inappropriate control groups or did not adjust for confounders while others calculated adherence or quality scores that were based on nonvalidated scoring mechanisms or partly based on guideline recommendations that were not evidence-based or (inter)national.

Concordance rates between independent reviewers in assessing risk of bias was high (92%), and any discrepancies were resolved by discussion or consulting a third author.

Amount of adherence to guidelines

The amount of guideline adherence was reported in all included studies (Table 1) and varied considerably between (range 18–100%) and within (range 0–100%) studies. Excluding studies with a high risk of bias^34,36,38 did not influence this variation.

Among the guidelines that were examined by more than one study, adherence was the highest in NICE CT-scan guidelines^35–39 (mean 87%, range 70–100%) and the lowest in BTF ICP monitoring guidelines^{10,26,28–39,32,40,41,46,47} (mean 31%, range 18–83%). Studies about the NICE CT scan guidelines were all performed at the ED in the United Kingdom and included patients with head injury. The majority had a single-center design. Studies about ICP monitoring were performed in Europe and North America and performed during ICU admission. Most studies used a multicenter design.

The studies with the lowest and highest percentage adherence to ICP monitoring guidelines were comparable multicenter studies performed in North America. The study with the highest percentage adherence was based on the TBI-Trac database, which is a database from the BTF aiming to track and improve adherence, while the study with the lowest percentage was based on general trauma databases. A visual display of adherence per guideline is provided in Figure 2. After removing studies with a high risk of bias (n = 3), adherence to the NICE guidelines was 75%. Adherence to other guidelines did not differ substantially.

FIG. 2.

Percentage guideline adherence for various guideline recommendations. Figure displays lowest, highest, and mean percentages adherence for various guideline recommendations. Numbers correspond with number of guideline recommendation and not to individual studies, because some studies reported on multiple guideline recommendations. “Other” is a summary measure of the following: Brain Trauma Foundation (BTF) intensive care unit protocol for patients with severe traumatic brain injury,⁴² BTF hyperventilation,⁴⁶ BTF barbiturates,⁴⁶ BTF antiseizure prophylaxis,⁴⁶ BTF intracranial pressure (ICP) directed therapy,⁴⁰ and BTF craniotomy.⁴¹ NICE, National Institute for Health and Care Excellence; CT, computed tomography

To assess whether strength of recommendation was related to guideline adherence, we divided guidelines into strong, and moderate/weak recommendations. Strong recommendations consisted of NICE CT scan guidelines, reported in five studies, and BTF steroids guidelines, reported in two studies. All other guideline recommendations were based on low levels of evidence. Mean adherence to strong recommendations was 93% (range 70–100%) while adherence to moderate/weak recommendations was considerable lower (mean 49%, range 18–94%). Percentages did not differ substantially after removing studies that were found to be at high risk of bias. One study⁴² was excluded from this analysis because it reported adherence to an ICU protocol that was based on both strong and moderate/weak recommendations.

In addition, we considered whether the invasiveness of the intervention was related to adherence. Across studies, relatively invasive interventions such as ICP monitoring and intracranial operations obtained a mean adherence rate of 30% (range 8–83%), while less invasive interventions such as CT scanning and antiseizure prophylaxis obtained a much higher adherence rate (mean 79%, range 51–100%).

Factors influencing guideline adherence

Ten studies identified factors influencing adherence (Table 3). Most studies assessed patient demographics and clinical characteristics. Three studies assessed treatment, hospital, or country characteristics. Taking the results together, the BTF guidelines, in particular the ICP monitoring recommendations, were consistently more often adhered to in younger patients with extracranial injury and more severe TBI (indicated by Glasgow Coma Scale [GCS], Head Abbreviated Injury Scale [HAIS], abnormal pupillary reactions, and intracranial pathology). The Scandinavian guidelines were more often adhered to in older patients with moderate head injury in comparison with mild and minimal head injuries.

Table 3.

Factors Significantly Associated with Adherence to Guidelines in at Least One Study

	BTF – ICP monitoring					BTF – direct transport	BTF – various recommendations		Scandinavian guidelines
	Alali (2013)	Biersteker (2012)	Farahvar (2012)	Griesdale (2010)	Talving (2013)	Härtl (2006)	Mauritz (2008)	Shafi (2014)	Harr (2011)	Heskestad (2008)
Patient and clinical characteristics
Age	– A	–	– A	– A	–	+		–	++	+/− A
Male sex	– A	+	+ A	+ A	+ A			++	+	+/− A
Insurance status	++ A*							+/−
Injury mechanism	++/– A	+/− A
GCS	− A	–	− A	– A	–**			+/− A
HISS									++	++ A
HAIS	+ A				+			++
Comorbidity	– A							+/− A
Extracranial injury		+ B			++			+/− A
Abnormal pupillary reactions		++	++ A		− A
Hypotension	– A	− A	− A	+ A	–			–****
Intracranial pathology	++ A	++	+ A	++ A	++			++
PTT					–
Process, hospital, and country characteristics
Decompressive craniotomy/ craniectomy				++ A	++***
Teaching status	++ A
Gross national product							++ A

+, Positive, nonsignificant effect; −, negative, nonsignificant effect; ++, positive, significant effect; –, negative, significant effect; +/−, = direction or statistical significance unknown.

^A, predictor is solely univariately assessed; ^B, predictor is significant in univariate analyses, but not in multivariable analyses.

^*Commercial insurance vs. noncommercial insurance (United States).

^**Lowest Glasgow Coma Scale (GCS) score within 24 h is statistically associated with adherence; median GCS was not statistically associated.

^***Decompressive craniotomy within 4 h is associated with adherence in univariate and multivariable analysis, decompressive craniotomy within 24 h is only associated in univariate analysis.

^****Authors measured systolic blood pressure. Higher systolic blood pressure is associated with more adherence

BTF, Brain Trauma Foundation; ICP, intracranial pressure; GCS, Glasgow Coma Scale; HISS, Head Injury Severity Scale; HAIS, Head Abbreviated Injury Scale; PTT, partial thromboplastin time.

Among studies with a relatively low risk of bias that assessed factors influencing adherence using multivariable analyses, age was significantly associated with adherence in all studies (younger age is associated with greater adherence in patients with severe TBI; older age is associated with greater adherence in patients with minimal, mild, and moderate TBI). Studies about ICP monitoring further reported that adherence was more often accomplished in patients with a lower GCS and the occurrence of intracranial pathology.

Factors that were studied but not significantly associated with adherence included race,^28,40 certain severity indices (GCS motor score²⁸; Acute Physiology and Chronic Health Evaluation [APACHE] II score³²), certain laboratory values (international normalized ratio and prothrombin time,⁴⁷ blood alcohol level³³), certain complications (tachycardia,⁴⁷ hypoxia⁴⁷), referral status,²⁷ and structural hospital characteristics (hospital type,²⁸ number of beds,²⁸ trauma center designation²⁸). For an overview of factors significantly associated with adherence in at least one study, see Table 3. For a complete overview of all factors studied, see Supplement D.

The association between guideline adherence and outcome

Eleven studies examined the association between guideline adherence and outcome (Table 4). All studies examined the BTF guidelines with six studies investigating ICP monitoring guidelines, one study examining direct transfer, and the remainder combining various BTF recommendations into a compliance or quality score.

Table 4.

The Association between Adherence to Guidelines and Patient Outcome

Study ID	Outcome variables	Direction of association
Alali (2013)	In-hospital mortality	–
Biersteker (2012)	6 month mortality	−
	6 month unfavorable outcome	+
	ICU LOS	++a
	Hospital LOS	++a
Bulger (2002)	In-hospital mortality	–
	Hospital LOS	−
Fakhry (2004)	Mortality	–a
	ICU LOS	–a
	Hospital LOS	–a
	Unfavorable outcome (GOSE) at discharge	–a
	Lower RLAS at discharge	–a
Farahvar (2012) Gerber (2013)	2-weeks mortality	–
Griesdale (2010)	In-hospital mortality	++
	28-days mortality	+
	ICU LOS	++a
Härtl (2006)	2-weeks mortality	–
Mauritz (2008)	ICU mortality	−/–
Rusnak (2007)	ICU mortality	−
	90 days unfavorable outcome (GOS)	−
	ICU LOS	+a
	Hospital LOS	−a
Shafi (2014)	In-hospital mortality	–
Talving (2013)	In-hospital mortality	–
	ICU LOS	++
	Hospital LOS	++

^**+, Positive, nonsignificant effect; −, negative, nonsignificant effect; ++, positive, significant effect; –, negative, significant effect. The direction of the multivariable analyses were noted. If there was no multivariable analysis performed, the univariate analysis was reported and a ^C was noted.

^a, Univariate association adherence – outcome.

ICU, intensive care unit; LOS, length of stay; GOSE, Glasgow Outcome Score Extended; RLAS, Rancho Los Amigos Scale; GOS, Glasgow Outcome Score.

Outcome measurements included in-hospital mortality,^{28,29,32,40,42,47} 2-week mortality,^30,34 28-day mortality,³² 6-month mortality,²⁷ Glasgow Outcome Scale (GOS) and Rancho Los Amigos Scale (RLAS) at discharge,⁴² 90-day Extended Glasgow Outcome Scale (GOSE),⁴⁶ 6-month GOSE,²⁷ ICU survival,^44,46 and ICU and hospital length of stay (LOS).^27,29,42,47

The majority of studies (n = 8) analyzed the adherence-outcome association with multiple regression adjusted for relevant confounders^{30,34,40,44,46} or for propensity scores.^27,32,47 Two multicenter studies analyzed the association on the hospital level by dividing hospitals into quartiles based on their percentage adherence²⁸ or by dividing hospitals into having an aggressive or nonaggressive approach.²⁹ One study univariately assessed the association.⁴²

Eight of 11 studies reported a statistically significant association between adherence and a reduction in mortality with odds ratios ranging from 0.15 to 0.96^{28–30,34,40,42,44.47} One study additionally described an association between adherence and higher scores on GOSE and RLAS.⁴² One study reported increased in-hospital mortality in those treated according to the guideline but no significant differences between groups in 28-day mortality.³²

For ICU and hospital LOS, three studies^27,32,47 reported an association with longer LOS, and one study reported an association with shorter LOS.⁴² All other associations were nonsignificant.

After adjusting for the risk of bias by removing studies with a high risk of bias on at least one of the criteria and outcomes that have been univariately assessed, all but one of the nine remaining studies³² reported an association between adherence and a reduction in mortality. Functional outcome was assessed in one study,²⁷ showing nonsignificant results. The association with LOS was assessed with multivariable analyses in two studies,^29,47 showing contradictory results. Statistical methods and results can be found in Supplement E.

Discussion

This systematic review provides an overview of adherence to guidelines, its determinants, and association with outcomes in patients with TBI. We included 22 studies, reported in 25 publications. Guideline adherence in TBI was found to be suboptimal overall and varied widely between studies (from 18–100%) and within multicenter studies. Guideline recommendations based on strong evidence were more often adhered to in comparison with recommendations based on lower level evidence. Guideline adherence was also influenced by age and severity (indicated by intracranial pathology and lower GCS). Importantly, guideline adherence appears related to patient outcomes, because adherence to BTF (especially ICP monitoring) guidelines was associated with a reduction in mortality in all but one study after correction for risk of bias.

This systematic review included three objectives and thereby provided an overview of the entire scope of adherence to guidelines in TBI. Five important notes should be made, however, regarding the completeness and applicability of the evidence.

First, despite the existence of more than 100 evidence-based guideline recommendations,⁵¹ adherence was assessed for only 13 recommendations. Results can therefore not be generalized to all guideline recommendations. Second, the variability in adherence might have been confounded by the invasiveness of the recommended intervention. We found a lower adherence rate in studies about invasive interventions such as ICP monitoring and craniotomy in comparison with studies with less invasive interventions. Invasive interventions require more experience and skills within the institution and therefore may face greater barriers to be implemented than less invasive interventions.

Third, no definitive conclusion about the efficacy of guidelines can be drawn from this review because we did not include any cluster RCTs. These results should encourage the conduct of cluster RCTs to more rigorously examine the efficacy of guidelines for TBI.

Fourth, all included studies were conducted in Europe and North America. Hence, our findings are not generalizable to non-Western countries because lack of resources restricts the routine use of aggressive treatment strategies in these countries.⁵² Related, our findings cannot be generalized to children because it is known that guideline adherence in children varies from guideline adherence in adults³⁶ and might also be influenced by other factors such as concern about radiation.

Last, the majority of current TBI guidelines are not based on high quality evidence. TBI is, however, emerging as an important topic in research with large-scaled, high-quality multicenter studies conducted all over the globe.¹³ These are likely to result in revised guidelines based on more rigorous evidence.¹³ The findings of this review might not be generalizable to a situation in which TBI guidelines are based on robust evidence, which underlines the importance of keeping this systematic review, as well as other systematic reviews in the field of TBI, “living.”

Overall, the methodological quality of the studies was good. The association between adherence to guidelines and outcome was, however, highly suspect for performance bias, because none of the studies sufficiently isolated the impact of the guideline studied from concurrent interventions. It is, nevertheless, plausible that patients who had, for example, an ICP monitor inserted also had a higher chance of receiving ICP lowering treatment and that this therapy might have caused the association with outcome.

Although selection bias/confounding did not seem a major threat to validity in the association between adherence and outcome, the risk of bias form we used did not account for confounding by indication. Observational studies in critical care may easily suffer from confounding by indication—i.e., a different a priori risk of unfavorable outcome between those treated and those not treated according to the guideline.^53,54 Although the majority of studies made attempts to reduce the risk of confounding by multivariable analysis or propensity score adjustment, these methods may still insufficiently resolve the problem of confounding by indication because they do not account for unmeasured confounders.^54–56 This is in contrast to an RCT, where comparability between groups is achieved on measured and unmeasured characteristics. In this review, two studies defined guideline adherence at the level of the hospital, which is more likely to provide a valid estimate of the effect of adherence on outcome.

Suboptimal adherence and between center variation have been reported in other systematic reviews about guideline adherence in critical care.^6,57 Ebben and associates⁶ reported a variation as large as 0–98% in a systematic review about guideline adherence in the pre-hospital and emergency care.

The large between-center variation suggests that guideline adherence is a management or structural characteristic, which is consistent with a qualitative study about guideline adherence in the ICU.⁵⁸ These authors reported that unit culture and communication were among the most important factors in guideline adherence. Further, the availability of electronic protocols, education, reminders, and an audit-feedback system were identified by participants as important determinants of guideline adherence. Surprisingly, only one of the included studies in this review assessed the association between hospital characteristics and adherence.²⁸

In this review, we found that strong recommendations were more often adhered to than recommendations based on lower level evidence. This is consistent with the findings of a study about oncology guidelines.⁵⁹ This may imply that clinicians are not convinced by the benefit of moderate and weak guideline recommendations, which is supported by our finding that intracranial pathology is associated with adherence to ICP monitoring guidelines. The recommendation to place an ICP monitor in patients without CT abnormalities but with additional risk factors stems from one prospective study published in 1982,⁶⁰ while the recommendation to place an ICP monitor in patients with an abnormal head CT is, albeit still controversial, based on more robust evidence.

Other clinical characteristics that were associated with guideline adherence were age and GCS. The negative association between age and adherence in patients with severe TBI is conceivable because older age is associated with medical comorbidity and pre-morbid anticoagulant or antiplatelet use.⁶¹ It has been suggested that these patients should not be treated aggressively,⁶² although the BTF guidelines do not specify any subgroups in their recommendations.

The positive association between lower GCS and adherence to BTF guidelines is in line with findings from methodological studies about confounding by indication in critical care, which describe that the most intensive treatments, such as ICP monitoring, are often reserved for the most ill.^53,63

The association between adherence and a reduction in mortality is consistent with a systematic review of protocolized management of patients with TBI in the ICU⁴ and a cost-benefit analysis about the effectiveness of the BTF guidelines.⁶⁴ Although these findings are consistent, they should be interpreted with caution because of the high risk of confounding by indication and performance bias in these studies.

Strengths of this systematic review include the use of a comprehensive search strategy and independent screening, data extraction, and quality assessment by two review authors. As there is no gold standard for risk of bias assessment in observational studies,⁶⁵ we developed and piloted our own form. This could be considered a review limitation; however, we attempted to describe the six threats to validity as described by the Cochrane Collaboration and used two validated forms. In addition, concordance rates in assessing bias were high, suggesting unambiguous items. Finally, despite an extensive search strategy, we found no unpublished studies. Although the performance of audits to test and improve guideline adherence is well practiced,⁶⁶ these reports are seldom published in international journals. Combined with the fact that we excluded non-English language studies, it is likely that some publication bias exists within this review.

The results of this review imply that guideline adherence in TBI is suboptimal. Certain subgroups, such as older patients or patients with severe TBI with a relatively high GCS are even less likely to be treated according to the guidelines. One solution may be for guideline developers to take into account specific subgroups of patients and tailor their recommendations accordingly.

The fact that strong guideline recommendations were more often followed than those based on less robust evidence speaks to the need for adequate investment in high-quality research to evaluate treatment efficacy and effectiveness, and for this research to be incorporated rapidly into guidelines. We would recommend high quality RCTs and large-scale comparative effectiveness studies using robust methods to adjust for confounding by indication for this purpose.

The large variation found in this systematic review highlights the importance of hospital characteristics and/or management strategies in guideline adherence. Although this has been reported in qualitative studies, further quantitative research may shed greater light on its importance and elucidate which characteristics inhibit clinicians from adhering to guidelines.

In this systematic review, we found an association between adherence to current guidelines and reduced mortality. These results should be interpreted as preliminary because only two studies accounted for confounding by indication and none could eliminate the effect of concurrent interventions. It is important that future studies investigating guideline adherence or treatment effectiveness use robust methods to adjust for confounding by indication and concurrent treatment interventions to estimate effectiveness.

Author Disclosure Statement

No competing financial interests exist.