A prospective cluster-randomized trial to implement the Canadian CT Head Rule in emergency departments

Abstract

Background

The Canadian CT Head Rule was developed to allow physicians to be more selective when ordering computed tomography (CT) imaging for patients with minor head injury. We sought to evaluate the effectiveness of implementing this validated decision rule at multiple emergency departments.

Methods

We conducted a matched-pair cluster-randomized trial that compared the outcomes of 4531 patients with minor head injury during two 12-month periods (before and after) at hospital emergency departments in Canada, six of which were randomly allocated as intervention sites and six as control sites. At the intervention sites, active strategies, including education, changes to policy and real-time reminders on radiologic requisitions were used to implement the Canadian CT Head Rule. The main outcome measure was referral for CT scan of the head.

Results

Baseline characteristics of patients were similar when comparing control to intervention sites. At the intervention sites, the proportion of patients referred for CT imaging increased from the “before” period (62.8%) to the “after” period (76.2%) (difference +13.3%, 95% CI 9.7%–17.0%). At the control sites, the proportion of CT imaging usage also increased, from 67.5% to 74.1% (difference +6.7%, 95% CI 2.6%–10.8%). The change in mean imaging rates from the “before” period to the “after” period for intervention versus control hospitals was not significant (p = 0.16). There were no missed brain injuries or adverse outcomes.

Interpretation

Our knowledge–translation-based trial of the Canadian CT Head Rule did not reduce rates of CT imaging in Canadian emergency departments. Future studies should identify strategies to deal with barriers to implementation of this decision rule and explore more effective approaches to knowledge translation. (ClinicalTrials.gov trial register no. NCT00993252)

More than six million instances of head and neck trauma are seen annually in emergency departments in Canada and the United States.¹ Most are classified as minimal or minor head injury, but in a very small proportion, deterioration occurs and neurosurgical intervention is needed for intracranial hematoma.^2,3 In recent years, North American use of computed tomography (CT) for many conditions in the emergency department, including minor head injury, has increased five-fold.^1,4 Our own Canadian data showed marked variation in the use of CT for similar patients.⁵ Over 90% of CT scans are negative for clinically important brain injury.^6–8 Owing to its high volume of usage, such imaging adds to health care costs. There have also been increasing concerns about radiation-related risk from unnecessary CT scans.^9,10 Additionally, unnecessary use of CT scanning compounds the Canadian problems of overcrowding of emergency departments and inadequate access to advanced imaging for nonemergency outpatients.

Clinical decision rules are derived from original research and may be defined as tools for clinical decision-making that incorporate three or more variables from a patient’s history, physical examination or simple tests.^11–13 The Canadian CT Head Rule comprises five high-risk and two medium-risk criteria and was derived by prospectively evaluating 3121 adults with minor head injury (Figure 1) (Appendix 1, available at www.cmaj.ca/cgi/content/full/cmaj.091974/DC1).⁶ The resultant decision rule was then prospectively validated in a group of 2707 patients and showed high sensitivity (100%; 95% confidence interval [CI ] 91–100) and reliability.¹⁴ The results of its validation suggested that, in patients presenting to emergency departments with minor head trauma, a rate of usage of CT imaging as low as 62.4% was possible and safe.

Figure 1.

The Canadian CT Head Rule, as used in the study. Note: CSF = cerebrospinal fluid, CT = computed tomography, GCS = Glasgow Coma Scale.

Unfortunately, most decision rules are never used after derivation because they are not adequately tested in validation or implementation studies.^15–19 We recently successfully implemented a similar rule, the Canadian C-Spine Rule, at multiple Canadian sites.²⁰ Hence, the goal of the current study was to evaluate the effectiveness and safety of an active strategy to implement the Canadian CT Head Rule at multiple emergency departments. We wanted to test both the impact of the rule on rates of CT imaging and the effectiveness of an inexpensive and easily adopted implementation strategy. In addition, we wanted to further evaluate the accuracy of the rule.

Methods

Design

We conducted a matched-pair cluster-design trial at 12 hospital emergency department sites in the Canadian provinces of Ontario, Alberta and British Columbia. These sites were stratified by hospital status (six teaching sites, six community sites) and then matched according to estimated baseline rates of ordering of CT scans of the head. Within each pair, sites were allocated randomly to either intervention or control groups. Outcomes were recorded during two consecutive 12-month periods (“before” and “after”). During the “after” period, strategies to actively implement the Canadian CT Head Rule were carried out at the intervention sites. The study took place from Nov. 1, 2002, to Oct. 31, 2005, with different sites starting the “before” and “after” periods at slightly different times.

Study population

We enrolled all alert and stable adults who presented to the study emergency departments after sustaining an acute minor head injury, which was defined as the presence of all of the following conditions: blunt trauma to the head resulting in witnessed loss of consciousness, amnesia or witnessed disorientation, an initial Glasgow Coma Scale score of 13 or greater, and occurrence of the injury within the previous 24 hours. Daily patient census logs were reviewed to identify potential study patients and patient eligibility was determined from emergency department records. Patients were excluded from the study if they were younger than 16 years of age, had penetrating skull injury, had focal neurologic deficit, had suffered a seizure before arrival at the emergency department, had a bleeding disorder or used warfarin, or returned for reassessment of the same head injury.

For each study site, approval was obtained from the research ethics board of the hospital involved, without the need for informed consent.

Study interventions

The design of the intervention was based on theoretical considerations, current evidence and consultation with knowledge transfer experts.^21–23 The Canadian CT Head Rule was implemented at the six intervention sites using strategies that were simple and inexpensive rather than complex and resource-intensive. Physician groups at the emergency departments discussed the goals of implementation and agreed to order CT scans according to the decision rule. Educational initiatives included distribution of articles that outlined the CT Head Rule and reported results of the validation study, pocket cards and posters, as well as a one-hour teaching session to review the supporting evidence and clinical application of the rule. Lastly, we implemented a mandatory, real-time reminder of the rule at the point of requisition for CT scans. All CT scans ordered in the emergency department required the requesting physician to check off the decision rule criteria or indicate the reason for overriding the rule before the scan would be performed by the diagnostic imaging department.

The Canadian CT Head Rule was familiar to many clinicians because of previous passive dissemination (i.e., through journal articles and conference lectures) and because seven of the emergency departments had participated in the derivation and validation studies. At the control sites, no specific intervention was introduced to alter the CT–scan-ordering behaviour of emergency department physicians, who had not been told about the study.

Outcome measures

The primary outcome was the proportion of eligible patients referred for CT scan of the head. Radiology reports and census lists were reviewed to determine if a CT scan of the head was performed.

We also measured the clinical impact of the intervention by assessing the number of clinically important cases of brain injury (defined as any acute brain-related finding on CT scan that would normally require admission to hospital and neurosurgical follow-up) that were not identified during the initial visit to the emergency department²⁴ and the number of deaths from head injury that occurred after the emergency department visit. Logs of patient visits to the emergency department were monitored to identify return visits within 30 days by patients who had not undergone a CT scan. In addition, we reviewed the logs for neurosurgery patients at all neurosurgical centres that were traditional referral sites for the study hospitals.

Performance of the Canadian CT Head Rule was a secondary study outcome. For cases in which physicians completed the study requisition and checked off the rule criteria during the “after” period at intervention sites, we evaluated the sensitivity of the decision rule and physicians’ accuracy in interpretation of the rule. The latter was assessed by comparing each physician’s notation on the CT scan requisition to the “gold standard” interpretation made by the investigators’ steering committee.

We conducted post-study interviews with investigators at intervention hospital sites to gather their perceptions of barriers to and facilitators of use of the decision rule.

Statistical analysis

The final analysis included all patients who satisfied the inclusion and exclusion criteria. No patient was excluded during the “after” period because of noncompliance with the rule by a physician. Subgroup analyses were used to evaluate teaching hospitals and community hospitals separately.

For analysis of dichotomous data, a standard paired t test was used to assess differences in event rates in intervention site and control site pairs (5 degrees of freedom).²⁵ For analysis of continuous data, a standard paired t test (5 degrees of freedom) was used to examine differences in mean response between intervention site and control site pairs. We calculated 95% confidence intervals (CIs) for relative reductions in event rates. For these relative reductions, the stratum-specific differences (percent change from baseline: (before-after) × 100/before) were calculated and pooled across strata using a random-effects meta-analysis.²⁶ The weighted paired t test was applied at the cluster level using the cluster-specific means, and the 95% CIs were calculated using the weights (n_j1n_j2)/n_j where n_jk is the size of the j^th stratum in the k^th group and n_j is the size of the j^th stratum. All reported p values are two-tailed. For the proportions of CT scans of the head that were ordered, the change in proportion from the “before” to “after” periods for each cluster was determined and used in the calculation of the differences in the event rates for each intervention and control site pair.

Sample size

Sample size calculation took into account the complexity of the study design (i.e., stratified, matched-pair and cluster-randomized [unit of randomization was the hospital and unit of analysis was the patient]).²⁷ A 15% relative reduction in the event rate (12.0% absolute reduction from the expected baseline rate of 80%) was considered a minimal clinically important change. Based on 80% power and a two-sided level of significance of p = 0.05, we estimated that six matched-pair clusters were required and a total sample of 4800 patients.

Results

Overall, 4531 eligible patients with head injury were seen at the 12 hospitals in the study. The characteristics of study patients that were similar when comparing the “before” period to the “after” period and the intervention group to the control group are listed in Tables 1 and 2.

Table 1.

Characteristics of 4531 patients with head injury

Characteristic	Intervention hospitals, no. (%)*		Control hospitals, no. (%)*
	Before n = 1049	After n = 1531	Before n = 876	After n = 1075
Age
Yr, mean (SD)	37 (19)	40 (19)	39 (19)	41 (20)
Range	16–99	16–97	16–96	16–97
Sex, male	737 (70.3)	1082 (70.7)	625 (71.3)	758 (70.5)
Hospital
Teaching	638 (60.8)	912 (59.6)	640 (73.1)	784 (72.9)
Community	411 (39.2)	619 (40.4)	236 (26.9)	291 (27.1)
Mechanism of injury
Motor vehicle collision	328 (31.3)	450 (29.4)	251 (28.7)	252 (23.4)
Rear-end motor vehicle collision	14 (1.3)	13 (0.8)	9 (1.0)	11 (1.0)
Ejection or rollover	77 (7.3)	124 (8.1)	49 (5.6)	43 (4.0)
Head-on collision	32 (3.1)	25 (1.6)	15 (1.7)	22 (2.0)
Fall	303 (28.9)	459 (30.0)	240 (27.4)	343 (31.9)
Assault	165 (15.7)	252 (16.5)	158 (18.0)	205 (19.1)
Bicycle	47 (4.5)	99 (6.5)	62 (7.1)	81 (7.5)
Struck while pedestrian	34 (3.2)	81 (5.3)	54 (6.2)	61 (5.7)
Struck in head	77 (7.3)	94 (6.1)	44 (5.0)	51 (4.7)
Sports	83 (7.9)	81 (5.3)	63 (7.2)	76 (7.1)
Other	12 (1.1)	15 (1.0)	4 (0.5)	6 (0.6)

^*Unless indicated otherwise.

Table 2.

Clinical outcomes of 4531 patients with head injury

Characteristic	Intervention hospitals, no. (%)		Control hospitals, no. (%)
	Before n = 1049	After n = 1531	Before n = 876	After n = 1075
Clinically important brain injury*	55 (5.2)	92 (6.0)	60 (6.8)	55 (5.1)
Epidural hematoma	11 (1.0)	14 (0.9)	10 (1.1)	9 (0.8)
Subdural hematoma	20 (1.9)	34 (2.2)	31 (3.5)	16 (1.5)
Intracerebral hematoma	0 (0.0)	2 (0.1)	2 (0.2)	2 (0.2)
Cerebellar hematoma	0 (0.0)	0 (0.0)	0 (0.0)	0 (0.0)
Diffuse cerebral edema	1 (0.1)	0 (0.0)	1 (0.1)	1 (0.1)
Cerebral contusion	27 (2.6)	44 (2.9)	28 (3.2)	24 (2.2)
Subarachnoid hemorrhage	25 (2.4)	54 (3.5)	33 (3.8)	27 (2.5)
Intraventricular hemorrhage	3 (0.3)	8 (0.5)	4 (0.5)	6 (0.6)
Pneumocephalus	6 (0.6)	7 (0.5)	6 (0.7)	7 (0.7)
Depressed skull fracture	2 (0.2)	6 (0.4)	1 (0.1)	1 (0.1)
Clinically unimportant brain injury	39 (3.7)	40 (2.6)	21 (2.4)	25 (2.3)
Neurosurgical intervention†	9 (0.9)	8 (0.5)	8 (0.9)	8 (0.7)
Craniotomy	6 (0.6)	6 (0.4)	6 (0.7)	6 (0.6)
Elevation of skull fracture	2 (0.2)	1 (0.1)	0 (0.0)	0 (0.0)
Isolated intubation for head injury	0 (0.0)	1 (0.1)	2 (0.2)	2 (0.2)
Isolated intracranial pressure monitoring	0 (0.0)	0 (0.0)	0 (0.0)	0 (0.0)
Death from brain injury	1 (0.1)	1 (0.1)	3 (0.3)	1 (0.1)
Admitted to hospital	214 (20.4)	386 (25.2)	177 (20.2)	168 (15.6)

^*Clinically important brain injury was defined as any acute brain finding on computed tomography (CT) scan that would normally require admission to hospital and neurosurgical follow-up. All brain injuries revealed on CT scan were considered clinically important unless the patient was neurologically intact and had one of the following lesions: a solitary contusion less than 5 mm in diameter, a localized subarachnoid hemorrhage less than 1 mm thick, a smear subdural hematoma (i.e., less than 4 mm thick), or an isolated pneumocephaly.

^†Only the most severe or invasive intervention is indicated for each patient.

Rates of CT imaging increased at all six intervention sites from the “before” period to the “after” period (Table 3), as well as at four control sites. At the intervention sites, the proportion of patients who received a CT scan increased from 62.8% in the “before” period to 76.2% in the “after” period (absolute difference +13.3%, 95% CI 9.7%–17.0%). At the control sites, use of CT scans also increased, from 67.5% to 74.1% (absolute difference +6.7%, 95% CI 2.6%–10.8%). When we assessed the relative change in mean imaging rates from the “before” period to the “after” period for intervention hospitals versus control hospitals, we found no significant difference (p = 0.16).

Table 3.

Rates of computed tomography use for 4531 patients with head injury during the 12-month “before” and “after” study periods

Site	No. of patients		Rate of CT imaging, %		Absolute change, % (95% CI)
	Before	After	Before	After
Intervention hospitals*	1049	1531	62.8	76.2	13.3 (9.7–17.0)
Teaching
Kingston General	125	147	36.0	49.0
Sunnybrook Health Sciences	122	164	54.9	64.0
University of Alberta	391	601	74.7	78.9
Community
Royal Columbian	166	315	72.3	94.6
Credit Valley†	133	186	60.2	72.0
St. Thomas Elgin†	112	118	49.1	70.3
Control hospitals*	876	1075	67.5	74.1	6.7 (2.6–10.8)
Teaching
Ottawa Hospital Civic Campus	242	278	77.7	87.1
London Health Sciences	84	108	73.8	82.4
Vancouver General	314	398	78.0	78.1
Community
Sturgeon†	63	127	42.9	37.8
North York General†	75	58	62.7	74.1
Surrey Memorial†	98	106	22.4	60.4

Note: CI = confidence interval, CT = computed tomography.

^*Hospitals are listed in order of matching.

^†Hospitals that did not participate in previous derivation and validation studies.

The subgroup analysis, which was stratified by community or teaching hospital, failed to show a positive impact of the intervention on CT imaging rates (Appendix 2, available at www.cmaj.ca/cgi/content/full/cmaj.091974/DC1). Rates of CT scanning increased more at community hospitals than at teaching hospitals, regardless of whether intervention or control. For the relative change in mean imaging rates from before to after, we found a significant increase in only the community-intervention subgroup (p = 0.001).

At the intervention hospitals, 150 physicians were involved in implementing the Canadian CT Head Rule during the “after” period. They completed the rule section of the radiology requisitions for 909 (78.0%) of the 1166 eligible patients referred for CT scan (Appendix 3, available at www.cmaj.ca/cgi/content/full/cmaj.091974/DC1). Physicians accurately interpreted the rule in 82.5% of the cases. Common errors were overcalling instances of a “dangerous mechanism” and missing scores of less than 15 on the Glasgow Coma Scale two hours post-injury. Physicians ordered CT imaging despite a negative rule in 141 cases and did not order CT imaging in six cases in which the rule was positive.

The decision rule could be interpreted for 897 patients for whom the CT scan requisition was sufficiently completed. The rule correctly identified all 88 clinically important brain injuries, achieving a sensitivity of 100% (95% CI 96%–100%). There were no missed clinically important brain injuries at any of the sites during the study period.

Interpretation

Our trial confirms that the Canadian CT Head Rule is highly sensitive (100%) for identifying clinically important brain injuries in patients with minor head injury.⁶ However, it did not show that implementation of the rule led to a reduction in emergency department use of CT imaging. In fact, there was an increase over time in use of CT scans at 10 of 12 sites, most notably at community hospitals. Many factors likely account for these findings, including our use of simple, inexpensive interventions, suboptimal compliance, crowding in Canadian emergency departments, increased ease of access to CT scans in emergency departments, and an overall secular trend toward increased use of CT imaging.

This implementation study, designed and conducted according to strict methodologic standards, represents the final stage in the development of a clinical decision rule.^11,13,15 Our previous studies to derive, validate and implement the Ottawa Ankle Rules, the Ottawa Knee Rule and the Canadian C-Spine Rule all resulted in large reductions in rates of imaging.^20,28–31 In these studies, we also achieved higher levels of physician compliance with the protocol (86%–95%).^20,28–30

It is unclear why this study, whose methods closely mirrored those used in the Canadian C-Spine Rule implementation study conducted contemporaneously in the same sites, failed to achieve the desired reduction in use of CT scans. Physicians responded differently to the same intervention for different clinical conditions, suggesting that the condition itself had a modifying effect on the implementation of rules in emergency departments. Our study directly replicated interventions in the same settings, which highlights the potential importance of the interaction of attributes of the targeted behaviour with the effects of interventions.³² Perhaps more intensive interventions were required to alter use of CT imaging compared with radiography for injuries of the cervical spine, knee and ankle.

Given that baseline rates of CT imaging were lower than the potential rate predicted in the validation study, the increase in rates of CT imaging at intervention sites may simply reflect correction of unwarranted variation in practice. The increase may also signal a secular trend in increasingly congested emergency departments, where ordering a CT scan and discharging the patient is perceived as being more expedient than conducting a thorough clinical evaluation. Access to CT scanning has increased substantially in recent years, and it can now be ordered quickly and easily at most medium and large hospital emergency departments. Finally, our findings may also be a symptom of increased use of emerging technologies in medicine, where CT imaging of the head is considered the standard of care.³³ We observed frequent use of CT imaging for patients with minimal head injury when there is no evidence of neurologic disturbance.

We evaluated potential barriers to successful implementation before the start of the study and found that the most common concerns expressed by clinicians were inability to remember the rule, belief that trauma services would order a CT scan even if the physician did not, and expectations by patients and families that a CT scan would be ordered. In our post-study interviews, physicians identified other potential barriers. Some were not comfortable with the definition of clinically unimportant injury and believed that even trivial findings on CT imaging should be documented. Others stated that CT imaging has become the local standard of care for patients with minor head injury. A common comment was that emergency department overcrowding led to expedient over-testing in the belief that this approach would lead to earlier discharge of patients. Finally, many physicians felt that they could easily circumvent the rule without consequences from study or hospital staff, suggesting that mandatory use of decision rules at the point of requisition for imaging may not be an effective strategy.

Limitations

Our study had limitations. The baseline imaging rates were not as balanced between intervention and control sites as planned. In addition, the baseline rates were far lower than expected, making the prospect of further reductions unlikely. Perhaps these low rates at baseline were a reflection of prior passive dissemination of the rules or of participation by seven of the sites in previous studies of the Canadian CT Head Rule.^6,14 Two sites had unusual results. At one intervention site, almost twice as many patients were seen and a large increase (to 94%) in CT imaging was evident. One control site tripled its rate of imaging from an exceptionally low baseline rate of 22%. Physician compliance with the requisition form was lower than anticipated, and it is difficult to ascertain if this finding reflects a weakness of the implementation strategy or other barriers to use of the decision rule.

Our findings indicate a need for further research to understand the specific barriers to implementing the Canadian CT Head Rule and to evaluate interventions to overcome them. The findings of this and the sister study of the Canadian C-Spine Rule showed that apparently identical interventions targeting the same emergency departments for different conditions had dramatically different results. This difference highlights the importance of replication using knowledge translation trials that would allow exploration of the factors involved.³⁴

The Canadian CT Head Rule again proved to be accurate and reliable, and can be used with confidence by clinicians to determine which patients with minor head injury would benefit from use of CT imaging. Its usefulness is particularly applicable to sites where getting access to CT imaging is difficult or transfer of patients to a large centre is required.

Conclusion

Our knowledge translation trial of the previously validated and highly sensitive Canadian CT Head Rule did not reduce rates of usage of CT imaging in Canadian emergency departments. Future studies should identify strategies to deal with barriers to implementation of the rule and explore more effective approaches to knowledge translation.