IMPACT Requests: Evaluating the Effectiveness of IMPACT Education for Emergency Department Referrers to Improve the Quality of CT Requests

ABSTRACT

Introduction

The quality of radiology reports is closely linked to clinical information provided by referrers. Inadequate information can lead to broader differential diagnoses, delayed decisions and lower‐value healthcare. To address this, the IMPACT (Information Medically Pertinent for Acute Computed Tomography) criteria were developed to guide referrers on essential clinical details for computed tomography (CT) requests in emergency settings. This study aimed to evaluate whether education based on these criteria—IMPACT Education—improved the quality of CT requests.

Methods

A cross‐sectional study was conducted in a large Australian, metropolitan, trauma hospital. Junior doctors in the Emergency Department (ED) were provided with IMPACT Education, a multi‐modal teaching program based on rationalised, mnemonic‐driven clinical criteria for chest, abdomen and multi‐trauma CT requests. A total of 1200 CT requests, 200 made before education (the control group) and 200 made by referrers after completing education (post‐education group) for each scan region were assessed for compliance with the IMPACT Criteria. Statistical analyses included Chi‐square and Mann–Whitney U tests to assess changes in individual criteria and overall request scores between the two groups.

Results

Overall mean request scores were significantly higher for all scan types in the requests made by those who completed the education, compared to the control group. For CT chest and abdomen, the majority of individual criteria showed significant improvements in the post‐education sample. CT multi‐trauma requests showed more modest gains, possibly due to clinical urgency and complexity.

Conclusion

The IMPACT Education program improved the quality of clinical information in CT requests from ED referrers for CT chest, abdomen and multi‐trauma presentations. The success of the education is attributed to the use of evidence‐based teaching strategies. The next phase of this project will investigate the impact of this improvement in request quality on the quality of resultant radiology reports.

The use of IMPACT Requests criteria standards to educate Emergency Department referrers improves the quality of clinical information provided in CT chest, abdomen and multi‐trauma requests. The next phase of this project will investigate the impact of this improvement in request quality on the quality of resultant radiology reports.

1. Introduction

The primary outcome of any radiological examination is the radiologist's report. Accurate, relevant and timely reports enable referrers to provide patients with more definitive diagnoses, explain prognoses in greater detail and provide clearer treatment pathways [1, 2]. Conversely, ambiguous, delayed or inaccurate reports have been shown to contribute to delayed diagnoses and treatment of symptoms [3].

The communication of clinical information to radiologists has been shown to have a positive impact on the quality of radiology reports, improving their accuracy, clinical relevance and diagnostic confidence [4]. This is particularly relevant for complex modalities such as computed tomography (CT). The enhanced detail and three‐dimensional capabilities of modern CT scanners provide improved abnormality detection capability by radiologists [5]. However, without quality clinical information from referrers, the result can be a longer list of differential diagnoses [5]. Referrers can add value to the diagnostic process by providing radiologists with greater clinical context, allowing them to reduce the number of differential diagnoses [5].

When a patient presents to a referring clinician, they are medically assessed. If a referrer concludes that a CT is indicated, a request is completed, utilising information about the patient's current presentation and medical history. In some Australian jurisdictions, this request takes the form of a referral. In such cases, the radiologist reviews the request, determines the most appropriate imaging modality and protocol required, tailoring imaging to each patient's needs. In most cases, the communication between referrer and radiologist is asynchronous, with the referral submitted via paper or, increasingly commonly, through electronic systems. This one‐way communication prevents the radiologist from seeking extra information and clarification when required. This renders the inclusion of relevant, comprehensive clinical information from referrers within requests as imperative. The radiologist also reviews clinical information in the request when interpreting imaging and formulating their report [4].

Value‐based healthcare (VBH) aims to achieve the best possible health outcomes through the efficient and judicious use of health resources [6]. Improving the quality of the requesting process can add value from both resource usage [6], and outcome optimisation perspectives [4]. Evidence from Australia [7], and overseas [8], has identified up to 20% of diagnostic imaging examinations as low‐value, comprising unjustified examinations, incorrect modality and examination selection, incorrect protocoling and inappropriate timing of examinations [7, 8]. Interventions aimed at reducing inappropriate imaging and incorrect protocoling have been described as useful for increasing the value of radiology [9].

From an outcome optimisation perspective, referrers must be educated to ensure they provide adequate clinical information when requesting CT examinations [4]. Effective communication between referrers and radiologists relies on the referrer's understanding of the specific clinical information deemed important by the radiologist. Misunderstandings about what constitutes adequate clinical information can negatively impact communication, imaging optimisation and patient outcomes [10]. Glenn‐Cox et al. reported that junior doctors value knowing what information to include in requests [11]. The need for post‐graduate training to improve referrers' skills in CT requesting is well‐established [12].

In response to the lack of published Australian guidelines regarding requesting [13], the authors undertook a 4‐stage e‐Delphi study aimed at creating a set of standards for this purpose: namely the Information Medically Pertinent for Acute Computed Tomography (IMPACT) requests study [14] (IMPACT requests). The outcome of this study led to the creation of criteria standards (IMPACT Criteria) for essential clinical information for referrers to provide when requesting chest, abdomen and multi‐trauma CT requests in an acute setting [14]. A recommendation of this previous study was the utilisation of the IMPACT criteria to formulate an education program for referring clinicians. The details of this education program, entitled IMPACT Education, are explained below.

The aim of this study was to determine whether the IMPACT Education program improved the quality of clinical information in CT requests. Given the links previously established between quality clinical information, usefulness of radiology reports and patient outcomes, the authors sought to establish whether the application of the standards created in the previous study [14], could contribute to improving communication between CT referrers and the medical imaging team (radiographers and radiologists) in the Emergency Department (ED).

2. Methods

2.1. Ethics

Ethics approval was obtained from Metro South Health Human Research Ethics Committee (MSH HREC) to conduct this study (HREC: LNR/2020/QMS/68233).

2.2. Study Setting and Participants

The setting for the IMPACT Education intervention was the ED of a large, Australian, metropolitan, public trauma hospital. This hospital sees only adult patients. CT imaging is provided via a dedicated CT scanner with 24‐h staffing by both radiographers and radiologists. The volume of CT imaging is large, with approximately 30,000 CT scans performed annually. All imaging referrals at this site are submitted electronically via an integrated electronic medical record system. The target group for this intervention was junior doctors (residents), as their short, 3 month duration in the ED had been previously identified by ED consultants at the investigating site as a contributor to lower quality requests.

2.3. Study Design

This study was conducted via retrospective review and objective assessment of matched datasets: requests made by referrers who had not completed the education (the control group) and those who had (the post‐education group). Two samples of 200 requests were collected for each scan type (CT Chest, CT Abdomen and CT Multi‐trauma): one sample from the control group and the other from the posteducation group. Participants in the post‐education sample self‐selected into the study by completing a post‐education assessment of their learning at the conclusion of the IMPACT Education program. Requests from these referrers made up the post‐education sample. No demographic or identifying participant data were collected.

2.4. Training Development and Delivery

The IMPACT Education program was developed by two of the authors (CC and TS) and focused on educating CT referrers. The program was reviewed and approved by the Director of Education for ED and co‐author (GL) at the investigating site. It was implemented over a 16‐month period and consisted of a one‐off, face‐to‐face group session with participants viewing a 12 min pre‐recorded video accompanied by a short question and answer session and a question participant quiz. Participants were granted ongoing access to the educational video to assist in their practice. Overall, 127 referrers participated in the study. The program was designed over multiple stages. The main factors considered were accessibility of learning, multi‐modal support to reinforce teaching, efficiency of learning and the use of evidence‐based, effective teaching methods.

The large number of items in the IMPACT Criteria [14] (101 for chest, 76 for abdomen, 80 for multi‐trauma) was considered impractical for effective application to practice. Hence, the criteria were rationalised. Initially, common themes were identified and the discrete items were grouped to form individual criteria. For instance, within the chest criteria standard [14], initial criteria items included, ‘side and site of each symptom’, ‘presence of fever/febrile’, ‘presence of shortness of breath/dyspnoea’, ‘presence of haemoptysis’, ‘duration of symptom’, onset of pain’, ‘presence or absence of pain’ and ‘side of pain’. These items were rationalised to form one criterion, namely ‘existing symptoms'. These individual criteria were the basis for the educational content. Mnemonic acronyms were developed to aid teaching and learning [15] (See Figure 1).

FIGURE 1.

Copy of teaching tool signage.

A review of education literature identified the most effective teaching strategies for a well‐educated, time‐poor cohort. Hattie's research [16], on the effect size of various teaching methods guided the selection of strategies, namely direct instruction, worked examples, formative evaluation, teacher clarity and teacher credibility.

Education was delivered over 16 months (from August 2021 to December 2022) by authors (CC, TS) who utilised existing, scheduled teaching time. Both facilitators hold clinical education positions at the investigating site and both possess educational qualifications: a Bachelor of Teaching (CC) and Certificate IV in Training and Education (TS). Content included information about the rationale for creation of the standards and detailed explanations of IMPACT criteria [17], themselves. Previous rapport and subject matter expertise had been established in prior teaching sessions; this enhanced teacher credibility. Further, the methods used to create the criteria standards were explained. Once the content had been delivered, examples of both high and low‐quality requests were outlined to promote understanding and retention. The education session concluded with a short quiz to help consolidate the new knowledge gained, and this functioned as a formative evaluation of overall content.

An important factor in the delivery of education was the time available to the learners. Workload demands on ED doctors negatively affect professional development and education [12]. Hence, our education program was kept as short as possible, and the content was also made available to referrers on demand. Referrers were also given a personal lanyard card outlining the mnemonics. Additional larger signage was placed around the ED.

2.5. Data Collection

A sample of 200 requests for each CT scan type (chest, abdomen, multi‐trauma) was collected from the Picture Archiving and Communication System (PACS) for each of the control group and post‐educational samples. A total of 1200 requests were collected. These scan types were selected as they corresponded to the previously created IMPACT Criteria. The control group samples were taken from requests at least one year prior to the education program from a different cohort of requesters. Requests were excluded from the study if the request included multiple body regions beyond the scope of the study, for example, abdomen requested, as well as head. This is because the clinical information provided may not have been specific to the scan types included in the study.

The inclusion criteria for the CT chest requests were ED patient location, acute and nontraumatic chest symptoms. The CT abdomen inclusion criteria were ED patient location, acute and nontraumatic abdominal symptoms. The inclusion criteria for multi‐trauma requests were ED patient location, multi‐trauma presentation and request for multi‐region (head, cervical spine, chest, abdomen and pelvis) CT scan. Only requests from referrers who had undertaken the education were included in the post‐education data collection.

For each included request the scan type (e.g., CT pulmonary angiogram (CTPA), CT abdomen with contrast), clinical information provided by the referrer, the referrer name and date of request were recorded. Before the scoring of the requests was conducted, both referrer name and the date of request were removed by one of the investigators (TS). The large number of requests in this study meant that the risk of recall bias was low. The data were stored electronically, specifically on an access‐controlled, authenticated, network computer drive hosted by the investigating site.

2.6. Scoring System

The clinical information provided in each included request was assessed against the criteria standards, with both total request score (out of 7 for CT chest and CT multi‐trauma, out of 6 for CT abdomen) and presence of each individual criterion recorded. Scoring was conducted by two assessors (CC, TS) with an arbiter (LC) used to resolve discrepancies, all of whom were registered as Australian Health Practitioners. If a criterion was considered present, it was given 1 point for that criterion; if considered not present, a score of 0 was given for that criterion. Before individual assessment, calibration of the assessors was conducted using 20 requests from each of the six samples. During this process, robust discussion enabled agreed definitions of what constituted compliance with each criterion. From this, examples were developed. This was designed to increase the individual assessors' ability to make objective assessments of each request, when assessing whether each criterion was present or not. For example, for the ‘CT CHEST’ IMPACT criteria (see Figure 1), to fulfil the ‘T' criterion, the request needed to mention the type of previous diagnosis of cancer (e.g., previous lung cancer) or must have explicitly stated no history of cancer. Following calibration, the control group and post‐education group samples for each scan type were combined and the order randomised to minimise risk of bias.

2.7. Data Analysis

Comparison of the relative compliance of the control group and post‐education group samples with the criteria standards was performed using different statistical tests. The Chi‐square test was used to evaluate the relationship between the categorical variables (participation in the education and the level of compliance for each individual criterion). Normality of the continuous variable (total request score) was assessed using the Shapiro–Wilk's test. Analyses of differences in this variable between those in the education program and those not were performed with the Mann–Whitney U test due to the ‘total’ variable not meeting parametric assumptions. Degrees of improvement (or decline in performance) for both mean total scores and individual criteria were calculated. These differences were expressed as a proportion (percentage) of the scores from the control group. All data analyses were performed in R version 4.4.2 (2024 − 10‐31) with RStudio Desktop version 2024.12.0.467.

3. Results

3.1. Overall Compliance With IMPACT Criteria

For all scan types, there were statistically significant increases in the mean total scores. These results are displayed in (Table 1).

TABLE 1.

Overall mean total score.

	Control group	Post‐education group	Improvement (%)	p
Chest (x̄/7) (SD)	3.37 (0.94)	4.15 (1.11)	23%	p < 0.001
Abdomen (x̄/6) (SD)	2.75 (1.14)	3.99 (1.32)	45%	p < 0.001
Multi‐trauma (x̄/7) (SD)	3.93 (1.20)	4.57 (1.29)	16%	p < 0.001

3.2. CT Chest

For CT chest requests, five of seven criteria (‘tumours/history of cancer’, ‘criteria‐Wells/D‐dimer’, ‘haemodynamic status’, ‘existing symptoms’ and ‘thoracic pathology history’) demonstrated statistically significant differences (in favour of the post‐education group) between the two samples. No significant differences were noted in the ‘clinical suspicion for acute aortic syndrome and ‘suspected diagnosis’ criteria. The results of individual criteria are displayed in (Table 2).

TABLE 2.

Percentage of CT Chest requests compliant with IMPACT criteria.

	Criterion	Control group (n = 200)	Post‐education group (n = 200)	Improvement (%)	p
C	Clinical suspicion for Acute Aortic Syndrome	35	42	18%	0.474
T	Tumours/ History of Cancer	34	62	72%	0.002
C	Criteria (Wells/D‐dimer)	87	117	31%	0.008
H	Haemodynamic status	34	92	156%	< 0.001
E	Existing symptoms	176	196	8%	0.003
S	Suspected diagnosis	186	189	‐1%	0.375
T	Thoracic pathology history	99	131	29%	0.004

3.3. CT Abdomen

For CT Abdomen requests, all six criteria demonstrated statistically significant increases between the control and post‐education samples. The results of individual criteria are displayed in (Table 3).

TABLE 3.

Proportion of CT Abdomen requests compliant with IMPACT Criteria.

	Criterion	Control group (n = 200)	Post‐education group (n = 200)	Improvement (%)	p
C	Clinical question	160	184	15%	< 0.001
T	Hypotensive symptoms	23	72	200%	< 0.001
A	Abdomen pathology/surgical history	120	148	23%	0.003
B	Bowel/urinary symptoms	99	137	38%	< 0.001
D	Definitive location/duration of symptoms	72	140	94%	< 0.001
O	Observations/tests	75	117	55%	< 0.001

3.4. CT Multi‐Trauma

For CT Multi‐trauma, three of seven criteria (‘clinical question’, ‘targeted search/tailored scan’, ‘injuries’) demonstrated statistically significant increases in compliance with the criteria standard, between the control and post‐education samples No significant differences were noted in the ‘type of injury’, ‘mechanism of trauma’, ‘unstable vs stable’ and ‘location of pain and suspected injury sites’ criteria. The results of individual criteria are displayed in (Table 4).

TABLE 4.

Proportion of CT multi‐trauma requests compliant with IMPACT criteria.

	Criterion	Control group (n = 200)	Post‐education group (n = 200)	Improvement (%)	p
C	Clinical question	54	122	126%	< 0.001
T	Type of injury	192	196	2%	0.241
M	Mechanism of trauma	159	165	4%	0.444
U	Unstable vs stable	80	89	13%	0.362
L	Location of pain and suspected injury sites	175	175	0%	1.000
T	Targeted search/tailored scan	96	120	25%	0.016
I	Injuries	30	46	53%	0.041

4. Discussion

This study demonstrates that the use of IMPACT Education can assist CT referrers in improving the quality of clinical information provided in requests. The results demonstrated statistically significant increases in the overall compliance with IMPACT Criteria between the control and post‐education groups for all scan types. In particular, the education had a large impact on the quality of information in CT abdomen requests, with the post‐education sample demonstrating a 45% higher score than that observed in the control group.

4.1. Interpretation of Results

The CT multi‐trauma results demonstrated only modest improvements across all criteria compared to the improvements achieved in the CT chest and CT abdomen groups. This may be attributed to the unique characteristics of the major, tertiary hospital where this study was conducted. The complex nature of trauma presentations and the high level of urgency required when requesting CT imaging may have negatively influenced the referrers' ability to apply their new learnings from IMPACT Education. The importance of time to CT for improving patient outcomes in serious trauma has been established [18]. Australian EDs have placed increasing emphasis on minimising this [19]. Time pressure has been identified as a risk factor for errors of omission by healthcare workers in emergency settings [20]. It is therefore possible that referrers' desire to comply with the criteria standards may have been impacted by this time pressure. That said, the significant increase in compliance with the clinical question, targeted search and injuries criteria may indicate that referrers were at least paying some heed to the education content.

The ‘haemodynamic status’ and ‘hypotensive symptoms’ criteria in both CT chest and CT abdomen results demonstrated statistically significant increases. The lower results in the control sample may be due to a lack of referrers' understanding of radiologists' specific clinical information needs to maintain high‐quality and accurate radiology examinations. The communication of these needs and the explanation of how the inclusion of this criteria influences subsequent decision‐making tasks like protocol selection and image interpretation, may have attributed to the increased compliance reported in the post‐education results. Although this concept is well‐known amongst radiologists and radiographers [21], it appears that participants were unaware of this or did not prioritise this when requesting. This is supported by the control group results.

Although the overall results for all scan types showed an improvement in compliance with the IMPACT criteria [14], compliance with some criteria was still relatively low in the post‐education samples. For example, the ‘clinical suspicion of acute aortic syndrome’ (21%) criterion within CT Chest and the ‘hypotensive symptoms’ (36%) criterion within CT Abdomen were met less than half the time, despite showing modest increases. This may have been due to the entrenched perceptions of referrers as to what CT requests require. Whilst the legislative requirement for medical imaging requests in Australia only requires a clinician to provide sufficient information to identify the item relating to the service requested [22], it is likely from the results of the authors' previous study [14], that radiologists view CT requests as more of a referral than simply a request. Referrals to other medical specialties have a higher threshold of information detail regarding relevant patient conditions. As radiologists often see evidence of varied pathological processes when reporting, they often require more information to place their observation into context [23]. Although this education program attempted to change the referrers' attitudes, it may have been insufficient to do so in isolation [23].

The barriers to implementing changes in clinical practice have been studied by Barach [24], who summarised that the complex nature of healthcare from both task and relational perspectives can challenge implementation efforts. In some cases, clinical guidelines have been shown to take up to 5 years to be fully adopted [24]. The complexity of shifting attitudes of clinicians has been studied further by Gupta, Boland and Aron [25], who described a multi‐stage process incorporating both un‐learning and learning. Several studies have reported on the benefit of repeat interventions, all of which demonstrated a sustained improvement with new processes over time [26, 27, 28]. It is possible that a change such as the one proposed in this education program, may require further efforts before it is embedded as a standard clinical process.

4.2. Comparison With Previous Studies

Our results are in keeping with another study that used departmental guidelines as a form of criteria standard [29]. These guidelines outlined recommendations for key clinical information required in requests. When used, this key information was found to demonstrate a positive effect on clinical relevance and confidence in reporting [29].

4.3. Implications for Practice

The authors consider the ultimate maturation of a program such as this to be the introduction of templates to guide Computerised Provider Order Entry (CPOE) into the Electronic Health Record. The easy‐to‐follow mnemonics created as part of our current study may provide a ready‐made solution, should such templates be implemented within radiology referral systems in the future. Similar approaches have been linked to improving healthcare quality by reducing inappropriate medical imaging examinations and decreasing the volume of high‐radiation medical imaging procedures, such as CT [30, 31]. Additional benefits of introducing a CPOE system include review of previous requests, identification of patterns on undesirable requesting and the provision of targeted advice to improve decision making [32].

4.4. Implications for Research

A recent systematic review by Tsang et al. [30] focusing on interventions aimed at improving the quality of referrals to radiology found low‐cost interventions to be suitable for first‐line interventions, as these are often associated with low‐disruption implementation. This study demonstrates IMPACT Education as an example of this type of intervention. Additionally, Tsang et al. [30], recommend a multi‐step approach for effective implementation of interventions of this type. The first of these is passive education requiring low effort, such as the posters and reminder cards used in our program. The use of display reminders or posters in referrer work areas and education sessions was featured in 84% of studies, which reported positive outcomes. Our use of existing teaching time to deliver educational content is in keeping with their recommendation of ensuring a low barrier for participants to access education. This approach may have contributed to the success of our intervention.

It is a historical and common complaint amongst radiologists that the clinical information contained in requests is inadequate and can affect associated report quality [4]. This study has been successful in demonstrating that the IMPACT Education (featuring IMPACT Criteria) [14], guides referrers to include more necessary information across the three featured scan types. The next logical step is to investigate whether this improvement in quality of clinical information influences the quality of associated radiology reports.

5. Limitations

The original plan for this intervention was to deliver interactive, face‐to‐face teaching to ED referrers. Due to COVID‐19 pandemic restrictions within the hospital at the time, the educational program was delivered both in‐person and online via an instructional video with no opportunity for clarification at the time of delivery. Using effective questioning in a psychologically safe environment is a useful tool for facilitators to check understanding, stimulate critical thinking and enhance the confidence of learners [33]. The asynchronous nature of this education did not allow for this to occur. Consequently, the online delivery of education may have reduced the influence of the intervention.

Similarly, the altered focus of the ED referrers caused by the high volume of new policies and processes they were expected to assimilate into their practice during this time may have diminished their ability to learn and apply new information. Working memory capacity, which refers to task‐relevant information and involves temporary storage of information that is being processed in any of a range of cognitive tasks, is limited to holding approximately seven elements when processing information [34]. Similarly, cognitive load theory recognises the idea that working memory limits the amount of information an individual can process [34]. Therefore, with the increased demand on referrers' cognitive load and working memory caused by the new policies and procedures at the time, it is possible that the effectiveness of the education may have been reduced.

Another prominent issue that may have impacted the effectiveness of the education was the global shortage of CT iodinated contrast [35]. The need to ration contrast reduced the number of CT requests submitted to only the scans deemed most essential. This lengthened the time period needed for data collection, which may have decreased the effectiveness of the education. Some requests included in the post‐intervention sample were made more than a month after referrers had completed the education. A study by Bell et al. [36], which measured the retention of knowledge from an online tutorial, demonstrated a considerable decrease in information recall, even after eight days. The authors attempted to mitigate the impact of this by providing a suite of visual reminders.

Similarly, the disruption to usual work processes caused by the limited availability of iodinated contrast may have been a distracting factor that limited the effectiveness of the education. During this time, extra steps were added to the requesting process, such as discussing the urgency of the scan and whether it could be performed without the use of contrast or even whether another modality was more suitable. All these added steps changed a regular task into a complex one.

Another limitation of this study was the mismatch of referrers who completed requests within the control and post‐education samples. As the referrers were different in these two groups, the quality differences observed may not be able to be directly attributed to the education. However, there were no other major changes during the study period that were likely to have influenced the quality of requests.

6. Conclusion

It appears that the use of IMPACT Education can improve the quality of clinical information provided in requests by ED referrers. The difference in referrers' baseline perceptions of radiologists' clinical information needs versus radiologists' actual needs for clinical information may be an explanation for the results of our study. The next phase of this project will investigate the impact of this improvement in request quality on the quality of resultant radiology reports.

Funding

This work was supported by the Australian Society of Medical Imaging and Radiation Therapy.

Conflicts of Interest

The authors declare no conflicts of interest.

Data Availability Statement

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.