Evaluation of a nudge intervention providing simple feedback to clinicians of the consequence of radiation exposure on demand for computed tomography: a controlled study

ABSTRACT

Computed tomography (CT) is readily available in developed countries. As one of the side effects includes an increased risk of cancer, interventions that may encourage more judicious use of CT are important. Behavioural economics theory includes the use of nudges that aim to help more informed decisions to be made, although these have been rarely used in hospitals to date. We aimed to evaluate the impact of a simple educational message appended to the CT report on subsequent numbers of CT completed using a controlled interrupted time series design based in two teaching hospitals in the UK. The intervention was the addition of a non-directional educational message on the risk of ionising radiation to all CT reports. There was a statistically significant reduction in the number of CT requested in the intervention hospital compared to the control hospital (-4.6%, 95% confidence intervals -7.4 to -1.7, p=0.002) in the 12 months after the intervention was implemented. We conclude that a simple, non-directional nudge intervention has the capacity to modify clinician use of CT. This approach is cheap, and has potential in helping support doctors make informed decisions.

Introduction

Computed tomography (CT) has undoubtedly been one of the major factors contributing to improvements in medical care over recent decades. As there is widespread variation in usage of diagnostic tests,¹ it is likely that some are unnecessary, which in the case of CT exposes the patient to avoidable ionising radiation. The estimates of the magnitude of this risk are uncertain; an estimate from 2007 suggested that use of CT in that year may account for 1.5 to 2% of new cancers in the USA,² although this would be expected to decrease as newer generation CT machines deliver less ionising radiation exposure. Clinicians are not always aware of the long-term adverse health consequences of CT,^3,4 suggesting that this is an area with potential for improvement. It was recently demonstrated that there is almost a three-fold increase in the risk of receiving a CT during the period of transition from paediatric to adult medical care,⁵ and one interpretation of these data is that paediatric clinicians may be better in using alternatives to CT than their adult counterparts.

Nudge theory is a category of behavioural economics that aims to use non-directional interventions to modify decision making.⁶ This generally aims to use opportunities to provide information or context that may allow better informed decision making, at a relatively low cost in terms of money or time. We hypothesised that providing referring clinicians with a simple statement about the consequences of radiation exposure may modify their subsequent decision making with regard to future use of CT. We tested this hypothesis in a busy teaching hospital in the UK using a prospective, controlled study design using the demand for CT as the primary outcome measure.

Methods

Study design

We used a controlled interrupted time series design.

Study population

The setting was Royal Derby Hospital, UK (RDH), with Nottingham University Hospitals, UK (NUH) as a control hospital being the nearest large hospital serving a similar population. The Royal Derby Hospital is a busy acute medical hospital that has 1,100 beds.⁷ The study was an evaluation of a health service modification and no ethical approval was required.

Intervention

The intervention comprised the addition of the message below to the bottom of all inpatient and outpatient paper and electronic CT reports. It was designed to highlight the type of patient who is most at risk after exposure to ionising radiation.

Message from the executive medical director: ‘Did you know that a chest, abdo and pelvis CT scan in a 20 year old female population is associated with approximately a 1 in 300 risk of subsequent cancer? The equivalent risk is much lower in 90-year-old men (less than 1 in 3000). Is there an equally effective alternative investigation that does not involve ionising radiation? If so, have you discussed all of the alternatives with your patient?’ http://www.xrayrisk.com/index.php

Outcome measures

Data on the number of weekly CT between February 2016 and February 2018 were collected, covering a period 12 months before the intervention was implemented to 12 months afterwards in RDH and NUH. A sensitivity analysis omitted CT of the head as these involve a lower ionising radiation dose.

Statistical analysis

Segmented regression was used for the interrupted time series analysis, using a generalised linear model and assuming a negative binomial error distribution, that is, we fitted a model which was appropriate to count data with ‘overdispersion’ where there is greater variability in the data than that normally expected in count data. We used the package tscount in R statistical software.⁸ We initially fitted a full model for a controlled interrupted time series, with:

• terms for the pre-intervention trend – to allow for any underlying baseline trend in CT rates

• baseline difference between hospitals – to allow for a difference in mean CT rates between hospitals at baseline

• difference in the pre-intervention trend between hospitals – to allow for any difference in the underlying trend between hospitals

• change in level for NUH – to allow for any change in CT rates in the control hospital post intervention due to confounding factors

• change in trend for NUH – to allow for any change in trend post intervention in the control hospital due to confounding factors

• difference in the post-intervention change in level between RDH and NUH – our measure of the effect of the intervention on the mean level of CT rates in the intervention hospital compared to any change in the control hospital

• difference in the post-intervention change in trend between RDH and NUH – our measure of the effect of the intervention on the trend in CT rates in the intervention hospital compared to the control hospital.

We therefore hypothesised an impact model for the intervention which might involve an immediate change in level or a gradual trend change in CT counts in RDH, which were tested by the last two parameters described above. A dummy variable was fitted to allow for the annual drop in CT numbers over Christmas.

We also fitted parsimonious (simplest) models including only parameters that were statistically significant, identified by backward elimination, removing those terms consecutively that were least significant until all parameters were significant at the 5% level and no parameters were significant at the 5% level when added to the model.^9,10 Models were checked for autocorrelation (that is, for the fact that observations near in time may be correlated). An autoregressive process of order 4 was added to the final model to allow for each observation to be correlated with the previous four observations.

Results

The weekly numbers of CT in RDH (intervention institution) and NUH (control institution) are shown in Fig 1. Rates were slightly higher in NUH than RDH, but the underlying baseline trends were comparable.

Fig 1.

Weekly total computed tomography for Royal Derby Hospital (interventional institution) and Nottingham University Hospitals (control institution), before and after the intervention. Intervention implementation in February 2017 shown as a dotted line.

The results of the resulting segmented regression analysis are shown in Table 1. In the full model, there was a significant difference in numbers of CT at baseline between hospitals, and a significant trend for increasing rates of CT at baseline, but no significant difference in the baseline trend between hospitals (p=0.8), and the level (p=0.055) and trend (p=0.3) for NUH did not change significantly post-intervention. There was also no significant difference in the change in trend post-intervention (p=0.3), in other words there was no significant effect of the intervention on the trend in CT rates.

Table 1.

Change in the number of computed tomography completed at Royal Derby Hospital (intervention institution) compared to Nottingham University Hospitals (control institution) after the informational feedback intervention was implemented. A dummy variable for the Christmas week was also included in the model. Only statistically significant variables are included in the parsimonious models.

	Rate ratio (95% CI) for all CT (full model)	Rate ratio (95% CI) for all CT (parsimonious model)	Rate ratio (95% CI) for CT other than head (full model)	Rate ratio (95% CI) for CT other than head (parsimonious model)
Pre-intervention difference between RDH and NUH	0.799 (0.753–0.847) p<0.001	0.811 (0.773–0.852) p<0.001	0.779 (0.727–0.835) p<0.001	0.794 (0.752–0.839) p<0.001
Pre-intervention trend (slope) for control NUH (per week)	1.001 (1.000–1.002) p=0.029	1.001 (1.001–1.002) p<0.001	1.001 (1.000–1.002) p=0.14	1.002 (1.001 to1.002) p<0.001
Pre-intervention difference in trend between RDH and NUH	1.000 (0.999–1.002) p=0.8	ns	1.000 (0.999–1.002) p=0.6	ns
Change in level post-intervention for control NUH	1.047 (0.999–1.097) p=0.055	ns	1.052 (0.993–1.114) p=0.08	ns
Change in trend post-intervention for control NUH	0.999 (0.998–1.001) p=0.3	ns	1.000 (0.998–1.002) p=0.7	ns
Difference in the change in level between RDH and NUH	0.904 (0.847–0.965) p=0.002	0.954 (0.926–0.983) p=0.002	0.879 (0.811–0.953) p=0.002	0.949 (0.915–0.985) p=0.005
Difference in the change in trend between RDH and NUH	1.001 (0.999–1.003) p=0.3	ns	1.002 (0.999–1.004) p=0.2	ns

CI = confidence interval; CT = computed tomography; ns = variables were not significant and were not included in the final model; RDH = Royal Derby Hospital (intervention institution); NUH = Nottingham University Hospitals (control institution).

In the final parsimonious model, there were 29% fewer CT carried out in RDH than NUH pre-intervention (p<0.001), and a significant increasing trend in the number of CT at 0.1% increase per week in both hospitals. However, there was a statistically significant reduction in level of CT in RDH compared to NUH post-intervention of 4.6% (p=0.002). The results excluding head scans were very similar.

Discussion

Statement of principal findings

These data demonstrate that providing simple, non-directional feedback on the long-term consequences of exposure to ionising radiation is associated with a 4.6% decrease in the number of CT completed compared to the control hospital over 1 year.

Strengths and weaknesses of the study

This study has a number of strengths and limitations. The use of a prospective study design with a control group allowed the change in the number of CT delivered to be compared against the background of a trend of increasing demand for healthcare annually. The intervention was added to the radiology reporting system and the text of the message was placed underneath the CT report, which all clinicians see when reviewing the scan result. The message on the CT report was designed collaboratively within a multidisciplinary team and piloted, but we are unable to conclude that it is optimal, and there is likely to be room for improvement in both the content and presentation of the message. As a consequence of designing and delivering ‘nudge’ informational messages in the context of health services research, there were a number of factors that required to be taken into consideration such as the opinions of important stakeholders, such as the radiologists, and the limitations of the institutional information technology systems. The data that the educational ionising radiation message was based on are necessarily population based, as it is impossible to provide precise individual level data on radiation risk. The key assumption in the controlled interrupted time series is that the change in the level and/or trend in the outcome variable is presumed to be the same for both the control group and, counterfactually, for the treatment group had it not received the intervention. These hospitals were reasonably similar at baseline with similar increasing trends in CT at baseline, no significant change in level or trend in the control group at the point of the intervention, and we are not aware of any other reason to expect differences in the change in levels or trends over time between the hospitals other than the intervention. We are unable to exclude the possibility of a second intervention being implemented at the same time as our educational one, but as most of the study team work in RDH, this is unlikely. We have not conducted any sex or age stratified secondary subgroup analyses, as the primary hypothesis was that there was be a change in demand for CT at the institutional level, and we wished to avoid the risk of multiple hypothesis testing.

As there were 52 paired data points before and after the intervention, we have refrained from secondary data sub-analyses, to avoid the problem of multiple hypothesis testing within a limited number of data points, which is also relatively underpowered compared to our primary analysis. These findings represent a positive proof-of-concept study, and demonstrates that the ionising radiation educational message intervention is one that can be implemented, within the context of the UK health service. The definitive study would be a larger, randomised controlled trial of more healthcare institutions to provide proof that this intervention is generalisable to other similar hospitals. This would be a large undertaking, that would require substantial resources to deliver.

Meaning of the study: possible explanations and implications for clinicians and policymakers

Nudge theory has potential in helping doctors make more informed decisions when choosing the optimal imaging approach for their patients; weighing up the immediate benefits of diagnostic knowledge against the longer-term consequences. As the information provided is non-directional, and simply informative, the doctors’ clinical autonomy is not infringed upon, as it is added to the context of many years’ medical training and experience for many UK healthcare teams. Providing the information after a scan has been completed and reported informed future practice, and had been demonstrated to be effective in modifying physicians’ behaviour with regard to use of blood tests after cost feedback was provided with the results.¹¹ This approach was preferred to the alternative of delivering the information at the time of deciding to do the scan, when it could impact on the efficient delivery on clinical care. In the UK, the decision to request a CT for a patient is generally made in a face-to-face discussion with the patient, when the benefits and costs can be weighed up, and a collective decision made between the physician and the patient. The CT examination is then subsequently requested, when the decision to scan has already been made. As a consequence, feeding back an educational message on the longer-term health effects of exposure to ionising radiation was the only viable intervention in this context, although point-of-care interventions can be implemented for other healthcare interventions, and have been effective in increasing prescribing of cheaper antibiotics when the price was presented at the point of antibiotics selection.¹²

Unanswered questions and future research

In summary, we have demonstrated that adding a simple educational ionising radiation awareness message to CT reports was associated with a 4.6% decrease in subsequent demand for CT compared to the control group or approximately 45 fewer CT delivered per week or 2,340 fewer CT over a year. This nudge approach has potential to improve patient care when used prudently. However, it is unclear in which settings it will be efficacious and more controlled intervention studies are required across a broad range medical areas to develop the evidence base. We anticipate that informing physicians of the long-term risks of ionising radiation is unlikely to negatively impact on healthcare delivery, but future studies should consider both the benefits and the risks of such interventions.

Funding

Funding for this study was provided by The Health Foundation.