Organizational Factors and Quality Improvement Strategies Associated with Lower Radiation Dose from Computed Tomography Exams

Abstract

Objective:

To identify organizational factors and quality improvement strategies associated with lower radiation doses from abdominal computed tomography (CT).

Methods:

Cross-sectional survey of radiology leaders along with simultaneous measurement of CT radiation dose among 19 healthcare organizations with 100 imaging centers throughout the U.S., Europe, and Japan with a common dose management software system. After adjusting for patient age, sex, and size, quality improvement strategies were tested for association with mean abdomen CT radiation dose and the odds of a high dose exam.

Results:

Completed surveys were received from 90 (90%) of imaging centers and 182,415 abdomen CT scans were collected during the study period. Radiation dosages varied considerably across organizations and centers. Univariate analyses identified eight strategies and systems that were significantly associated with lower average doses or lower frequency of high doses for abdominal CT exams: tracking patient safety measures, assessing the impact of CT changes, identifying areas for improvement, setting specific goals, organizing improvement teams, tailoring decisions to sites, testing process changes before full implementation, and standardizing workflow. These processes were associated with an 18–37% reduction in high dose examinations (P<.001–.03). In multivariate analysis, having a tracking system for patient safety measures, supportive radiology leaders, and obtaining clear images were associated with a 47% reduction in high dose exams.

Conclusions:

This documentation of the relation between quality improvement strategies and radiation exposure from CT exams has identified important information for others interested in reducing the radiation exposure of their patients.

Summary sentence:

Improvement in radiation doses from CT exams requires a conscious organized effort to improve the processes and systems used to deliver these services.

Computed tomography (CT) exams have provided clinicians with a substantial improvement in their ability to make diagnoses. However, there is enough concern about over usage that the American Board of Radiology Foundation convened a summit meeting of 60 organizations in 2009 to identify causes and solutions.¹ Overutilization is a problem for controlling health care costs, but unnecessary exams also expose patients to high radiation doses, increasing the risk of later development of cancers.^2,3 In one study of nearly one million people followed for over 3 years across the U.S, 70% of people had at least one radiation-exposing imaging procedure, with 20% receiving at least moderate radiation doses.⁴ Importantly, this radiation exposure risk is also subject to wide variation in dosage across different procedures and institutions. In a study of 151 organizations across seven countries, abdominal CT exams had a fourfold range in mean effective radiation (7.0–25.7 mSv) and a 17-fold range in the proportion of high dose examinations (4–69%).⁵

Such large variations in performance are widely recognized within the quality improvement field as an opportunity to identify the causes of variation in order to focus improvement efforts on those causes that are associated with more desirable outcomes.^6,7 Sistrom has published a conceptual framework for radiology improvement and Selby et al have shown that even when there is little variation, there may still be a large opportunity for improvement.^8,9 However, there has been little application of quality improvement methods in radiology, in part because information is lacking about the important contributory factors and effective change strategies for better care.

In order to explore the relationship between radiation dosage and the factors amenable to change, we enrolled 19 diverse radiology organizations with 100 separate imaging sites conducting CT exams from throughout the US as well as from Europe and Japan for a trial of different approaches to optimizing radiation doses. The goal of this paper is to identify the organizational factors and strategies that were associated with lower radiation exposures for patients in those centers at baseline, prior to any intervention.

METHODS

Recruitment:

Radiology healthcare organizations that used Radimetrics™ software (Bayer, Whippany NJ), a radiation dose management tool, were invited by email to participate in the trial. This inclusion criterion provided both an indication that the purchasing organization was interested in managing radiation dose exposures and a consistent means for collecting and measuring radiation doses across organizations in a standardized fashion. Overall, 19 diverse healthcare systems in 26 US states, England, Netherlands, Germany, Japan, and Switzerland agreed to participate (see Table 1). Most were academic/teaching systems, but one was a large network of freestanding imaging centers. Out of the total of 19 organizations, 6 included one participating center, and the others had 5–30 associated imaging centers. The institutional review boards at UCSF and collaborating organizations reviewed and approved the study or relied on the UCSF IRB to do so.

Table 1.

Description of Participating Organizations

^*Pediatric Hospital - data shown for patients aged ≥ 15 years

Data Collection

Organizational survey:

To collect descriptive information about each organization and imaging center as well as information about their general approach to managing CT services, we developed a short survey that included questions about each center providing CT exams within that organization. These surveys were to be completed by the organization’s principal investigator and a leader at each individual center. The questions asked about the organization’s location, structure, services, and patient population. They also asked about their CT protocols, role of medical physicists, and experience with optimizing CT radiation dosages. The surveys were distributed electronically via REDCap and sent via email to the PIs and site leaders. Follow-up began via email and continued via phone call until we had responses from each participating facility.

Implementation survey:

This survey was designed to identify how CT procedures were implemented within organizations and how each organization addressed the optimization of radiation doses for patients. The survey combined revised versions of two well-established surveys with questions specific to CT dose optimization and was designed to be completed by organizational or site leaders. The four components were:

A. Physician Practice Connection (PPC).

This questionnaire was originally created by the National Committee for Quality Assurance to assess implementation of the Chronic Care Model systems in primary care. It has been demonstrated to be reliably completed by a practice leader and to be associated with outcome measures and has been used in many federally-funded studies.^10–12 Some wording changes were made to focus on the presence of systematic approaches to managing CT exams and their associated radiation doses. It contained 14 questions about specific systems, asking whether a particular system is present and works well (1.0 point), present but needs improvement (0.5 points), or absent (0 points) and scored by as the percentage of possible points.

B. Change Process Capability Questionnaire (CPCQ), Part 1 on Practice Readiness:

This questionnaire was originally developed to measure the organizational factors identified by an expert panel as very important to a practice’s ability to improve care.^13,14 It has also been used in many federally-funded studies for a wide variety of topics. The 23 questions were answered on a 5-point agreement scale and scored to produce a total score as the percentage of possible points available.

C. CPCQ, Part 2 on Use of Change Strategies:

Developed in conjunction with Part 1 from the recommendations of an expert panel, these 16 questions assessed whether a practice has used each of 16 strategies for improving imaging services. Like Section A, it has 3 answer categories for Used and works well, Used but needs improvement, or No (not used) and a similar summary score is produced based on the percentage of possible total points achieved

D. Miscellaneous:

This component contained 9 questions about the organization’s CT protocols, quality improvement experience and use of the dose measurement software. One question asked about the organization’s priority for improving radiation dosage in relation to all other priorities. These questions were created to collect specific information and not to produce an overall score.

Survey Process

The survey was distributed electronically via REDCap email. If the survey was not completed, there were up to 3 reminders automatically sent every 8 days. For the purposes of this study, each imaging center was represented by a single survey response. For centers where we received multiple survey responses, the survey response for inclusion was selected using the following hierarchy: lead radiologist, other radiologist, lead technologist, other technologist, lead medical physicist, other medical physicists, administrator, and others/unknown.

Assessment of Radiation Dose

All analyses were adjusted for patient size, which was estimated using a mean water equivalent diameter of the scanned region. A radiation dose registry was created at UCSF to pool and store data on consecutive CT scans performed at collaborating organizations.⁵ Radiation dose data stored in Digital Imaging and Communications in Medicine (DICOM) format at each institution were exported to a local server directly from the CT machines or via the Picture Archiving and Communication Systems (PACS) used to review these exams. Data were stripped of patient identifying information other than study date and time, and transferred to UCSF in real time. Ongoing quality control performed at UCSF ensured that any gap in CT data submission was rapidly corrected and that missing studies were filled in. Any machine that recorded data for less than 10 abdominal CT scans during the study period was removed from the study. The total radiation dose imparted to a patient by the scanner was determined by the dose length product (DLP).This number can be converted to effective dose, a measure that takes into account the total imparted radiation and the future risk of cancer from this radiation, using a conversion factor of 0.015 for abdominal scans¹⁵

Analysis

The survey answers were scaled 0–1 for analysis. For questions whose available answers range on a five-point scale (strongly disagree, strongly agree, neutral, agree, strongly agree) the answers were translated into numerical values of 0 (strongly disagree), 0.25 (disagree), 0.5 (neutral), 0.75 (agree), and 1 (strongly agree). For questions whose answers range on a three-point scale, the answers were translated into 0 (absent), 0.5 (present but need improvement), 1 (present and works well). For questions whose answers are true or false, the answers were translated into 0 (false) and 1 (true). For questions whose answers may be given as any number, the responses were rescaled to vary roughly from 0 to 1, where extreme outliers are ignored in the re-scaling. Compound scores are computed for questions in sections CPCQ1, CPCQ2, and PPC by averaging all given survey responses for each respondent.

The primary outcomes were mean imaging center radiation dose, and the imaging center proportion of doses above the 75% at baseline. To study the effect on the mean dose of each survey question, as well as the compound scores of sections CPCQ1, CPCQ2, and PPC, we fit log-linear mixed models with the question or compound score of interest as the primary fixed effect and the DLP for each CT scan as the outcome. To study the effect of each survey question on the likelihood of a high dose study, we created a logistic mixed model where the outcome was whether the DLP was above the 75% at baseline, defined as a DLP of 1140 mGy-cm (corresponding to an effective dose of 17 mSv). In all models, patient diameter was included as a fixed effect and the specific machine on which the scan was performed is added as a random effect. For each log-linear model, we computed the expected reduction in mean dose in the population if all respondents strongly agreed. For each logistic model, we computed the expected reduction in the odds of high dose examination. For all questions but two, a higher-valued (i.e. – strongly agreeing) survey response would correspond a priori to a decrease in dose. For those two questions the survey responses were inverted to be consistent in direction.

We fit an additional multivariate log-linear mixed model to find the optimal strategy for lowering dose. This identified the survey answers that in combination were associated with the lowest doses. This was done via a forward-selection algorithm whose initial step was a log-linear mixed model with only the patient size as a fixed effect and the machine used as a random effect. Additional fixed effects are added in the form of survey question responses, with the metric for inclusion being the expected dose reduction to a patient in the study population if centers changed behavior to strongly agree with all questions in the model. A question was only added to the model if it had a significant impact on dose not accounted-for by questions already in the model, and the outcome of interest was the combined impact of all questions added to the model. A question was only eligible for inclusion in the multivariate model if it had a p-value of at least 0.05 in the univariate model and if its addition to the current model would be statistically significant. The forward-selection ended when the inclusion of additional questions did not lead to at least an additional 0.5% dose reduction. We fit a multivariate logistic mixed model using the same forward-selection algorithm.

RESULTS

Completed implementation survey responses were collected for 90 (90%) of 100 imaging centers and all 19 healthcare organizations, including survey responses from 5 lead radiologists, 40 non-lead radiologists, 6 lead technologists, 35 non-lead technologists, 3 administrators, and 1 respondent of unknown role. During the study period (November, 1, 2015 to October 28, 2016) 183,415 abdomen CT scans and their associated radiation dose measures were assembled from these sites. Table 1 provides descriptions for each of the 19 organizations. Most organizations identified as academic hospitals, but there was also one that contained a large network of outpatient imaging centers in 16 U.S. states.

Overall the mean radiation dose for abdominal CT was 788 milliGray-centimeters (mGy-cm, interquartile range 384 to 1040 mGy-cm, standard deviation 588 mGy-cm.). This corresponds to an average effective dose of 12.7 mSv (interquartile range 6.26 mSv to 16.3 mSv, standard deviation 9.7 mSv). The imaging center mean DLP varied from 355 mGy-cm – 1826 mGy-cm when all patients are standardized to be of median abdominal circumference. Overall the proportion of high dose examinations was 22.1%, and the imaging center proportion of high dose examinations varied from 2.1–78.9%.

The organizations with the lowest mean doses were either children’s hospitals or European, both of which are known to be more sensitive to radiation concerns. In univariate analyses (see Table 2), the summary scores on the implementation survey for both practice readiness and use of change strategies showed a significant relationship to the frequency of high dose examinations (reductions of 39% and 32% respectively, if all facilities were to strongly agree with all questions in these sections) but not to average dose. There were 14 individual questions that were significantly associated with radiation doses.

Table 2.

Survey questions that are significantly related to radiation dose in univariate analysis, the mean score for each question, the percent of imaging centers whose respondents strongly agreed or disagreed with the statement, and the percent reduction in average dose and probability of high dose examinations if all facilities strongly agreed with the question

Survey Questions‡	ean Score	Percent of Respondents who		Average Dose*		High Dose**
		Agree†	Disagree††	% Reduction p value		%Reduction p value
A. Practice Systems				0.3	0.96	17	0.34
We track patient safety measures (non-radiology)	55	36%	47%	10	.01	32	<.001
B. Practice Readiness				8	.17	39	.016
1.We assess the impact of CT changes made	73	71%	8%			31	.01
2.Rad leaders support dose optimization activities	83	87%	2%	7	<.001	27	<.001
3.We lowered CT dose in past year	83	85%	3%	7	.01	25	<.001
4.We review processes & identify areas for improvement	75	71%	8%	3	0.27	18	.03
5.Leaders of optimization are enthusiastic	69	72%	2%	5	0.13	23	.03
C. Change Strategies				7	.11	32	.03
1.We set specific goals for improving radiation dose and image quality	55	32%	24%	8	.04	37	<.001
2.We organize a team to improve CT dose	51	33%	31%	8	.01	33	<.001
3.We tailor decisions to site needs	66	44%	15%	8	.04	31	.02
4.We pilot test process changes before implementation	58	37%	23%	8	.04	30	.03
5.We standardized workflows to encourage dose optimization	83	65%	1%	5	.03	31	.01
D. Miscellaneous
1.Number of protocols	5	-	-	19	.01	38	001
2.Number of active QI projects on radiation dose		-	-	6	.01	22	.01
3.We obtain clear images of abdomen CT	92	92%	9%	2	<.001	8	<.001

^‡Abbreviated versions of the questions

^†Includes responses of “Yes, and Works Well” or combined responses of “Agree” and “Strongly Agree,” depending on question

^††Includes responses of “No” or combined responses of “Disagree” and “Strongly Disagree,” depending on question

Imaging centers that track any patient safety measures, that assess the impact of any CT changes that are made, that set specific goals for improving radiation dose, that organize teams to improve doses, that pilot test process changes before full implementation, and that standardize workflow to encourage dose optimization were associated with at least 30% reduction in high dose examinations in comparison to organizations that did not report these activities. Further, radiation leaders who support dose optimization and organizations with leadership enthusiastic about dose optimization had at least 20% fewer high dose examinations.

It also shows that both the total number of radiology protocols and the number of active quality improvement projects that are focused on optimizing radiation dose are significantly associated with both average dose and the frequency of high dose exams. Interestingly, the attitudes of radiologists (and rad techs), limited resources, and organizational stress were not related to either mean dose or the likelihood of high radiation doses. There was no significant relationship between the organization’s priority for optimizing dose and mean dose.

The multivariate analysis showed that three survey questions informed both mean dose and high dose reduction (see Table 3). This suggests that if all imaging centers reported tracking non-radiation measures of patient safety, had radiology leaders who supported dose optimization activities, and had clear images of abdominal CT exams (meaning the lower doses did not come at the expense of imaging quality), the average radiation dose would decrease by 12% and the probability of high dose exams would decrease by 47%.

Table 3.

Multivariate Analysis of Statements that are Significantly Related to Radiation Dose

Model	Source	Statement	Reduction in Dose If All facilities Strongly Agreed Percent (95% CI)
Average Dose	Practice Systems	We track patient safety measures (non-radiology)
	Change Capability	Rad leaders support dose optimization activities	12% (11.8%, 13.1%)
	Misc	We obtain clear images of abdomen CT
High Dose	Practice Systems	We track patient safety measures (non-radiology)
	Change Capability	Rad leaders support dose optimization activities	47% (44.5%. 49.6%)
	Misc	We obtain clear images of abdomen CT

DISCUSSION

In summary, this cross-sectional analysis of the relation between CT radiation exposure and various organizational activities and behaviors identified a number of factors strongly associated with both average and high radiation dose. Most importantly, the tracking of patient safety measures, and having radiology leadership who support dose optimization was associated with nearly a 50% reduction in high dose examinations. All of the identified factors can be modified and radiology organizations and imaging centers should consider these if they are interested in optimizing the radiation doses for their patients.

The most important factor (associated with both lower average dose and fewer high dose exams) is having support from radiology leaders for dose optimization activities. Virtually every study of quality in any type of organization has found that is the critical foundation for better quality. However, it is less clear why tracking patient safety measures and having clear images were part of the multivariate package. We suspect that these two factors are simply markers for broader aspects of achieving radiation safety goals. Tracking other safety measures was one of the least frequent activities, so it may mean that organizations doing that have taken an unusually broad approach to improving quality. On the other hand, it is likely that even strong leadership support will not be effective if it pushes dosage levels so low that radiologists can no longer be confident of their readings for images that are not sharp.

Several of the factors identified in the univariate analysis didn’t make it through the multivariate analysis, but this does not mean they were not important, but rather that they may have simply been markers for other unmeasured factors. These practices provide supportive evidence that the systematic approach to quality improvement used in other settings will probably also be effective for dose optimization in imaging centers. It is hard to imagine a serious quality improvement effort in any industry that would not start by identifying processes for improvement, setting specific goals, organizing a multidisciplinary team, combining standardization with tailoring change to site needs, pilot testing before implementation, and then assessing the impact of the changes made, all strategies that were identified as important in the univariate analysis.

Finally, it may also be helpful to know that imaging centers are able to achieve dose optimization regardless of radiologist or other staff attitudes and despite limited resources and organizational stress, none of which were associated with either average dose or frequency of high dose exams.

These findings may not be surprising to those experienced in quality improvement, but since most quality improvement initiatives in medicine have focused on primary care or inpatient services, it is helpful to have them confirmed in a new field. It may also be helpful to be aware that change in performance measures in other medical fields is generally slow, despite exhortation from enthusiastic leaders who want to change long-established traditions and care processes overnight. Our studies of change in performance over time in many primary care clinics as they become patient-centered medical homes have demonstrated the need for prolonged efforts and awareness that the rate of change will be slow, even when health care systems are exerting large efforts to improve.^16,17

LIMITATIONS”

The principal limitation of this study is the cross-sectional nature of the data. However, many of these organizations have been working on dose optimization for some time before this study and that was one of the reasons for their interest in participating. Thus, we can only say that at the time of the surveys, certain activities and factors were associated with lower dosages. More definitive relationships will require analysis of changes over time. We are also relying on the self- report from a single leader at each site who may have a limited understanding of some of the factors addressed and is not documentation of actual practice. However, in the original survey’s validation studies, self-report correlated fairly well with an independent audit.¹⁰ Finally, these participating organizations are probably not representative of all imaging providers, both in being larger and more interested in addressing this topic (as indicated by their purchase of Radimetrics software as well as agreeing to participate in this study).

It is important to note that the primary outcomes of interest in this paper are the expected impacts of survey questions on population dose, not individual dose. That is, our outcomes do not seek to compare a theoretical imaging center who disagrees with survey questions with a theoretical imaging center who agrees with survey questions, but rather illustrate the opportunity available in the population of imaging centers for improvement if they all changed from their current behaviors to strongly agree with survey questions. Under this paradigm, a survey question which is highly associated with dose levels, but which most imaging centers already agree with, would be considered “low impact,” in that there are few opportunities for the population to improve. On the other hand, a survey question which has comparatively modest association with dose, but whose responses vary greatly across imaging centers, presents a high opportunity for the population to improve, and is thus “high impact.” These findings are consistent with what has been demonstrated in other settings and fields and topics, so they should be helpful to those interested in addressing the topic of dose optimization in radiology. In particular, they highlight again that improvement in radiation doses from CT exams requires a conscious organized effort to improve the processes and systems used to deliver those services. Improvement will not come from efforts to change the attitudes, knowledge, and behaviors of individual health care personnel, even though some attention to those factors may be needed to facilitate the uptake and use of more systematic changes.

Take-home points:

• There is wide variation in radiation doses from the same CT exams among leading radiology organizations

• Fourteen different changeable factors were significantly associated with lower radiation doses

• Statistically speaking, if all imaging centers in this study tracked non-radiation measures of patient safety, had radiology leaders who supported dose optimization activities, and had clear images, the average dose would decrease by 12% and the probability of high dose exams would decrease by 47%