What influences clinician’s satisfaction with radiology services?

Abstract

Aim

PACS and teleradiology systems have led to marked changes in the traditional relationship between referring clinicians and hospital radiology departments. The aim of this study was to assess which factors influence clinicians’ satisfaction with modern radiology services.

Method

An Internet-based survey questionnaire was sent to all referring clinicians within a large hospital network.

Results

Fifty-eight percent of 316 clinicians responded to the survey. Seventy percent felt PACS installation had improved reporting time, and 56% felt it had improved working patterns for medical staff. Approachability of radiologists was the only factor significantly associated with increased satisfaction (p = 8 × 10–8). A number of factors were found to be significantly associated with the perceived value of radiology reports, and these are discussed. An increase in clinicians’ confidence in their own radiological skills was not associated with a decrease in the value they placed on radiology reports.

Conclusion

The only factor significantly associated with improved clinician satisfaction was the availability of an approachable radiology service. Availability of PACS did not appear to undermine the value placed on radiology reports.

Introduction

Providers of radiology services have a responsibility to audit their performance on a regular basis and to address any deficiencies that are identified as a result [1].

This raises the question of how a radiology department should audit its performance. A traditional approach to this has been through performance parameters—similar to those used in financial management [2–4]. Overall, when assessing the performance of a radiology department six indicators are most commonly used: productivity, reporting time, ease of access, finance and satisfaction (of both clinicians and patients) [5]. In this respect a department’s performance can be partly gauged by these ‘dashboard’ indicators [6]. ‘Dashboard indicators’ are intended to give a quick view of organisational performance. The name is derived from the example of the dashboard of a vehicle, which can at a glance provide an idea of overall function and act as a means of identifying areas for further assessment.

In recent years, the installation of PACS and teleradiology, and rapid provision of reports using speech recognition software, have resulted in marked changes in the traditional interactions between referring clinicians and their local radiology departments [7–9]. It is clearly important to ensure that the introduction of these systems does not have any negative effect on clinicians’ satisfaction with the service. In addition, the potential for images and reports to be disseminated widely could mean that clinicians become less likely to discuss the results of investigations with radiologists.

PACS installation across the three hospital sites comprising the Belfast Trust was completed in 2009. This provided an opportunity to assess any effect on clinical satisfaction with the radiology service, in the context of a wider assessment forming part of the departmental quality assurance programme.

Methods

Data were collected from three hospitals within the Belfast NHS trust (catchment population 1.6 million): the Royal Victoria Hospital, the Belfast City Hospital (including the Northern Ireland Cancer Centre) and the Royal Belfast Hospital for Sick Children. All specialties, comprising 316 consultant clinical specialists, were included in the survey group.

The research committee for the trust was contacted, and ethical approval was deemed unnecessary for this study. With regard to the means of distributing the questionnaire, a number of studies have shown that e-mail surveys are associated with a smaller response rate than traditional mail surveys [9, 10]. However, combined e-mail surveys, with telephone follow-up, have been found to be the most efficient means of conducting a survey [11]. A response rate over 50% was chosen as satisfactory, reflecting the mean response rates of physician surveys in the literature [9]. A reminder was sent to all staff after a period of 2 weeks. Provision was made to contact non-responders by telephone to achieve an adequate response rate; however, a single reminder by e-mail was sufficient to achieve this.

The subscription to the survey engine was paid for by the PACS provider for the Royal Victoria Hospital site. Neither this company nor the other PACS providers were consulted in the construction of the survey, and are not represented amongst the authors.

The survey questions were divided into five areas: responders demographic information, inpatient work, outpatient work, radiology systems (i.e. ease of access, on call service and PACS) and overall impression (Appendix 1).

Chi-square and Fisher’s exact test were used to determine if associations identified between responses were statistically significant. Where negative responses were obtained, these groups were subjected to further investigation to try and identify a cause.

Results

A total of 185 responses were obtained from 316 surveys originally sent out. This represents a response rate of 58%. All specialties were represented in the response group with only dentistry (25%) and oncology (10%) having a less than 50% response rate.

Inpatients

A total of 117 (69%) respondents were either satisfied or very satisfied with the average waiting time for inpatient investigations, with only 10% reporting that they were dissatisfied with waiting times (Fig. 1). Eighty-nine percent of respondents stated that they were either satisfied or very satisfied with the approachability of radiology staff when requesting investigations; 93% reported that radiology staff positively expressed interest in discussing clinical cases with referring clinicians. Only one respondent (0.54%) felt that radiology staff had no interest in the clinical information regarding patients. Four respondents (2.2%) felt that the radiology staff was unapproachable.

Fig. 1.

Histogram showing satisfaction with waiting time for inpatient investigations

Outpatients

Fifty-four percent selected either ‘satisfied’ or ‘very satisfied’ with waiting times for outpatient investigations; 27% were ‘neither satisfied nor dissatisfied’. However, 19% reported ‘dissatisfaction’ with outpatient waiting time (Fig. 2).

Fig. 2.

Histogram showing satisfaction with waiting time for outpatient investigations

Further analysis of the outpatient responses identified a significant association between increased usage of plain radiography (p = 0.0005), ultrasound (p = 0.006), CT (p = 0.007) and MRI (p = 0.02) and increased satisfaction. This association was not found in inpatient investigations, although the overall high level of satisfaction with inpatient investigations meant that there were not enough dissatisfied responses to carry out statistical analysis.

Radiology systems

When assessing satisfaction with the PACS system, the most frequent complaint reported was the limited availability of workstations, rated as either fair or poor by 23%. While the majority (68%) reported that workstation availability was either excellent or good, the amount of clinicians dissatisfied was unexpected and prompted further investigation. However, this found that there was no departmental clustering of responses, such as would allow a focused increase in provision of workstations.

Clinicians were asked to state whether they used PACS for viewing reports, images or both. Radiology reports were viewed with a similar frequency to images (59% vs. 64%).

Seventy percent reported that PACS had improved the speed of reporting, with 57% reporting that it had improved the working patterns of junior medical staff.

With the increasing tendency toward shift work in medicine in general, the authors felt it was important to gauge the satisfaction with the on-call service provided by radiology. Seventy percent responded positively that the ease of access to a radiologist was either good or very good; 24% were neutral, and only 6% stated that it was poor. Eighty-two percent felt the radiologist showed a positive interest in the clinical history, with 76% reporting that radiologists were keen to discuss reports with the referring clinical teams.

Overall impression

Ninety-four percent of respondents felt that, in general, radiology reports were either ‘informative and decisive’ or provided ‘a useful list of differentials’. Six percent felt that radiology reports were vague and/or of no value to clinical teams. The survey also asked respondents to grade their own perceived ability to interpret the investigations they would most commonly request. Sixty-nine percent rated their ability as either good or excellent (Fig. 3). Analysis with Pearson’s chi-square test and Fisher’s exact test showed no association between perceived ability to interpret investigations and age or specialty.

Fig. 3.

Pie chart showing results for clinicians” perception of their own ability to interpret radiological investigations

Finally respondents were asked to give an overall rating for the radiology service in the trust and to provide free text comments if desired. Ninety-one percent rated the radiology service as either good or excellent. A single respondent felt the service was below average. No responses rated the service as poor.

Further analysis

A statistically significant association was identified between value of radiology reports and the length of time a respondent had been a specialist (p = 0.026). The pattern displayed was that a greater number of junior specialists (<5 years in post) than expected had a low opinion of the value of radiology reports, whereas a greater number of senior specialists (10 years+) than expected valued radiology reports highly (Table 1).

Table 1.

Perceived value of radiology reports versus length of time as a consultant

Value of radiology reports versus length of time as a consultant
Opinion of Radiology Reports	Length of time as Consultant	Less than 5 years	5–15 years	15+ years
Very informative and decisive		15	33	25
Useful list of differentials		21	33	15
Little value		6	1	2
				p = 0.026

A statistically significant association was found between increased satisfaction with waiting times and an increase in the perceived value of the radiology report (p = 0.01).

No statistically significant association was found between a clinician’s perceived ability to interpret their own investigations and the value of radiology reports (p = 0.8).

Identifying factors that affect the overall satisfaction with the radiology service was the most important function of this study. Factors identified in advance to require further analysis were the effect of waiting times on satisfaction; approachability of staff; opinion of the PACS system, and if a respondent was dissatisfied with radiology, to assess the modality that they most frequently used. However, using chi-square or Fisher’s exact test for statistical analysis of this group was of limited value as, while 13 rated the service as average, only a single respondent gave a negative rating of the service. The authors looked for any associations between the above predetermined factors amongst the less satisfied respondents. Using Fisher’s exact test there was an association between approachability of staff and satisfaction (p = 8.78e⁻⁸) (Table 2).

Table 2.

Approachability of radiology staff versus overall opinion of the radiology service

Overall satisfaction versus approachability of radiology staff
	Overall satisfaction
Satisfaction with approachability	Excellent	Good	Average	Below average
Very satisfied	49	43	2	1
Satisfied	7	33	4	0
Neither	1	5	5	0
Dissatisfied	0	2	2	0
			Fisher’s exact test	p = 8.78e-08

Overall satisfaction was not related to their opinion of the PACS system or waiting time for investigations.

There was no association between the perceived value of the radiology reports and satisfaction with the radiology service, as six of the ‘less satisfied’ respondents in the group rated radiology reports highly (very informative or useful list), and two of the five respondents with a low opinion of radiology reports reported high satisfaction with the overall radiology service (excellent or good). The only positive association noted was that the dissatisfied group tended to be less frequent users of the radiology service. This did not achieve significance due to the small number in the dissatisfied group.

Discussion

The quality of a radiology department is obviously not measured by the satisfaction of the referring clinicians. Nevertheless, radiology is responsible for offering a service to patients, who are most commonly referred from clinicians. As such appreciating the service clinicians feel they require for safe and effective treatment of patients is part of providing this service.

The overall aim of this project was to assess what factors affected satisfaction with radiology services amongst clinicians. The end result of this analysis, covering multiple facets of clinicians’ interaction with radiology, was that only one factor had a significant association with overall satisfaction and this was the approachability of radiologists. This desire to be able to contact radiologists directly is topical given the increasing possibility to out-source radiology services provided by recent technological advances, as discussed in the recent Royal College of Radiologists document [12]. This finding indicates that removal of radiology to an off-site location may not reflect the desire of clinicians.

Overall, the results for this satisfaction survey were positive, especially as satisfaction surveys have been reported to underestimate satisfaction [11]. The response rate of 58% was higher than the average response rate of clinician surveys reported in the literature [9].

A potential problem with using e-mail to circulate the survey is that it is not possible to confirm that the recipients actively use their trust e-mail, and as such it is not possible to confirm that the survey was received. Nevertheless, the high response rate from almost all directorates indicated that no one group of clinicians was excluded.

Responders had an overall positive opinion of the impact of PACS installation, with the majority feeling it improved the reporting time (70%) and working pattern for medical staff (57%).

With the dissemination of radiological images to all clinicians via PACS there might be a risk of radiology reports being disregarded as clinicians become more adept at interpreting imaging. In fact, the opposite effect has been reported in the literature [13]. In our study, the similar frequency with which reports and images were viewed was one of a number of indicators that radiology reports have not been devalued by PACS systems and increased clinician access to investigations. Furthermore, an increase in a clinician’s perceived ability to interpret investigations was not associated with a decrease in the value placed on radiology reports. A positive association was found between increasing specialist seniority and increasing value of radiology reports, which might reflect the effects of increasing clinical experience. It is also notable that there was no devaluation of radiology reports with increasing confidence in imaging interpretation amongst clinicians.

Analysis of satisfaction with waiting times showed that, contrary to pre-survey expectations, this had no association with satisfaction with the overall service. There may be a number of reasons for this. Firstly, government targets in place limit the waiting time for outpatient investigations, which are considerably less than they have been historically. Secondly, inpatient investigations are, in general, processed rapidly if requested directly by the referring specialist, and as such they may have a slightly privileged view of the process.

Satisfaction with waiting times was associated with an increase in the perceived value of radiology reports. This finding may indicate that clinicians who are happy with the waiting time for investigations receive results within a useful time frame, so that these results can be used to treat patients. As timely reports will help guide treatment, a higher value would be placed on them by the clinicians.

The results also indicate that dissatisfied respondents tend to be those who use the radiology service less frequently. This may indicate that they have misplaced expectations of what to expect. The other interpretation is that dissatisfaction with the radiology service is leading clinicians to request investigations less often. The latter explanation seems less likely, as if a patient requires an investigation to be carried out, it is unlikely that a clinician would avoid requesting this out of prejudice.

Modality-specific analysis illustrated that, with regard to outpatient investigations, more frequent users tend to be more satisfied with the service and, in the case of plain films, to have a higher opinion of the value of radiology reports. This finding may indicate that frequent users have insight into the pressures placed upon radiology departments, and therefore have a more realistic understanding of what to expect.

Conclusion

Is this project able to shed more light on question posed in the title—what factors influence clinicians’ satisfaction with radiology services? Statistical analysis of the results obtained indicates that the only factor significantly associated with increased satisfaction amongst clinicians was having an approachable radiology service to interact with. This result is felt to be particularly topical given the changes currently underway in radiology practice worldwide, where there is often a drive to make radiology an ‘off site’ specialty. With the development of PACS and teleradiology, there is the potential for clinicians to become increasingly distanced from radiologists and less dependent on their reports. However, this study indicates that clinicians continue to value the local availability of radiologists with whom to discuss their patients, and that they also value their reports, especially if waiting times for investigations are acceptable. These considerations should be taken into account by those responsible for the provision of radiological services.

Appendix 1

Demographic Data:

(1) What directorate do you work in?

(2) How long have you been a consultant in this trust?

Inpatients:

(1) How often would you use the [radiology modalities/specialties provided at each site] for inpatient investigations?

(2) How would you rate your overall satisfaction with the waiting times for inpatient investigations? (choice of five from very satisfied to very dissatisfied)

(3) How satisfied are you with the approachability of staff in Radiology? (choice of five)

(4) In general, how interested are radiology staff in clinical information relating to patients?

Outpatients

(1) How often would you use [the modalities/radiology specialties provided at each site] for outpatients?

(2) How would you rate your overall satisfaction with waiting times for outpatient examinations? (choice of five)

Radiology Systems

(1) Do you have a log-in for the PACS system?

(2) How did you learn to use the PACS system? (choice of four: formal training, from a colleague, written instruction, never had instruction)

(3) How often would you use PACS to view the following?

- Images (choice of four from: daily; weekly; monthly; not Often)

- Reports (same options)

(4) How would you rate PACS for the following?

- Availability of workstations

- Quality of images

- Reliability

- Ease of use

(5) Do you feel PACS has improved…

- Speed of reporting

- Waiting time for investigations

- Working pattern of junior medical staff

(6) What are your opinions with regard to these aspects of the out-of-hours service provided by the radiology department?

- Ease of access to a radiologist?

- Radiologist interest in the clinical history?

- Radiologist interest in discussing the findings?

Overall Opinion

(1) How would you rate the usefulness of radiology reports in reaching a diagnosis? (choice of five)

(2) How would you rate your own ability to interpret the investigations that you most commonly request?

(3) On a scale of 1–5 (with 1 being least satisfied and 5 being most), please rate your satisfaction with the following [radiology modalities/specialties provided at each site]. (If you do not use a specialty please select N/A)

(4) Please rate your own satisfaction with the radiology department. (choice of five)

(5) If you have any other comments, please add them here.