Improving education and supervision of Queensland X‐ray Operators through video conference technology: A teleradiography pilot project

Abstract

Introduction

X‐ray Operator (XO) supervision in Queensland is performed by radiographers in a site removed from the XO site. This has historically been performed by telephone when the XO requires immediate help, as well as post‐examination through radiographer review and the provision of written feedback on images produced. This project aimed to improve image quality through the provision of real‐time support of XOs by the introduction of video conference (VC) supervision.

Methods

A 6‐month pilot project compared image quality with and without VC supervision. VC equipment was installed in the X‐ray room at two rural sites, as well as at the radiographer site, to enable visual and oral supervision. The VC unit enabled visualisation of the X‐ray examination technique as it was being undertaken, as well as the images produced prior to transmission to the Picture Archiving and Communication System (PACS).

Results

Statistically significant improvement in image quality criteria measures were seen for patient positioning (P = 0.008), image quality (P < 0.001) and diagnostic value (P < 0.001) of images taken during this project. No statistically significant differences were seen during case level assessment in the inclusion of only appropriate imaging (P = 0.06), and the inclusion of unacceptable imaging (P = 0.06), however improvements were seen in both of these criteria. The survey revealed 24.6% of examinations performed would normally have involved the XO contacting the radiographer for assistance, although, assistance was actually provided in 88.3% of examinations.

Conclusion

This project has demonstrated that significant improvement in image quality is achievable with VC supervision. A larger study with a control arm that did not receive direct supervision should be used to validate the findings of this study.

Introduction

X‐ray Operators (XOs) are employees of a health service provider in Queensland, who do not hold formal qualifications in diagnostic radiography. They are licensed to perform a limited range of radiographic imaging in rural and remote locations, usually in addition to their primary role. XOs perform essential roles in locations which have infrequent or low levels of X‐ray demand and are used in place of, or in the support of radiographers. The locations in which XOs can be employed is controlled by need and managed by the Radiation Health Unit of Queensland Health.1

Supervision of XOs in Queensland is a requirement of their radiation Use Licence; it is usually performed via the telephone due to the remote location of the supervisor from the supervisee. The lack of a visual component with telephone communication makes supervision of the highly visual and varied set up of an X‐ray examination difficult.

Smith and Fisher2 report that a considerable proportion of remote XOs do not feel competent to perform radiography. This perceived lack of competence is reinforced by the radiography community with a long held view that XO performed radiography provides lower image quality than that performed by a radiographer.3 XOs report feeling that they need continuing education in radiography, in addition, many rural health professionals feel that their remote location and difficulty being released from their primary role reduces their exposure to face to face education.2, 4 Telehealth is suggested as a solution for rural health professionals to access continuing education and ongoing training.5, 6, 7

Telehealth is defined by Queensland Health as the delivery of health services and information, using telecommunication technology, such as email, telephone and video conference (VC). The availability and use of video conferencing is increasing with over 4000 telehealth systems now being available in over 200 hospitals and community facilities throughout Queensland.8 Clinical telehealth encounters typically “ … involve a patient, … and at least one health‐care provider … ” however, telehealth by its very definition can also be used for the sharing of information between healthcare professionals, such as for the purposes of supervision, training and education.9

The use of video‐based telehealth (such as video conferencing) for educational purposes has progressed from the instructional or lecture style of material delivery to a more interactive teaching method to at least supplement, if not replace, face to face interactions. Telementoring is defined by the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES)10 as the real‐time interactive teaching of techniques by an expert surgeon to a student not at the same site, while Kramer and Demaerschalk5 define it as guidance from a distance using virtual classrooms. Telepresence is defined as being the use of an audio visual platform which can manoeuvre around the environment, creating a sense of the remote supervisor ‘being present’ in the room.5 Using these principals, the term teleradiography is introduced in this study as the real‐time interactive teaching and supervision of radiographic techniques via a remotely operated video‐based telehealth system.

It is suggested that the key to improving XO image quality is through the introduction of real‐time supervision of XOs in their own work environment. Teleradiography is proposed as a solution to this problem through telepresence which has shown to be as effective as in‐person supervision in the delivery of supervision and guidance of colleagues.11 This would increase remote healthcare workers’ access to face to face supervision and reduce the time away from their site. With no evidence of such techniques being currently used in the field of radiography, this project comprised a pilot involving three sites to test the hypothesis without the need for a large financial investment for the installation of the equipment in multiple sites.

This pilot project aimed to determine if XO acquired X‐ray images could be improved through VC supervision as compared to the traditional telephone supervision methods. The project involved assessment of XO image quality, before and after the use of VC supervision during image acquisition. The study also used surveys to collect data on self‐reported confidence in the performance of radiography technique.

Methods

Study setting and design

Ethical approval was granted for the project by the Prince Charles Hospital Human Research and Ethics Committee. Informed written consent was obtained from all Queensland Health staff and patients directly involved in the project.

The 6‐month pilot project, conducted from January to June 2012 comprised of a retrospective image review of XO performed X‐ray examinations with and without VC supervision at two rural Queensland hospitals. All XOs working at the hospitals consented to participate in the study and were allocated an individual identifying code to be used during the project. The existing supervision arrangements of the two hospitals were maintained, whereby four radiographers at a single regional hospital supervised the rural XOs.

All general X‐ray examinations performed by XOs at the rural hospitals on Monday through Friday between 8am and 4pm were supervised via VC during the 6‐month project. If a participating radiographer was not available to oversee the examination during these times, the examination was performed under the existing telephone support system and excluded from the study.

Two cart‐based Cisco TelePresence Quick Set C20 (Cisco, San Jose, CA, USA) VC systems were purchased for the project, see Figure 1. These were installed in the X‐ray rooms at the rural XO sites with Video Graphics Array (VGA) connection to the computed radiography plate reading computers, permitting the VC transmission of X‐ray images.

Figure 1.

X‐ray room with video conference supervision set up.

Cost of purchase and installation of the two units, including additional power and data point installation and VGA connections, were $17,370.30 and $17,823.50. The supervising radiographers used a desk top Cisco TelePresence EX60 (Cisco, San Jose) VC system. Purchase of this unit cost $8468 with no additional installation costs incurred.

Data collection

Image assessment

Two image assessment tools were developed for use during the project to measure image quality factors and diagnostic value; these included separate individual image assessment and case image assessment tools. The term ‘image quality’ is used in this project as a measure of optimal image acquisition rather than just the physical parameters of digital images such as resolution, noise and artefacts. To the best of the researcher's knowledge, no widely accepted image assessment tool existed at the time of the research which would enable evaluation of radiographic performance of X‐ray images. The assessment tool created needed to provide a relative rating of performance so as to measure change. The image assessment tool created uses a number of 0–10 visual analogue scales to measure image criteria, such as in the following:

• Required anatomy

• Patient positioning

• Appropriate collimation

• Image quality (including exposure), and

• Overall diagnostic value for identifying pathology.

The X‐ray images were assessed, using the above tools by a senior trauma radiographer with over 7 years experience in supervision and training; they were not blinded to whether the images were pre‐ or post‐intervention.

The pre‐intervention data comprised of retrospective studies performed by each XO immediately prior to project commencement. XO examinations conducted under VC supervision during the pilot formed the post‐intervention data set, with those performed without VC supervision being excluded from the project.

Examination surveys

All XOs completed short paper‐based examination surveys at the conclusion of all VC supervised examinations; the surveys recorded confidence levels in performing the X‐ray examination, case difficulty, perceived need for assistance and assistance actually received. These surveys were mailed back to the investigator for collation and analysis.

Data analysis

The data was analysed using IBM's Statistical Package for the Social Sciences (SPSS) Software (version 22). Independent sample t‐tests were used as X‐ray examinations were independent pre‐ and post‐intervention. Chi‐squared tests were used to assess if video conferencing had an effect on binary outcomes, Fischer's exact tests were used when expected cell counts were low. Mean and standard deviation values were reported, P values of less than 0.05 were considered significant. Descriptive statistics were used where inferential testing would not provide meaningful results.

Results

Image assessment results

All nine XOs involved in the project held a chest and extremities (rural and remote – extended) Use Licence or its trainee precursor licence issued by the Radiation Health Unit. Their range of experience was from less than 12 months (trainee licence holder) to 16 years, with an average of 6.2 years.

Each XO performed between 0 and 21 examinations in each arm of the pilot. Pre‐intervention saw the inclusion of between 4 and 21 examinations per XO and post‐intervention examination numbers ranged from 0 to 16 per XO. A total of 155 X‐ray examinations (consisting of 326 images) comprised the pre‐intervention data set. During the project period 148 X‐ray examinations (234 images) were performed, however, only 79 examinations (164 images) were supervised via VC, which comprised the post‐intervention data set. There were a variety of reasons why VC supervision was not provided, including the lack of radiographer availability to supervise (n = 21), the examination being performed out of supervision hours of the pilot (n = 42), the examination being performed with a mobile X‐ray machine (n = 3), simultaneous use of the supervisor VC system (n = 1), VC connection issues (n = 1), and one patient was unable to provide informed consent.

All measured aspects of individual image quality significantly improved through VC supervision. Assessment of the included anatomy required on images saw the least significant improvement; with image quality and diagnostic value of acquired images demonstrating the most significant improvements through VC supervision. Table 1 depicts the results for each measured image criterion.

Table 1.

Results of X‐ray image assessment with and without video conference (VC) supervision

Variable	Mean (standard deviation)		t‐value	P‐value
	Without VC supervision	With VC supervision
Required anatomy included	6.4 (2.9)	6.9 (2.4)	−2.12	0.034
Patient positioning	6.7 (2.3)	7.2 (2)	−2.65	0.008
Appropriate collimation	5.5 (1.8)	5.9 (1.4)	−2.86	0.004
Image quality (including exposure)	6.3 (1.6)	7.0 (1.0)	−5.94	<0.001
Diagnostic value of image	6.0 (2.4)	6.9 (2)	−4.25	<0.001

On a case level, when images were assessed, the overall quality and diagnostic value of images demonstrated significant improvement with VC supervision. Overall case quality scores significantly improved from a mean pre‐pilot score of 5.3 (SD = 2.3) to a mean pilot score of 6.4 (SD = 1.8), t(167.749) = −4.10, P < 0.001. The overall diagnostic value of the images within each case attained significantly higher scores through VC supervision, mean pre‐pilot scores of 5.7 (SD = 2.3) increased with the use of VC supervision to 7 (SD = 2), t(156.668) = −4.41, P < 0.001.

The association between VC use and the likelihood of all appropriate projections being included for the case was of borderline statistical significance, χ ² = 3.0, P = 0.058. Despite this, a pattern of results indicate that a greater proportion of cases included all the appropriate views in post‐intervention data (88.7%) compared to pre‐intervention results (79.2%). The inclusion of unnecessary images decreased with the use of VC supervision with 5.2% of cases in the pre‐intervention data reducing to 1.4% of cases in the post‐intervention data. However, no statistically significant association was demonstrated between VC supervision and the inclusion of unnecessary images, χ ² = 1.8, P = 0.164. No significant association was demonstrated between VC supervision and the inclusion of unacceptable images in the case, χ ² = 2.8, P = 0.064. Despite this, a pattern of results suggest that a smaller proportion of cases included repeatable images in the post‐intervention (23.9%) compared to pre‐intervention cases (35.1%).

Examination survey results

VC supervision was used for 79 X‐ray examinations and 65 completed examination surveys were returned. XOs reported that in 24.6% of cases they would have normally telephoned a radiographer for help. Surveys indicated that 64 of the 65 (98.5%) XOs wanted or needed help during the examination (the question was not answered on one of the returned surveys). Help was received by the XO during the examination in 53 of the 60 responses (88.3%). Table 2 depicts the different categories of help that were required and received by XOs during VC supervision; more than one area of assistance was available during each examination.

Table 2.

Areas of help wanted or received by X‐ray operators during video conference supervision

	Help wanted/needed (n = 64 responses)	Help received (n = 60 responses)
Patient positioning	48 (75%)	46 (76.7%)
Radiation exposure selection	28 (43.8%)	31 (51.7%)
Image processing	15 (23.4%)	15 (25%)
Masterpage computer program	0	3 (5%)
RIS computer program	0	0
Approval for imaging	3 (4.7%)	0
No help indicated	1 (1.6%)	5 (8.3%)

Number of examinations within category selected (percentage of examinations with this category selected) Respondents were able to select more than one category in each column.

Discussion

Image quality changes

This project demonstrates that teleradiography, which replaces the use of telephone support with real‐time VC supervision, significantly improves the quality of images taken by XOs in rural facilities. The project shows significant improvement in images acquired with VC supervision through: required anatomy included; patient positioning; collimation used and image quality relating to exposure used.

The inclusion of required anatomy saw the least improvement with VC supervision but still demonstrated a significant improvement from a mean score of 6.4 pre‐intervention to 6.9 with VC supervision, P < 0.05. Appropriate collimation scores, even with VC supervision only reached an average of 5.9 out of a maximum of 10 which indicates improvement is still needed in this region. Collimation may have been difficult to visualise by VC due to the angle of the camera and the brightness of the collimation light against the ambient light in the X‐ray room. This would have made radiographer assessment prior to image acquisition difficult, however, once the image was produced the VC would facilitate a timely discussion on improvement, if required.

Eliminating the need to repeat an image is not always possible as, while much care and attention is taken, positioning and image processing errors can occur which require a repeat. The fact that images that should have been repeated were sent to PACS is an issue, but again, this might have been due to the subjective nature of image assessment,12 and lack of a gold standard in which image quality can be compared,13 thus, variation in opinion occurs.

Video conference methods

VC supervision provides both visual and oral communication about the examination between the radiographer and XO. The addition of this visual display and delivery of support in the form of teleradiography has been shown to improve the quality of the images produced. Such support methods allow for an increase in the number of locations in which a single radiographer's knowledge and technical expertise can be applied, irrespective of the physical location in which that examination is undertaken.

The increased training and education time spent between XO and radiographer with this type of supervision is expected to improve the quality of images taken by XOs, and reduce the need for supervision on a long‐term basis. An increased input from radiographers in the examinations performed by XOs could negatively impact XO confidence levels and create a reliance on radiographers being present via video conferencing. Reliance on radiographers by XOs for decision making is not a limitation of the project. It would be considered best practice for the most experienced staff member to take the lead in decision making in relation to patient care. However, the transfer of knowledge in appropriate learning scenarios through VC is expected to result in an improvement in XO awareness of acceptable image quality and, consequently, enable them to make more appropriate independent decisions.14, 15

Supervision conducted via teleradiography can have an immediate impact on the quality of X‐ray images taken at rural facilities and positively affect the resultant patient care derived from those images. This potential improvement in patient care will, however, come at the expense of an increased supervisor burden on radiographers; although this burden is expected to reduce as competence of XOs increases. Once the equipment is installed there is also the opportunity for radiographers to conduct training sessions using role play. Access to VC may also prove beneficial to trouble‐shooting of equipment issues.

Contact issues

A long held issue with XO supervision is the lack of contact between XO and radiographer when issues arise in image acquisition as well as the delay in feedback, as radiographers usually review XO images days or weeks after examinations are completed. This lack of contact and feedback timeliness means that the quality of the images taken could have improved, but this relies on the XO calling for assistance. XOs are reluctant to call for assistance; our surveys showed that in only 24.6% of examinations XOs would have called for assistance, however, help was received in 88.3% of the post‐intervention examinations. This mismatch, or lack of awareness that help is required, needs to be addressed to improve the X‐ray service available in rural facilities.

The categories in which help was wanted and received were similar, indicating that XOs were able to identify which area they needed or wanted help, but the reluctance to make the call for assistance seems to be a limitation in receiving the help needed. The accuracy of these category data and the conclusions drawn is affected by the method and time in which these data were collected; XOs completing the form after the help was provided may have influenced the answers given.

The frequency of VC supervision provided as part of the project design may not be sustainable long term due to the time burden on the supervisors. However, advantages of this method of supervision include increased availability and effectiveness of support and precision of feedback as supervisors can see the actual clinical scenario and the imaging technique used, rather than just the final images obtained. Real training opportunities such as guidance at the time of image acquisition, are often more valuable than retrospective discussion or simulated role play situations. The frequency and use of VC can be negotiated between individual XOs and their supervisors taking into account supervisor availability and the reported reluctance of XOs to call for support identified in this project.

Image assessment tool

Image assessment is a routine part of image acquisition but with no existing tool for radiography personnel to use, one was created which could detect change in the criteria assessed. Without clear criteria on image assessment, radiographers heavily rely on personal experience in image quality assessment.16 The European Commission published guidelines in 1996 in an attempt to characterise a minimum level of acceptable image quality, however, they provide guidance only on a small range of examination types and do not provide a scalable measure of compliance.17 Staff who are under supervision, such as XOs, are often confused by conflicting information in the assessment of image quality which could be reduced by the use of a single image quality assessment tool.

The image assessment tools for this project were created by consensus of a group of radiographers who are experienced in supervision and image assessment. The tool has not yet been validated for inter‐observer consistency; however, the use of a single reviewer in this study limited the effect of this variation in results.

Limitations and Future Recommendations

Mandating the use of VC supervision for all examinations in this study meant that more examinations were performed with radiographer support than would normally occur via telephone.

The use of an unvalidated assessment tool without evidence of its reliability could affect the outcomes of this project, however, the use of a single reviewer removes the inter‐observer variation experienced by an untested assessment tool. Without intra‐observer reliability of the tool being established, nor blinding of the reviewer, the effects these actions have on the results are unknown. Further research into the use of VC in the support and supervision of XOs is needed to further validate the findings of this pilot study and discover additional use of this telecommunication method. Since the completion of the study the researcher is aware of VC continuing to be used to deliver XO training and support through examination supervision, delivery of tutorials and performance of annual XO licence assessments.

Individual performance levels of XOs could be a factor due to different levels of experience, i.e. how long they had held their licence which equates to the number of hours of training and experience, or the number of studies performed with VC supervision during the project. These sub analyses were not possible to perform on this pilot project data as the data set was too small.

The difficulty level of examinations performed might also affect the image quality changes experienced during the project and this was not taken into account, but should be a consideration for future projects in this area.

Conclusion

The use of VC supervision improved all measured image quality criteria for XO performed imaging examinations. Overall image quality and diagnostic value scores for examinations saw highly significant improvements with the use of VC supervision. These improvements come as the result of well‐timed support of XOs and the inclusion of a visual component in the communication practice between XO and supervising radiographer.

Teleradiography provides effective feedback to XOs at the time of image acquisition and enables radiographers to assess the methods in which the XO is acquiring the images rather than just the resulting image. This education assists in XO learning while also immediately affecting the image quality produced at rural sites. This outcome relies on the VC equipment being readily available in the X‐ray room. The existing reluctance by XOs to call for help will not change if barriers to accessing help exist. Sites should look at the reallocation of existing VC units within their healthcare setting to take advantage of this improved image quality.

Further research into the effects of VC supervision methods and confirmation of these results in larger populations is encouraged as is further improvement in the delivery and support of X‐ray services in rural and remote communities in Queensland.

Conflict of interest

The authors have no conflict of interest to declare.

Supporting information

Appendix S1. X‐ray image and case assessment tools.

Appendix S2. Exam survey.

J Med Radiat Sci 64 (2017) 244–250