Cautiously optimistic: A survey of radiation oncology professionals’ perceptions of automation in radiotherapy planning

Highlights

• •Majority of respondents felt automation will improve consistency, productivity and quality of planning.
• •27% of respondents believe automation will reduce job satisfaction.
• •Majority of respondents (71%) thought that automation will cause a loss of skills.
• •This study highlights the need for continued education to ensure that skills and knowledge are not lost with automation.

Abstract

Introduction

While there is evidence to show the positive effects of automation, the impact on radiation oncology professionals has been poorly considered. This study examined radiation oncology professionals’ perceptions of automation in radiotherapy planning.

Method

An online survey link was sent to the chief radiation therapists (RT) of all Australian radiotherapy centres to be forwarded to RTs, medical physicists (MP) and radiation oncologists (RO) within their institution. The survey was open from May-July 2019.

Results

Participants were 204 RTs, 84 MPs and 37 ROs (response rates ∼10% of the overall radiation oncology workforce). Respondents felt automation resulted in improvement in consistency in planning (90%), productivity (88%), quality of planning (57%), and staff focus on patient care (49%). When asked about perceived impact of automation, the responses were; will change the primary tasks of certain jobs (66%), will allow staff to do the remaining components of their job more effectively (51%), will eliminate jobs (20%), and will not have an impact on jobs (6%). 27% of respondents believe automation will reduce job satisfaction. 71% of respondents strongly agree/agree that automation will cause a loss of skills, while only 25% strongly agree/agree that the training and education tools in their department are sufficient.

Conclusion

Although the effect of automation is perceived positively, there are some concerns on loss of skillsets and the lack of training to maintain this. These results highlight the need for continued education to ensure that skills and knowledge are not lost with automation.

Introduction

Radiotherapy is a rapidly developing field with a constant drive to improve the quality and efficiency of patient care. There is an urgency around the use of technology in healthcare to advance care and improve patient outcomes, as well as to create efficiencies, particularly through the greater use of automation and artificial intelligence (AI).

Several applications for the use of automation and AI are well established or emerging across the multidisciplinary radiation oncology workflow in the clinic. These include image fusion and registration [1], contouring [1], treatment planning [2], [3], [4], [5] and quality assurance [6], [7]. Treatment planning functions such as auto-planning (AP) provide tools to reduce the amount of user interaction required to achieve optimal complex plans such as intensity modulated radiation therapy (IMRT) and volumetric modulated arc therapy (VMAT). Automated knowledge-based planning system allows reduction in the time to create a treatment plan and in most cases can by more than an hour while reducing variability in plan quality and producing comparable or better plans compared to manual planning [8], [9]. Planning tools such as the ability to create templates or scripts further enable automation of many steps required to complete and check a treatment plan. The use of automated segmentation for contouring also leads to time saving when users are able to recognise when automated contours requires modification [10]. Although automation in radiotherapy planning is gaining a momentum globally, its diffusion and extent of use in Australia is not known.

Automation has driven changes in productivity resulting in disrupted labour markets [11] and much of the broader debate on automation assumes technologies will replace human roles [12]. While there is evidence to show the positive effects of automation and AI strategies in improving overall productivity and efficiency in radiotherapy planning, the impact on radiation oncology professionals has been poorly considered to date. Will there be vast improvements in productivity, freedom from performing boring tasks, and improved quality of life or will there be job threats and organisational disruption? Gillan et al. [13] recently reported that radiation oncology professionals thought that automation and AI will most likely result in displacement of tasks than rather than replacement of professionals. However, this was based on only 24 participants in focus groups from a single centre in Canada [13]. It is important to evaluate a large group of radiation oncology professionals from multiple centres to understand, anticipate, and balance staff perceptions against the potential benefits of automation.

This study aimed to understand radiation oncology professionals’ perceptions of automation in radiotherapy planning in Australia. This study also aimed to understand the diffusion of automation in Australia. By gaining an insight into the perception of automation by radiation therapists (RTs), medical physicists (MPs) and radiation oncologists (ROs), the workforce will be able to maximise the potential of automation.

Materials and methods

A questionnaire was designed to understand radiation oncology professionals’ perceptions of automation in radiotherapy planning (Supplementary Material 1). Survey questionnaire included: demographic information (generation, organisation, profession, and years of work); current level of automation established in the department; planned automation in the next 2 years; perceived importance of automation; perceived effect of automation on work output; perceived impact of automation on jobs; opinion on loss of skills; opinion on current training and education tools provided; and if automation will allow them to pursue new tasks/roles. The survey methodology and content were reviewed by the multidisciplinary group of authors and approved by the local human research ethics committee. The survey questionnaire was validated by establishing face validity where the questions were reviewed by three other parties not involved in the study and pilot tested on a subset of participants. The structure of the answers included multiple choice answers and Likert scale. A link to the online survey, along with an email outlining the aims of the survey and confidential nature of the results, was sent to the chief RT of all 93 Australian radiotherapy treatment centres. It was requested that the survey be sent to all RTs, MPs and ROs within their institution. The survey was also distributed through the Australasian College of Physical Scientists & Engineers in Medicine (ACPSEM), the New South Wales (NSW) Research RT network, the Australasian RT Clinical Educator network, and through social media platforms (Twitter and Facebook). Participants were requested to respond within 4 weeks. For non-responders, reminder emails were sent at 4 and 8 weeks after the initial email. The survey was closed after a period of 12 weeks (May 2019–July 2019). The survey contained questions on; demographics, current and planned level of automation in departments and opinions on the effect of automation on specific tasks, roles and jobs.

Results

Demographics

A total of 325 radiation oncology professionals completed the survey. Responses were received from 204 RTs, 84 MPs and 37 ROs. Indirect distribution prevented response rates from being calculated; however, we estimate 8.5% of RTs, 18.6% of MPs, and 9.8% of ROs registered in Australia responded. The source of statistics was Australian Health Practitioner Regulation Agency, personal communication from ACPSEM and personal communication from Royal Australian and New Zealand College of Radiologists (RANZCR). The estimated overall response rate for the three professional groups is 10%. Table 1 summarises the demographics and employment details of the respondents. The majority of respondents, 63%, were from Generation Y (1977–1995) followed by 25% from Generation X (1965–1976), 8% from Baby Boomers (1964 and before) and 4% from Generation Z (1996 and later). Most respondents (32%) have been practising in their roles between 11 and 20 years. Majority of respondents were from metropolitan (67%) and public practice (82%).

Table 1.

Demographics and employment details of respondents.

	All	RT	MP	RO
Generation
Generation Z (1996 and later)	14 (4.3%)	8 (3.9%)	5 (6.0%)	1 (2.7%)
Generation Y (1977–1995)	205 (63.1%)	138 (67.6%)	52 (61.9%)	15 (40.5%)
Generation X (1965–1976)	80 (24.6%)	48 (23.5%)	22 (26.2%)	10 (27.0%)
Baby Boomers (1964 and before)	26 (8%)	10 (4.9%)	5 (6.0%)	11 (29.7%)
Years practising
<5 years	68 (20.9%)	40 (19.6%)	21 (25.0%)	7 (18.9%)
5–10 years	84 (25.8%)	48 (23.5%)	25 (29.8%)	11 (29.7%)
11–20 years	103 (31.7%)	64 (31.4%)	29 (34.5%)	10 (27.0%)
>20 years	70 (21.5%)	52 (25.5%)	9 (10.7%)	9 (24.3%)
Work location
Metropolitan	219 (67.4%)	138 (67.6%)	54 (64.3%)	27 (73.0%)
Regional	93 (28.6%)	60 (29.4%)	24 (28.6%)	9 (24.3%)
Metropolitan and regional	13 (4.0%)	6 (2.9%)	6 (7.1%)	1 (2.7%)
Service provider
Public	266 (81.8%)	179 (87.7%)	59 (70.2%)	28 (75.7%)
Private	41 (12.6%)	19 (9.3%)	21 (25.0%)	1 (2.7%)
Both public and private	18 (5.5%)	6 (2.9%)	4 (4.8%)	8 (21.6%)
Managerial/senior role
Yes	144 (44.3%)	91 (44.6%)	39 (46.4%)	14 (37.8%)
No	181 (55.7%)	113 (55.4%)	45 (53.6%)	23 (62.2%)

Level of automation

All participants reported various level of automation of radiotherapy planning tasks in their departments at the time of the survey (Fig. 1a). Planning tasks that were reported by the majority of respondents to be manually completed were target contouring (68%), plan checking/quality assurance (43%) and machine pre-treatment quality assurance (QA) (45%). More than 30% of respondents reported that the following tasks were somewhat automated; organ-at-risk (OAR) contouring (45%), beam placement (32%) and dose reporting (31%). The most common planning task reported to be automated with manual tuning was image registration (65%), while the most common task automated with a single click was plan/image transfer to record and verify system (28%). The top three tasks that are planned to be automated in the next 2 years were plan optimisation (66%), OAR contouring (64%), and plan checking/QA (50%) as displayed in Fig. 1b.

Fig. 1.

(a) Current level of planning tasks automation, (b) planned level of planning tasks automation in the next 2 years. Abbreviations: QA – quality assurance; R&V – record and verify; OAR – organ at risk.

Attitude towards automation in planning

Overall, 69% of respondents felt very probably/probably empowered to drive decisions about implementing automated planning processes with comparable responses from each profession (Fig. 2a). ROs and MPs (76%) and Gen X (79%) felt the most empowered to drive decisions about implementing automated planning processes. When asked about their opinion on the importance of automating planning processes, 66% of respondents indicated they thought automation was very important/important (Fig. 2b). More MPs (86%) and Baby Boomers (77%) thought that automating planning processes was very important/important compared to the other groups.

Fig. 2.

(a) Empowerment to drive decisions about implementing automation, (b) opinion on the importance of automating planning processes. Abbreviations: RT – radiation therapist; MP – medical physicist; RO – radiation oncologist; Gen Z – Generation Z; Gen Y – Generation Y; Gen X – Generation X; Baby Boom – Baby Boomers.

Opinions on how automation of the planning process will affect the work group and errors is displayed in Table 2. The majority of respondents felt automation will increase; consistency in planning (90%), work output and productivity (88%) and quality of planning (57%). A greater proportion of MPs (74%) and ROs (70%) thought that automation will improve quality of planning compared to RTs (48%). Only 44% of respondents felt that systematic errors will decrease, while 74% felt that random/human errors will decrease as a result of automation.

Table 2.

Opinions on how automation of the planning process will affect the work group and errors.

	All			RT			MP			RO
%	Will increase	Will decrease	No change	Will increase	Will decrease	No change	Will increase	Will decrease	No change	Will increase	Will decrease	No change
Work output and productivity	88	1	11	86	1	13	94	1	5	89	3	8
Quality of planning	57	13	30	48	17	35	74	5	21	70	5	25
Consistency of planning	90	1	9	88	2	10	94	0	6	92	3	5
Staff focus on patient care	49	9	42	41	13	46	64	6	30	54	0	46
Systematic errors	20	44	36	18	45	37	29	39	32	16	46	38
Random/human errors	9	74	17	10	68	22	4	93	3	14	65	21

Overall, 44% of respondents strongly agree/agree that automation will provide only positive benefits for their departments with ROs giving the most positive responses (Fig. 3). The majority of respondents (75%) strongly disagree/disagree that their departments are already too over-reliant on automation. When asked if automation will cause a loss of understanding of general underlying principles of radiotherapy, 71% of respondents strongly agree/agree with RTs agreeing the most (79%), while only 49% of ROs strongly agree/agree. Almost half of respondents (46%) strongly disagree/disagree that the current training and educational tools provided by their departments are sufficient to ensure staff do not lose understanding of general underlying principles of radiotherapy with MPs feeling most strongly about this.

Fig. 3.

(a) automation will provide only positive benefits for your department; (b) your department is already too over-reliant on automation; (c) automation will cause a loss of understanding of general underlying principles of radiotherapy; (d) current staff training and educational tools provided by your department are sufficient to ensure staff do not lose understanding of general underlying principles of radiotherapy. Abbreviations: RT – radiation therapist; MP – medical physicist; RO – radiation oncologist.

Opinions were almost equally divided with regards to the impact of automation on job satisfaction; will not impact job satisfaction (37%), will increase job satisfaction (36%) and will reduce job satisfaction (27%) (Table 3). However, when looking at individual professional group, 38% of RTs thought that automation will reduce job satisfaction compared to MPs (8%) and ROs (11%). MPs gave the most positive response with 61% stating that this would improve job satisfaction.

Table 3.

Attitude and perceived impact of automation, and tasks and/or roles to be pursued.

%	All	RT	MP	RO
Attitude toward planning process automation
Will reduce job satisfaction	27	38	8	11
Will increase job satisfaction	36	24	61	46
Will not impact job satisfaction	37	38	31	43
Perceived impact of automation on jobs
Will change the primary tasks of certain jobs	66	66	65	65
Will allow me to do the remaining components of my job more effectively	51	44	71	49
Will eliminate jobs	20	24	18	5
Will not have an impact on jobs	6	5	4	14
Not at all concerned with automation	9	9	7	14
Tasks and/or roles to be pursued
Learning new skills	66	62	69	76
Research and development activities	74	66	88	89
Being involved in implementation processes	58	59	62	46
Increased patient care focus	56	63	37	62
Training	50	49	56	43
Role expansion/Advanced practice	65	72	61	38

When asked about the perceived impact of automation on jobs, the following responses were recorded (Table 3); will change the primary tasks of certain jobs (66%), will allow staff to do the remaining components of their job more effectively (51%), will eliminate jobs (20%), will not have an impact on jobs (6%), and not concerned at all with automation (9%). More RTs (24%) and MPs (18%) thought that automation will eliminate jobs compared to ROs (5%). More ROs (14%) also seem less worried about the impact of automation jobs compared to RTs (5%) and MPs (4%). However, more MPs (71%) responded that automation will allow them to do the remaining components of their job more effectively compared to RTs (44%) and ROs (49%).

When asked if automation will allow them to pursue new tasks/roles and/or role expansion (question 16, Supplementary Material 1), 83% of respondents agreed. The top 3 tasks and/or roles all respondents would like to pursue are; research and development activities (74%), learning new skills (66%), and role expansion/advanced practice (65%) (Table 3). When looking at individual professional groups, RTs (63%) and ROs (62%) were also interested in pursuing an increased focus on patient care, while MPs (62%) were interested in being involved in implementation processes.

Discussion

There is scarce literature on the perception of medical and health care professionals on automation, with limited number of studies reporting on this topic in the area of robotic surgery [14], [15] and radiology [16]. To the best of our knowledge, this is the first large multi-centre study to evaluate radiation oncology professionals’ perceptions of clinical and professional challenges and benefits, and the evolving tasks and roles with automation. The data from this survey serves as a snapshot of current penetration of automation in radiotherapy practices. The opinions of respondents capture the attitudes of the staff using automation and could be used to inform workforce redesign strategies. There is an acknowledgement of both emergent opportunities and challenges for all professions in implementing automation. The results have several implications for workforce planning and redesign as a result of automation. Although this survey was targeted at radiation oncology professionals’ perceptions of automation in Australia, the results are likely to be reflective of the general feelings within their professional communities worldwide. Furthermore, our findings are consistent with the findings of a Canadian study [13]. An important point to note when interpreting these results internationally is how job descriptions of RTs and MPs may vary. For example, in Australia, radiation therapists are involved in both treatment planning (referred to as dosimetrists in some international settings) and treatment delivery. Similarly, MPs in some countries undertake the treatment planning.

A common perception of automation from participants in this study was that it would increase work output, productivity, quality and consistency in radiotherapy planning. Most respondents thought that automation will change the primary tasks of their jobs and allow them to do the remaining components of their job more effectively. This is inevitable as automation takes over some traditional tasks and generates new tasks to be undertaken by humans. A recent commentary addressed if AI will replace professionals in radiation oncology and reported that automation will not only replace many manual tasks performed today, increase efficiency and reduce the time spent on planning, but also create new opportunities [17].

Professional groups could feel threatened if changes in the work area involve redefinition of professional roles, in particular where it might be perceived as devaluation of established skills or contributions [13], [18]. Only 36% of respondents thought that automation will increase job satisfaction. This opinion was not consistent across the three professional groups being highest for MPs (61%) compared to ROs (46%) and RTs (24%). Similarly, higher number of RTs (24%) thought that automation will eliminate jobs compared to MPs (18%) and ROs (5%). Gillan et al. [13] also reported that RTs in their focus group debated whether fewer staff numbers in treatment planning roles would be required, noting a general fear of job loss with the introduction of automation. We believe that some of the apprehension for RTs may be due to the fear that the more stimulating tasks of their role (i.e. autonomy in treatment planning) may be replaced by automation, whereas for ROs and MPs it's more of the tedious tasks that will likely be replaced by automation. However, it’s important to understand that the treatment plans generated through automation need to be continually improved in order to increase the gains from AI, and this step relies on skilled human intervention with an understanding of radiobiology and the practicalities of treatment delivery [19]. We need to ensure that automation is beneficial to patients and clinical workflows, and empowering for staff without an adverse impact on their well-being. It is also critical to ensure that staff have the time to engage in the work of redesigning clinical pathways and workflows to accommodate the technology, a crucial factor that is often underestimated. Participants in a previous study advocated that they should be involved in guiding the introduction of AI, rather than passively accepting new roles or the elimination of jobs [13]. Radiation oncology professions can play an active role in ensuring optimal outcomes for the well-being of both the workforce and the patients [20].

Although the effect of automation is perceived positively with respect to work output and productivity, there are some concerns on loss of skillsets and the lack of training to maintain this. These results highlight the need for continued education to ensure that basic skills and knowledge of the principles of radiotherapy are not lost with automation of tasks in radiation oncology. The risk of losing certain knowledge and skills, and the need for competencies and education required for using AI was also discussed by participants in a focus group [13]. Participants referred to the need to be equipped with an understanding of the principles, functionalities and limitations of AI, in order to work responsibly with it in the clinical context [13]. Along with individual departments implementing automation, professional organisations such as RANZCR, ACPSEM, and Australian Society of Medical Imaging and Radiation Therapy also must develop strategies for automation. As a leader in setting standards for radiation oncology, RANZCR has already formed the Artificial Intelligence Working Group and is prioritising a number of considerations to guide implementation [21]. It is also important for professional organisations and departments to consider ethical and legal consequences of automation and AI. Any concern for the risks to which individuals are exposed as a result of decisions made by automation, not humans, needs to be addressed. Traditionally, medical professionals take on legal responsibly for errors in patient care, however, should this responsibility be shared with the software company as AI uptake increases? In the era of personalised medicine and patients making informed treatment pathway decisions, should patients be given the option of opting into or out of AI involvement in their care? Professional organisations and employers may also need to review the level of indemnity offered to staff by considering whether the risk of human error become less, or whether there is increased risk as staff take on high level tasks as AI or automation complete the more simple and repetitive tasks. This highlights the importance of quality assurance at every stage of the planning process to ensure patient safety and quality of treatment delivery, and ensuring legal protection of workforce and legal rights of patients.

Despite some concerns on automation, 83% of respondents thought that automation will allow them to pursue new tasks and role expansion. Similarly, the Canadian study respondents said that roles could be displaced by AI, or evolved in response to its introduction, rather than replaced [13]. Our study showed that the three professional groups surveyed had slightly different preferences on the roles they would like to pursue due to automation. RTs (72%) were most interested in role expansion and advanced practice compared to MPs (61%) and ROs (38%). This is in keeping with a previous study demonstrating that RTs had a preference for advanced RT roles over administration roles [22]. Time saved by automation and AI in certain areas could facilitate focus on other tasks and roles that previously could not be afforded the necessary time.

There are several limitations to this study. The estimated overall response rate for the three professional groups is only 10%. Indirect distribution of email invitations and reminders that were not personalised to individuals may have negatively affected the response rate. Responses may also have been biased towards staff who wanted to make a statement about the impact of automation and AI. Survey fatigue may also have contributed to the small response rate where the targeted audience may have been overwhelmed by the growing number of online surveys and chose not to participate. Other potential reasons include research interests of participants and lack of incentives for participating. However, given the wide distribution of the survey and the number of responses received from the three professional groups, there was sufficient data to understand the challenges and benefits of automation in radiotherapy planning in Australia. We also acknowledge that the presence of ‘bots’ to answer survey questions is an important concern with non-personalised surveys distributed through social media channels [23]. However, the responses were thoroughly checked to ensure data integrity. We know that the survey takes 10 minutes to complete, so if the time to complete was less than 7 minutes, the responses were reviewed carefully. The duplication tool was also used in Excel to review duplicate cases. We also acknowledge that the perception of how automation is defined can be a potential ‘grey zone’. Automation in treatment planning is also dependent on the vendor of the system and the version number (for example; the level of scripting supported). Another limitation is the lack of questions to understand if sufficient training was provided through participants’ education or by the department introducing the specific automation tool. Professionals with no specific automation or AI background or training may consider it a ‘black box’ and, consequently, distrust it. Despite these limitations, the results of this study reflect the opinions of radiation oncology professionals on automation and serve as an actionable insight.

Conclusion

The radiation oncology professionals in Australia remain cautiously optimistic of automation in radiotherapy planning, particularly radiation therapists. This survey highlighted various differences and similarities in opinions on automation between the three radiation oncology professional groups. It is important to consider the opinions of these professional groups when implementing automation and modify their training to plan and respond effectively to new workflows, skills and continuing changes in technology.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary material

The following are the Supplementary data to this article: