Barriers and facilitators to the adoption of artificial intelligence in radiation oncology: A New Zealand study

Highlights

• •Majority of respondent likely to adopt AI.
• •Innovation factors account for $51\%$ of total variance.
• •Perceived Benefit is a facilitator to adoption of AI.
• •Organisational factors are a moderate factor to adoption of AI.
• •Low levels of expertise are a barrier to adoption of AI.

Abstract

Introduction

Advances in computing capabilities and automated data collection have led to an increase in the use of Artificial Intelligence (AI) in radiation therapy. This has implications to workflow and workforce planning in radiation oncology departments. A survey was conducted in New Zealand to determine the likelihood of departments adopting AI into their practice. Survey responses were used to determine barriers and facilitators to the adoption of AI.

Materials and Methods

An online electronic survey was sent to all ten radiation therapy centres in New Zealand. The survey was sent to radiation oncologists, medical physicists and senior radiation therapists involved in treatment planning. Descriptive analysis, factor analysis, analysis of variance and hierarchical multiple regression were used to analyse the data.

Results

AI usage was low across the country and there was middling expertise. Most respondents found AI had a lot of perceived benefits. On the whole, respondents reported a high likelihood to adopt AI. There were significant differences on the Expertise factor between the staff groups$\left(p=0.016\right)$ with radiation therapists reporting more expertise than oncologists. Innovation factors (Perceived Benefit) on their own accounted for over $51\%$ of total variance and was the biggest predictor of likelihood to adopt AI$(p<0.001)$. Organisational factors (Expertise) was a moderate predictor$(p<0.059)$.

Conclusion

The survey results have been used to investigate the barriers and facilitators to the adoption of AI. These results demonstrate that respondents are likely to adopt AI in their practice. Perceived benefits were a facilitator as high scores were correlated with high likelihood of adoption of AI. Low expertise on the other hand was a barrier to adoption as the low scores were linked to lower likelihood of adoption.

Introduction

Artificial intelligence (AI) and machine learning (ML) are computerized approaches to identify complex mathematical relationships within observational data [1]. Advances in computing capabilitiesand automated data collection have led to an increase in the use of AI in radiation therapy. Numerous data sources in radiation oncology provide opportunities for AI applications and methodologies to improve quality and safety of cancer-care delivery [1]. As a discipline, radiation oncology has strong quality assurance and data-driven frameworks providing a strategic and compelling foundation for the ongoing and future development of AI integration into clinical patient care pathways [1]. Advances in AI in the treatment planning of head and neck cancers using intensity modulated radiation therapy has resulted in advancements in patient outcome modeling, autosegmentation and treatment plan automation [2].

In the future, supported by the use of AI, the foundation of work within radiotherapy will move from a focus on routine tasks, to tackling more complex and creative work, requiring critical thinking and collaborative efforts [3]. Several AI applications are emerging across the radiation oncology workflow [4] all aimed at benefitting patients through addressing such issues as improved standards of care, fewer side effects and an enhanced quality of life. Automation and AI brings with it the advantages of improved efficiency, standardised patient care and the potential to develop decision support systems. There is also a potential cost advantage; AI in health is projected to reach US$6.6 billion in value in 2020 and save the American healthcare industry US$150 billion per year by 2026 [5].

In New Zealand, emerging technologies like AI have not yet had any significant effects on the labour markets [6]. Studies to evaluate the perceptions of radiation oncology professionals to AI have been done in other jurisdictions with results indicating a cautious embrace of the new technology [4], [7].

AI and ML are set to transform the practice of radiation oncology [8], and there is an opportunity for radiation oncologists, radiation therapists and medical physicists to play a prominent role in shaping a future dominated by the emerging technology [9]. Despite the perceived benefits, the growing interest in AI and ML comes with some risk of misaligned expectations as to what the capabilities and potentials of the models are. AI systems excel at replicating, automating and standardising human behaviour on manual chores but judgement and evaluation remain the domain of human insight and intelligence [10], and inappropriate training of AI models has been shown to lead to disparities in health outcomes [1].

This study aimed to find out the likelihood of New Zealand radiation therapy professionals adopting AI into their departments especially in treatment planning. The study further sought to investigate the barriers and facilitators to the adoption of AI into a radiation therapy department.

Materials and methods

Research design

The survey was a self-administered electronic questionnaire (Supplementary Material Appendix 1). Survey questions were adopted from validated questionnaires [11], [12], [13], [14]. Experts in the field were asked to review and further validate the items; looking at the extent to which the items measured the intended attributes [14]. Questions were on a five (5) point Likert scale [15].

Purposive sampling was used to select participants as answering the questionnaire required some specialist knowledge in treatment planning [16]. Medical physicists, radiation oncologists and senior radiation therapists involved in radiation treatment planning were surveyed. The survey link was directly emailed to 73 participants. It was additionally emailed to 3 heads of departments who further distributed it to their direct reports. The survey was open for five (5) weeks between October 2019 and November 2019.

Data was collected via an electronic questionnaire using Qualtrics^XM (Utah, US) and was analysed using the SPSS® software package [17]. Descriptive analyses were carried out and point estimates were used to measure the mean and standard deviation of responses [16]. Data analysis was simplified by dimension reduction of the ordinal responses using principal component analysis factor analysis (PCA) [18].

One way analysis of variance (ANOVA) was used to gain information about the relationship between the three (3) staff groups; sexes; experience levels and the different age groups [19]. Hierarchical multiple regression was used to determine which factors strongly predicted likelihood of adoption of AI.

Results

Demographics

101 professionals responded to the survey. It was not possible to calculate the actual rate of response as some of the surveys were indirectly distributed, however, it is estimated to be approximately$26\%$. Sixty three $(63)$ completed all the questions in the survey and 38 only answered in part. The partial response rate varied from $2\%to96\%.$ Table 1 below summarises the demographic data.

Table 1.

Demographic characteristics.

Variable/Attribute	n	%
Staff Group
Radiation Oncologist	15	23.4
Radiation Therapist	27	42.2
Medical Physicist	22	34.4
Gender
Female	34	54.0
Male	28	44.4
Other	1	1.6
Years worked in radiation therapy
0–5	8	12.7
6–10	12	19.1
11–15	10	15.9
16–20	12	19.1
≥21	21	33.3
Age Group
≤30	13	20.3
31–40	16	25.0
41–50	18	28.1
51–60	14	21.9
≥61	3	4.7

$54\%$ of the respondents were female, $44\%$ or 24 were male and $2\%$ identified as other. The majority of respondents ($52\%$) had over 15 years’ experience in the radiation oncology field whereas $13\%$ had 0–5 years’ experience.

Although a large proportion of study participants had 21 + years’ experience, only two of respondents were over the age of 61 years. Nearly three quarters$(73\%$) or $47$ of the respondents were 50 years old or under.

Summary of scores

Harmsen et al (2005) developed a barriers and facilitators instrument based on literature review and an expert panel consensus procedure [12]. They divided their items into four categories: innovation, care provider, patient and organisational characteristics. The survey questions were grouped into each of these categories and the results are summarised in Table 2 below.

Table 2.

Average of respondent scores per domain.

Construct/Domain	Mean Score (Max score = 5)	Standard Deviation (SD)	Comment
Organizational measures	$2.42$	0.81	Use of AI tools is currently low
Department measures	$3.09$	0.67	Somewhat agree department is geared to adopting AI
Provider	$3.81$	0.43	Agree that they are comfortable with AI tools
Innovation	$3.38$	0.68	Somewhat agree it is useful

Current usage of AI in organisations was low ($2.42)$ among the respondents, with most participants reporting that they Never/Sometimes use AI tools (Appendix 2a). In terms of using AI tools for auto segmentation only 35% of users reported using AI tools About half the time/Most of the time. 11% reported never using these tools. However 62% reported using a combination of manual and AI methods for segmentation. 45% of respondents reported using knowledge based planning tools About half the time/Most of the time or Always. None of the respondents reported using diagnosis or decision support systems in their departments.

Respondents reported that while their organisations were geared towards implementing AI solutions ($\left(3.09\right)$, they thought they did not have the necessary resources to do so. $52\%$reported they Somewhat Agree/Strongly Agree that their organisation is committed to introducing AI (Appendix 2b). $82\%$ reported that their organisations are largely successful at implementing new technologies. However, only $20\%$ of respondents Somewhat Agree/ Strongly Agree that they had access to the right experts and a strategic plan for implementing AI.

Respondents supported the use of AI in their departments (Appendix 2c). More than $85\%$ reported that they Somewhat Agree/Strongly Agree that they will benefit from using AI and that this will improve their efficiency. In addition, $95\%$ reported that they are keen to learn how to use new technologies. Only $28\%$ responded that they Agree or Somewhat Agree that they will lose some of their autonomy by using AI and approximately $30\%$ responded that they had a challenge with constantly changing technology.

More than half the respondents Somewhat Agreed or Agreed (Appendix 2d) that introduction of AI would require their departments to make substantial changes to their workflow. The majority (>$50\%$) also Somewhat Agreed or Agreed a change to AI will leave room for them to make their own conclusions and that AI will leave room to accommodate the wishes of the patient.

$55\%$ of respondents believed their departments would be adopting AI solutions in the near future while more than $95\%$ of respondents reported they were willing to try out AI tools developed by researchers and were willing to use them if they knew it was right for the patient (Appendix 2d). $79\%$ stated they intended to use AI tools to help patients.

Principal component analysis

After dimension reduction, the forty-two (42) original items across four (4) constructs were reduced to seven (7) new factors named: AI (Usage of AI), Expertise (Expertise in AI), ChangeExperience (Experience with change), TechSavvy (Proficiency in using technology), FFP (Fit for Purpose), INCF (Inconvenience Factor), and PBenefits (Perceived Benefits). The results are summarised in Table 3.

Table 3.

New Factors.

Construct	New Factor Name	Calculated Mean (on 5 point Likert Scale)	Standard Deviation	Cronbach’s Alpha
Organisation	AIUsage	$2.38$	0.78	$0.72$
	Expertise	$2.69$	0.79	$0.87$
Provider	TechSavvy	3.73	0.65	0.79
Innovation	FFP	3.40	0.66	0.70
	INCF	2.48	0.85	0.77
	PBenefits	4.10	0.71	0.94

For organisational factors, fourteen (14) measures were grouped into three (3) principal components and they were redefined as in Table 3. The new factors are shown with their mean values and Cronbach alpha values.

The scores demonstrate that on average, usage of AI (AIUsage) across the country was low ($2.4$) and there was middling Expertise ($2.7$).

The provider related measures generated one principal component, TechSavvy. Respondents reported a positive attitude towards adoption of new technology ($3.7$).

Innovation related factors contained 17 measures and generated three principal components. While respondents could see the benefits of using AI over current systems $\left(PBenefit=4.10\right)$ they thought it would be complex and inconvenient to implement$\left(INCF=2.48\right)$.

Anova - Demographics

Staff groups

There were statistically significant differences on the Expertise factor between the staff groups$\left(F\left(2,56\right)=4.43,p=0.016\right)$. RTs and ROs generated the most difference, with RTs reporting higher levels of expertise$\left(p=0.016\right)$. The mean difference in RT/RO responses was$0.72(RTs=3.00\pm 0.79,ROs=2.28\pm 0.67)$.

Biological Sex

Female usage of AI (AIUsage) was 0.56 scores greater than their male counterparts$\left(F\left(1,55\right)=6.63,p=0.013\right)$. They also thought that introduction of AI to their departments would be more compatible or less disruptive (INCF) than males$\left(F\left(1,60\right)=4.163,p=0.046\right)$. Female respondents reported higher levels of likelihood to adopt (Likelihood factor) than males$\left(F\left(1,59\right)=4.72,p=0.030\right)$. This is more likely explained by the fact that they report higher AIUsage hence they are more familiar with the innovation.

Age and experience

The difference between different age groups on Expertise was not significant$\left(p=0.058\right)$. A significant difference was noted on examining the post hoc Tukey results for respondents under 40 and respondents over 51 years. Respondents aged 40 years and under recorded higher levels of Expertise than respondents aged 51 years and over$\left((F\left(2,59\right)=3.00,p=0.046\right)$. The reasons for this are unclear but it would seem the older the respondent, the more they are likely to be reluctant to change their way of doing things. There were no statistically significant differences between the means of people with different levels of experience across all the factors.

Hierarchical multiple regression – Predictors of likelihood to adopt

Nearly $62\%$ of the variance in the Likelihood to adopt AI was accounted for by the combined set of factors $\left(F\left(11,44\right)=9.12,p<0.05,R^2=0.695,R^{2}Adjusted=0.619\right)$(Table 3).

Innovation factors alone accounted for over $51\%$ of the total variance.

As illustrated in Table 4, PBenefit was a significant predictor $\left(\beta =0.72,t\left(56\right)=8.917,p=0.000\right)$ for Likelihood to adopt of AI. Expertise was the only other moderately strong predictor of adoption$\left(\beta =0.200,t\left(56\right)=1.941,p=0.0590\right)$. There is a positive correlation between these factors and likelihood of adoption: i.e. the greater the perceived benefits and expertise, the greater the likelihood to adopt.

Table 4.

Variable associated with likelihood to adopt AI.

	Unstandardized Coefficients		StandardizedCo-efficient	Sig.	95% CI for B
	B	Std. Error	Beta		Lower	Upper
Demographics
Sex	0.208	0.129	0.154	0.113	−0.051	0.468
Experience	−0.206	0.113	−0.243	0.076	−0.434	0.023
Age	0.216	0.126	0.259	0.094	−0.038	0.471
Innovation
FFP	0.016	0.084	0.018	0.848	−0.153	0.185
INCF	−0.026	0.075	−0.033	0.732	−0.176	0.125
PBenefits*	0.724	0.088	0.760	0.000	0.546	0.902
Organisation
AIUsage	−0.127	0.083	−0.143	0.132	−0.294	0.040
Expertise	0.200	0.103	0.225	0.059	−0.008	0.409
ChangeExperience	0.075	0.084	0.091	0.375	−0.094	0.245
Provider
TechSavvy	0.106	0.097	0.103	0.280	−0.089	0.302

$\left(F\left(11,44\right)=9.12,p<0.05,R^2=0.695,R^{2}Adjusted=0.619\right)$

Discussion

Survey results show that the Likelihood to adopt AI score was generally high among respondents. RTs had higher levels of expertise which is not surprising as they form the largest group and are heavily involved in computer use for optimising treatment plans. Females reported a higher level of AIUsage and Expertise which may be because $87\%$ of the RT group identify as female [20] and RTs were the biggest responders to the survey$\left(42\%\right)$. Respondents aged 40 years or younger reported a higher level of expertise than those aged over 51 years. Only a minority ($11\%)$ of respondents are using auto segmentation tools only but over 60% are using both auto segmentation and manual tools. A similar study in Australia reported $45\%$ of respondents used somewhat automated methods to contour organs at risk [4]. The relatively low usage of automated tools may suggest that current tools are not sufficiently developed to meet the minimum standards and need manual modification. About 45% use knowledge based planning and this is one area that shows the most promise. Use of a knowledge based planning system at the authors institution has resulted in shortening of treatment planning times from 16 hours over two or three days to 2 hours in a single day.

From the hierarchical multiple regression analysis, the biggest predictor for adoption of AI was PBenefit, with Expertise being a moderate predictor. The more the perceived benefit (PBenefit), the more likely the adoption of AI, perceived benefit in this instance is a measure of relative advantage. The positive correlation between higher relative advantage and higher likelihood to adopt an innovation is consistent with findings from Rogers (2003) who notes that relative advantage is one of the strongest predictors of an innovation’s [rate of] adoption [21]. The relative advantage in a radiotherapy department can be considered in terms of service efficiency. The biggest short term benefit of the implementation of AI is the improved efficiency in the treatment planning process, where system processes are substantially sped up from simulation to treatment, reducing the time burden of human interaction [1]. In addition AI systems have demonstrated improved planning efficiency and plan quality consistency [22].

Regression analysis shows that the higher the Expertise, the higher the Likelihood of adoption. Expertise is a provider factor and individuals can become experts through their own efforts however the attainment of said expertise can be hindered by organisational policies.

It is envisaged in the future that rapid and reliable automation will reshape resource utilisation, staffing levels and training requirements and successful implementation will depend on human engagement to utilise AI to complement human skills [1]. One of the measures contained in the Expertise factor was the availability of an AI expert within the team. The average measure for this score was low$\left(2.10\right)$, i.e., most providers somewhat agree that they do not have an AI expert in the department, this role would become more important as the rate of adoption is affected by the extent of the experts’ promotional efforts and being driven further by opinion makers (clinical directors, prominent experts in the field etc.), as this may accelerate the rate of adoption [21]. The Expertise is middling $\left(2.69\right)$and at this level, it may be a barrier to adoption of AI.

The Productivity Commission reports that labour productivity in New Zealand is low partly because of low uptake of new technology [6]. This maybe because adoption of AI heightens stress and risk aversion related to fear of redundancies as a result of companies leveraging labour productivity afforded by automation and AI [23]. The history of automation however shows that jobs are not lost but roles are redefined, with humans left to tasks that require a human element [9]. This is especially true in healthcare where AI will be integrated in organisations to assist with care provision not replace it [24]. The Royal Australian and New Zealand College of Radiologists (RANZCR) have issued a position statement recognising that AI will have a dramatic effect on radiation oncology practice in the next few years [24]. The call is for specialists to adapt and transform together with the technology. Along with the challenges of mastering the technology also lie legal, governance and quality assurance issues that need to be explored further before rolling it out into clinical practice [25]. A balanced approach that enhances accountability on one hand and facilitates innovation on the other will be required to fully optimise the benefits of AI [26].

In the current economic environment, New Zealand’s District Health Boards (DHBs) are increasingly being asked to do more without a corresponding increase in budget [27]. The increased adoption of AI will potentially have the effect of reducing costs, leading to higher volumes of care delivered, which in turn leads to more employment opportunities [28]. AI has the potential to improve efficiency in service delivery, provide consistent quality of care and equity in access through standardising clinical practice across New Zealand [30]. This is consistent with findings from Australia where researchers found the majority of respondents felt automation will increase consistency in planning, work output and productivity, and quality of planning [4]. The efficiency gains can be channelled to tasks that require human interventions and emotions such as plan checking and longer review clinics. This may lead to redefinition or expansion of roles for staff who are currently doing roles that are easily replicated by AI to include the clinical aspects that require human intuition. The author puts forward the following suggestions to increase the likelihood of adoption of AI:

• 1.To facilitate implementation of efficient and cost effective AI innovations, departments need to identify and train or recruit skilled employees in AI and data science methods. Further, they should actively encourage and support continual professional development activity in this field via conferences, collaborations with vendors, workshops, higher education and links with centres of excellence. Current methods of manual target volume segmentation and dose optimisation in treatment planning are labour intensive and time consuming but can be reduced to hours by deploying AI systems [22].
• 2.Safe adoption of AI in oncology hinges on the successful merger of data science and clinical oncology. At present, clinicians have no training or expertise in data science. This limits their ability to understand how the systems work therefore limiting informed choices when it comes to adopting appropriate AI algorithms. Similarly, data scientists have little to no clinical knowledge, and are limited in knowledge and understanding when needing to identify important clinical parameters for consideration in algorithm use and deployment [31]. In a survey of Australian radiation therapy profession, it was reported that $66\%$ of respondents thought automation will change the primary tasks of certain jobs [4]. To develop a sustainable workforce driven by and competent with AI and ML processes, the core professional groups; radiation oncologists, medical physicists and radiation therapists in a tripartite forum, must work with universities and radiation oncology administrators to explore the creation of a new professional role within radiation oncology.

The current study has the following limitations:

• a.It did not explore the opinion of patients or patient advocates. A future study could examine the willingness of the patients to cooperate with the innovation; and the degree to which the patients are aware of the health benefits of the innovation [32].
• b.The survey instrument is quantitative only. A future survey could include a free text option to answer some questions or face to face interviews.
• c.The radiation therapy centres respondents worked at were not identified. This hinders the ability to determine which centres are doing better than others and to further contextualise barriers and facilitators.
• d.The responses to the survey were also self-reported and responses may not align with actions on the ground. Independent evaluators could be engaged to note observations and validate the results from each department.

Conclusion

This is the first survey in New Zealand to survey oncology departments on their likelihood to adopt AI. The survey results have been used to investigate the barriers and facilitators to the adoption of AI. These results demonstrate that respondents are likely to adopt AI in their practice. However adopting at present is at its infancy as users and organisations try to understand how it will fit with their workflow. Perceived benefits (relative advantage) were a facilitator as high scores were correlated with high likelihood of adoption of AI. Low expertise on the other hand was a barrier to adoption as the low scores were linked to lower likelihood of adoption. The low level of expertise and shortage of AI champions suggests education will be key in getting current professionals comfortable with AI systems. In the long term, new roles in radiation therapy with a focus on machine learning and data science may need to be explored. The level of usage and perceptions on the use of AI broadly align with findings from Batumalai et al (2020) [4], which are ‘cautiously optimistic’; optimism bolstered by the sense of responsibility that professionals have of learning new technologies, but tempered by the reality of the current lack of appropriate human resources and the requirement to make significant changes to work flows. Results from this study and suggestions put forward can be used to navigate the barriers and enhance the facilitators to the adoption of AI.

Ethics statement

Ethics approval was sought and granted through the University of Waikato Human Research Ethics Committee (WMS 19/90).

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.