Stakeholders’ views on the organisational factors affecting application of artificial intelligence in healthcare: a scoping review protocol

Abstract

Introduction

Artificial intelligence (AI) offers great potential for transforming healthcare delivery leading to better patient-outcomes and more efficient care delivery. However, despite these advantages, integration of AI in healthcare has not kept pace with technological advancements. Previous research indicates the importance of understanding various organisational factors that shape integration of new technologies in healthcare. Therefore, the aim of this study is to provide an overview of the existing organisational factors influencing adoption of AI in healthcare from the perspectives of different relevant stakeholders. By conducting this review, the various organisational factors that facilitate or hinder AI implementation in healthcare could be identified.

Methods and analysis

This study will follow the Joanna Briggs Institute framework, which includes the following stages: (1) defining and aligning objectives and questions, (2) developing and aligning the inclusions criteria with objectives and questions, (3) describing the planned approach to evidence searching and selection, (4) searching for the evidence, (5) selecting the evidence, (6) extracting the evidence, (7) charting the evidence, and summarising the evidence with regard to the objectives and questions.

The databases searched will be MEDLINE (Ovid), CINAHL (Plus), PubMed, Cohrane Library, Scopus, MathSciNet, NICE Evidence, OpenGrey, O’REILLY and Social Care Online from January 2000 to June 2021. Search results will be reported based on The Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews guidelines. The review will adopt diffusion of innovations theory, technology acceptance model and stakeholder theory as guiding conceptual models. Narrative synthesis will be used to integrate the findings.

Ethics and dissemination

Ethics approval will not be sought for this scoping review as it only includes information from previously published studies. The results will be disseminated through publication in a peer-reviewed journal. In addition, to ensure its findings reach relevant stakeholders, they will be presented at relevant conferences.

Strengths and limitations of this study.

• This will be the first scoping review to map out the organisational barriers and facilitators in application of artificial intelligence (AI) in healthcare from the key stakeholders’ perspectives.

• The findings will be limited to what is reported in peer-reviewed published literature, therefore, the authors aim to conduct a follow-up primary research to include more data sources.

• The proposed scope may exclude some other relevant aspects in application of AI in healthcare.

• While the scope of this review is focused on the organisational issues related to AI implementation in the healthcare sector, the authors recognise that the findings will further need to be contextualised within a specific healthcare environment.

Introduction

Artificial intelligence (AI) is a general term used to describe computing technologies, which perform functions that aim to reproduce the capabilities of human mind such as reasoning, learning, adaptation, sensory cognition and creativity.¹ Rapid technological advances in the last decade broadened the portfolio of AI-based tools and widened their areas of applications. The use of AI in the healthcare sector is gradually increasing and expanding into areas such as medical diagnostics and treatment (eg, disease diagnosis, medical imaging, robot-assisted surgery), management and decision making (eg, design of patients’ pathways, allocation of resources), public health and epidemiology (eg, predictions about the likelihood of an infectious disease outbreak and its dynamics, risk factors for non-communicable diseases), patient care (eg, personalised health advice, remote diagnosis, patients’ monitoring), elderly care (eg, healthcare robots assisting older adults at care homes) and many more.^1–4

Despite the growing use of AI in healthcare and its potential to transform patients’ experience and quality of care, there is emerging evidence that the integration of AI-based tools has not been happening as quickly as the technology has been advancing.⁵ Quite often AI developers and software vendors are held responsible for failures in AI implementation due to their inability to deliver reliable products.^{6 7} However, previous studies suggest that technical factors can only explain up to 20% of AI failures, while most of the unsuccessful cases are directly linked to the lack of sociotechnical consideration.^8–10 This means that failures in the introduction of AI should be considered not just as a problem in technology, but also as a problem in organisational change.

Consequently, there is a growing body of research suggesting that to accelerate the integration of AI-based tools in healthcare organisations, the interactions between an innovation and the complex organisational setting and factors must be taken into consideration. Organisational factors can be thought of as conditions, strategies, operational attributes and processes, which may hinder or facilitate the use of AI in a healthcare organisation.¹¹ These factors can also include administrative support, procurement, training, communication and coordination mechanisms, team resources and many more.^{11 12} Although a little is known about the organisational changes required to enhance AI adoption, there is a growing body of research suggesting that this issue should be given more attention.^{13 14} For example, some studies reported that healthcare workers may be resistant to accept a new advanced technology as it may change work processes and routines, which can consequently result in a heavier workload.^{15 16} There is also evidence suggesting that in order to incentivise healthcare workers to use AI, a review of current payment systems should be conducted.¹⁷ The authors jointly concluded that adequate funding would overall incentivise healthcare organisations as the introduction of AI requires significant financial expenditures and changes to operational processes.3 10 17 18 Another reported organisational issue is related to a perceived loss of clinician control and autonomy, as some healthcare workers are concerned that policymakers, insurers, and administrators may use technology as a way to influence clinical decision making and actions.¹⁸ Some studies also reported that the absence of a clear legal framework and relevant policy developments, especially when it comes to data protection and accountability for the care outcomes, can serve as potential organisational barriers for successful adoption of AI.^{19 20} Finally, one of the most commonly reported issues affecting AI adoption is training and competencies of healthcare workers as the end-users.^{21 22} Some studies advised that such factors like too long or too short training requirements,²¹ the lack of an AI champion²³ and the lack of user involvement during system design²² can make healthcare workers feel less comfortable using AI-based tools.

It is evident that AI has a potential to transfer the overall healthcare system, which indicates that a holistic approach should be taken when implementing new advanced technologies incorporating different organisational management perspectives and knowledge from cognate disciplines. Given that healthcare organisations comprise many professional subgroups and power structures, it is important that all the relevant stakeholders, who are either direct (eg, using AI as part of a medical treatment) or indirect (eg, managing a surgery/hospital where AI is used) users of AI, have a shared vision on its usage in healthcare. In addition, these stakeholder groups should also include those professionals, who support AI implementation at different levels (policymakers, AI experts, health managers and so on). This means that AI implementation should be considered as a multidisciplinary process and engage various clinical teams, healthcare managers, AI experts, policymakers and other stakeholders to enhance its successful adoption. Therefore, the aim of the scoping review is to assess the state of the literature regarding the stakeholders’ views on the organisational factors influencing AI adoption in healthcare, to inform future research agenda in this area and provide further evidence to facilitate a smooth integration of the technology in the sector.

Study rationale

As previously mentioned, various stakeholders influence adoption and implementation of AI in healthcare including health workers, AI experts, pharmaceutical companies, legislative, regulatory, government and public sector bodies. These stakeholders have different and sometimes conflicting interests and perceptions on the benefits, risks, opportunities and limitations of integrating AI in healthcare.^{3 9} There have been several scoping and systematic reviews capturing these various stakeholders’ perspectives on implementing AI in healthcare,^{2 3 9} but they are mostly focusing on clinical outcomes and individual factors shaping AI adoption. However, none of these reviews focused on the wider organisational factors that can facilitate or hinder successful implementation of AI in the sector. Therefore, it is essential to map out the perspectives of the relevant stakeholders on the organisational factors affecting AI implementation in healthcare from the existing primary studies to identify how it works, for whom, and under which circumstances.

Study objective

This study will provide an overview of the perspectives of different relevant stakeholders on the existing organisational barriers and facilitators to the application of AI in healthcare. By conducting this review, we aim to identify the various organisational factors that may enhance AI implementation and usage in the healthcare sector.

Methods and analysis

Theoretical frameworks

In order to support data analysis, categorisation and synthesis of the results, this scoping review will adopt the diffusion of innovations (DoI) theory²⁴ as a guiding conceptual framework. DoI theory has been previously successfully employed to predict how individuals make decisions to adopt a new innovation by exploring their adoption patterns and its structure.^{25 26} Broadly speaking, this theory can help understand why some new technologies spread faster and wider than others while taking into consideration not just individual level, but also team and organisational.²⁷ Rogers defined DoI as a process by which a new innovation is communicated through certain channels over time among the members of a social system and highlighted that adoption of an innovation should be considered as a social process.²⁸ This social process, or the innovation-decision process, comprises five stages (see figure 1), which are: (1) knowledge: individuals or adoption units gain the first knowledge of an innovation; (2) persuasion: individuals or adoption units form an attitude towards the innovation; (3) decision: individuals/adoption units make a decision on whether to adopt or reject an innovation; (4) implementation: an innovation is implemented by individuals/adoption units (which can also be considered as a trial) and finally (5) confirmation: individuals/adoption units verify their decision to adopt or reject an innovation. The process is influenced by the characteristics of the organisation (or decision-making unit) including socioeconomic characteristics, personality variables and communication behaviours, as well as by the perceived characteristics of the innovation such as relative advantage, compatibility, complexity, trialability, and observability.

Figure 1.

Integrated theoretical framework (adapted from Rogers²⁴).

According to Rogers, the first two stages of ‘knowledge’ and ‘persuasion’ are the most critical elements in the innovation-decision process as at these stages, individuals (adopters) and/or decision-making units weight the advantages and disadvantages of a new innovation to reduce uncertainty about its usage.²⁴ This is why as part of this guiding conceptual frameork we will also incorporate technology acceptance model (TAM)²⁹ which focuses on two key factors influencing individuals’ decision about using a new innovation: perceived usefulness (PU) and perceived ease-of-use (PEU). The PU can be defined as ‘the degree to which a person believes that using a particular system would enhance his or her job performance’ and the PEU as ‘the degree to which a person believes that using a particular system would be free from effort’.³⁰ Integrating both of these theories will not only help explore the specific organisational issues of adopting and implementing AI (by using DoI) but also explain the stakeholders’ general perception towards AI use in the healthcare context (by using TAM).

Lastly, given that we aim to map out organisational factors affecting AI implementation in healthcare through the lens of the relevant stakeholders’ perspectives, we will also incorporate the stakeholder theory.³¹ Stakeholder theory provides an alternative to a traditional input/output view of an organisation and it considers the interests of all stakeholders to be of intrinsic value. In its normative form,³² it serves to identify different individuals or groups who have legitimate interest in procedural and/or substantive aspects of organisational activity. In the healthcare domain, stakeholders can be defined as ‘any person or group of people who have a significant interest in services provided, or will be affected by, any planned changes in an organisation or local health community’.³³ Clarkson³⁴ advised that in the context of organisational management, it is useful to distinguish between primary and secondary stakeholders: primary (or participant stakeholders) are the one without whose direct participation the organisation cannot survive, and secondary (or non-participant stakeholders) are those who influence (or can be influenced by organisation) without directly participating in its transactions.³⁵ Stakeholder theory can be linked with the DoI which assumes that there are different types of ‘users’ who are involved at the process of technology adoption at various stages.³⁶ Stakeholder theory expands the notion of a ‘user’ to all individual or organisations that might have an impact or be impacted by the introduction of the innovation.

Protocol design

This study will follow the methodological framework suggested by the Joanna Briggs Institute (JBI),³⁷ which comprises the following stages: (1) defining and aligning the objectives and questions, (2) developing and aligning the inclusions criteria with the objectives and questions, (3) describing the planned approach to evidence searching and selection, (4) searching for the evidence, (5) selecting the evidence, (6) extracting the evidence, (7) charting the evidence, and summarising the evidence in relation to the objectives and questions. Throughout the process, feedback will be sought from the life and medical sciences librarian as well as a medical expert with a related background when required. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension for scoping reviews guidelines (PRISMA-ScR)³⁸ have been followed to ensure that the structure and content of this protocol comprise all the required elements, which is provided in online supplemental appendix 1.

Stage 1: defining and aligning the objective(s) and question(s)

Both the objective of this scoping review and research questions were developed using the PCC mnemonic (p=population, C=concept, C=context) where possible. The population of interest will be any relevant stakeholders who have had experience of directly (eg, using AI as part of a medical treatment) or indirectly (eg, managing a surgery/hospital where AI is used) employing AI in the context of healthcare. We do not have predefined groups of the relevant stakeholders as these will be mapped as part of this scoping review, however, it is expected that these will be comprised of health workers, health managers and decision makers, and AI experts.³⁹ The following four broad groupings proposed by the WHO classification of health workers⁴⁰ will be used: health professionals (ie, those, who study, advise on or provide preventive, curative, rehabilitative and promotional health services based on an extensive body of theoretical and factual knowledge), health associate professionals (ie, those who perform technical and practical tasks to support the healthcare delivery), personal care workers in health services (ie, those who provide direct personal care services in healthcare and residential settings) and health management and support personnel (ie, those who form management and support personnel including a wide range of other types of health systems personnel, such as health service managers, health economists, health policy lawyers, biomedical engineers and so on). As for the concept of interest, any types and subfields of AI employed in healthcare will be considered as this area is currently emerging. Lastly, the context of interest in this study is the general context of healthcare. Based on the objective of the scoping review, we will aim to answer the following primary research question (RQ1) ‘What are the stakeholders’ perspectives on the organisational issues in application of AI in healthcare?’. In addition, three secondary research questions will be used to guide this scoping review: (RQ2) ‘What organisational factors might facilitate or hinder application of AI in healthcare?’, (RQ3) ‘How converging or diverging are the perspectives of different types of stakeholders on application of AI in healthcare?’ and (RQ4) ‘How similar or dissimilar are stakeholders’ perceptions on AI adoption across different healthcare services and functions?’.

Stage 2: developing and aligning the inclusion criteria with the objectives and questions

To identify and refine eligibility criteria as well as formulate the search strategy and search terms, an initial limited search of two appropriate online databases (PubMed and MEDLINE(Ovid)) will conducted to explore what keywords/index terms are used in the abstracts of the retrieved papers. After that, a preliminary search strategy was developed with the help of a University librarian and in conjunction with topic area knowledge as well as discussion among the authors of this scoping review. The preliminary search strategy is shown in online supplemental appendix 2.

The initial search will be limited to studies that are meeting all three PCC criteria and explore the views/perspectives of the relevant stakeholders on AI use in healthcare only. The type of studies that will be included are any primary research studies, which were published in peer-review journals and written in English. Publication date will be limited to dates between 01 January 2000 and 30 June 2021 to include studies that focus on the use of contemporary data-driven AI based on machine-learning techniques as opposed to more traditional rule-based AI (eg, expert systems). Studies reporting only clinical outcomes of using AI in healthcare without taking into consideration any organisational issues will be excluded from the review. In addition, studies exploring the use of ‘ordinary’ computer systems and/or any other non-AI-based technologies in healthcare will also be considered as ineligible. Lastly, we will exclude studies focusing only on technical aspects of using a particular type of AI in healthcare (eg, performance improvements of the AI algorithms).

Stage 3: describing the planned approach to evidence searching and selection

Before commencing the literature search, a pilot search will be performed, and the first 30 documents will be screened on the two online data bases (PubMed and MEDLINE(Ovid)). Two researchers will independently screen the titles and abstracts against the eligibility criteria. Any disagreements will be then discussed by the authors, and if the agreement cannot be reached the opinion of the third author will be sought. The authors will then screen three randomly selected full texts of the eligible articles to pilot-test data extraction and agree on the charting form. Once the data extraction form is refined, the new five randomly selected papers will be reviewed by all three authors to compare and discuss the captured information following the recommendations of Valaitis et al.⁴¹ To assess the level of agreement between the reviewers, inter-rater reliability will be calculated using Cohen’s kappa to iteratively calibrate and refine the process. Overall, this approach will enable the review team to become familiar with the scoping review protocol and minimise the effect of reporting bias.⁴² In addition, it will help ensure that the eligibility criteria are feasible and used by the authors in a consistent manner.

Stage 4: searching for the evidence

Following the recommendations of the JBI,⁴³ the literature search will comprise three steps. During the first step, all the previously selected keywords will be augmented to formulate a comprehensive search strategy. The second step will comprise creating search strings according to the requirements of the databases using Medical Subject Heading terms, Boolean operators, as well as date and language limiters as search filters. The databases that will be searched are MEDLINE (Ovid), CINAHL (Plus), PubMed, Cohrane Library, Scopus, MathSciNet, NICE Evidence, OpenGrey, O’REILLY and Social Care Online. During the final third step, manual search of the reference lists of all the selected papers for full-text review will be conducted in order to identify more relevant studies. The PRISMA flow diagram will be used to record how many studies are included/excluded at each stage of the search process and from what databases.⁴⁴

Stage 5: selecting the evidence

All the search results will be imported onto EndNote bibliographic software (V.9, Clarivate Analytics, Philadelphia, Pennsylvania, USA) and merged. Once the duplicates are screened and removed (both electronically and manually), the titles, abstracts and summaries of the remaining articles will be screened against the set eligibility criteria by two independent researchers (R1 and R2). The reason for excluding each article will be recorded. Any articles with yes/yes or yes/maybe will be advanced to full-text review. Any disagreements will be resolved by the third researcher (R3). Subsequently, the full-text versions of the articles will be reviewed by the researchers R1 and R2 independently.

Stage 6: extracting the evidence

The charted data from the included studies will be managed using an Excel spreadsheet. All data will be extracted by the researchers R1 and R2 independently, and then agreed and merged. Disagreements will be resolved by the researcher R3. The data extraction sheet will include as a minimum the following information: source ID, full citations, authors’ names, title and a year of publications, name of a journal, publication type, study purpose(s), study design, sample size, types of AI being discussed, the healthcare context, the relevant stakeholders, organisational barriers and facilitators, recommendations for organisational change and reviewers’ notes.

Stage 7: charting the evidence, and summarising the evidence in relation to the objectives and questions

To synthesise the results of this review, a narrative synthesis approach will be adopted in accordance with the ‘guidance on the conduct of narrative synthesis in systematic reviews’ and the JBI guidelines. First, preliminary synthesis of the findings will be conducted to identify various patterns across included articles while taking into account the chosen theoretical frameworks. Second, a thematic analysis, the findings will be analysed by following the six-phase thematic analysis framework developed by Braun and Clarke.⁴⁵ These six stages comprise familiarisation with the extracted data, generation of initial codes, identifying and reviewing themes, naming themes and writing-up the results.

Patient and public involvement

No patient involved.

Limitations

Some of the limitations of this study are related to the nature of scoping reviews, which use secondary data and are prone to subjectivity and bias in selection and analysis of primary sources. The authors plan to use the results of this study as a basis for an extended programme of research that will include primary data collection from relevant stakeholders. The intent is also to reduce the potential for subjectivity and bias by involving all three authors in the review process. Moreover, it is expected that different background of the authors (psychology, healthcare management, operational research, computer science) will provide additional lenses for analysing data and synthesising the results. Finally, while the scope of the review is healthcare in general, the authors recognise that the results will need to be contextualised within specific healthcare services and functions.

Discussion

Various AI-based technologies and tools are developing rapidly with many being introduced and deployed in healthcare organisations. Consequently, healthcare organisations need to put in place the necessary strategies and plans to support AI applications and the right infrastructure to facilitate a seamless AI implementation in order to benefit from the technologies. Achieving these objectives requires a holistic approach, which goes beyond the technical aspects of AI to include the organisational management factors influencing its adoption and integration. On that basis, it is important to map out the views of the relevant stakeholders on the organisational consequences of implementing AI to get an understanding of how they react to new advanced technologies, as well as to shed light on how the technology may smoothly fit into healthcare systems and become part of the routine and activities of healthcare services.

Ethics and dissemination

Ethics approval will not be sought for this scoping review as it will only include information from the previously published studies. The results of study will be disseminated through publication in a peer-reviewed journal. In addition, to ensure that its findings reach the relevant stakeholders and wider academic and non-academic communities, they will be presented at relevant conferences.

Study status

The review is ongoing and is expected to be completed by the end of 2021. The authors have now completed Stage 3 ‘Describing the planned approach to evidence searching and selection’ and started Stage 4 ‘Searching for the evidence’ of the protocol design.