Knowledge and perception of primary care healthcare professionals on the use of artificial intelligence as a healthcare tool

Abstract

Objective

The rapid digitisation of healthcare data and the sheer volume being generated means that artificial intelligence (AI) is becoming a new reality in the practice of medicine. For this reason, describing the perception of primary care (PC) healthcare professionals on the use of AI as a healthcare tool and its impact in radiology is crucial to ensure its successful implementation.

Methods

Observational cross-sectional study, using the validated Shinners Artificial Intelligence Perception survey, aimed at all PC medical and nursing professionals in the health region of Central Catalonia.

Results

The survey was sent to 1068 health professionals, of whom 301 responded. And 85.7% indicated that they understood the concept of AI but there were discrepancies in the use of this tool; 65.8% indicated that they had not received any AI training and 91.4% that they would like to receive training. The mean score for the professional impact of AI was 3.62 points out of 5 (standard deviation (SD)  =  0.72), with a higher score among practitioners who had some prior knowledge of and interest in AI. The mean score for preparedness for AI was 2.76 points out of 5 (SD  =  0.70), with higher scores for nursing and those who use or do not know if they use AI.

Conclusions

The results of this study show that the majority of professionals understood the concept of AI, perceived its impact positively, and felt prepared for its implementation. In addition, despite being limited to a diagnostic aid, the implementation of AI in radiology was a high priority for these professionals.

Introduction

Artificial intelligence (AI) is defined as ‘the ability of machines to mimic human behaviour’. ¹ It is a set of mathematical models, expressed in the form of algorithms, capable of learning and analysing large amounts of data, in different formats and at high speed, to help in the accuracy and speed of diagnosis, facilitate and streamline clinical care, and support public health interventions, among many other applications in the field of health.^2,3 Despite being recent concepts, the rapid digitisation of healthcare data and the sheer volume being generated means that artificial intelligence (AI) is becoming a new reality in the practice of medicine. ⁴

Currently, AI is already able to support in patient triage, suggest diagnoses and result alerts, ⁵ process natural language to automate clinical documentation by text summarisation, ⁶ make diagnoses from radiological images ⁷ or drawings such as Archimedes’ spiral, ⁸ etc. Several studies have demonstrated the potential usefulness of these AI algorithms in many medical specialties such as ophthalmology, ⁹ dermatology, ¹⁰ neurology ¹¹ and radiology, ¹² the latter being one of the healthcare fields where AI has had a greater impact, especially in chest radiographs.^13,14

Despite being one of the most widely used typologies,^15,16 chest radiography is a complex modality to interpret ¹⁷ and often creates discrepancies between the radiologist's and family physician's assessment.^18,19 The region of Central Catalonia has the particularity of having few radiologists. As a result, primary care (PC) physicians are increasingly required to interpret X-rays. For this reason, the use of AI algorithms capable of interpreting thoracic radiographs approaching the efficiency of a radiologist could represent a great benefit in the clinical setting as a diagnostic support tool.

Knowing how healthcare professionals perceive the use of AI in the healthcare setting is of vital importance in order to tailor the technology to the needs of professionals, align it with their values and make it easy to use.^20–23 Several studies have detected hope and optimism among healthcare professionals in its application, but also some concern about its ethical implications, lack of knowledge and training on its use, which may contribute to some reluctance among professionals.^24–26

Diaz et al., ²⁷ through a survey, observed that the majority of healthcare professionals agree that AI will improve their day-to-day lives, but that work needs to be done on the need for specific AI training. An integrative review by Shinners et al. ²⁸ showed that healthcare professionals were less likely to use AI if they did not trust the technology or understand how it was used to improve patient outcomes. The study by Sit et al. ²⁹ demonstrated that students were aware of the importance of AI in their careers and selected the radiology specialty less due to the perception that AI would replace them in the future. Thus, it is necessary to introduce AI directly into the academic training of healthcare professionals so that they can enter the working world with knowledge of this tool.^30,31

Jha et al., ³² when asked about the impact of AI in primary healthcare information systems, observed that physicians were sceptical about the ability of AI to perform better than humans. Another study ³³ found that healthcare professionals share concerns about data management, knowledge development, deterioration of the doctor–patient relationship, and alteration of the diagnostic and decision-making landscape.

AI applied to healthcare has great potential, although analysing the perception of stakeholders is key to ensuring its correct implementation. Thus, the aim of this study is to describe the perception and knowledge of PC healthcare professionals on the use of AI as a healthcare tool, and its impact in radiology, through a validated survey. The results obtained will be essential to develop strategies aimed at increasing the acceptance of AI use and reducing possible resistance to change.

Methods

Study design and sample

Observational cross-sectional study aimed at all PC medical and nursing professionals in the Central Catalonia Health Region. This region includes the counties of Anoia, Bages, Berguedà, Moianès and Osona, and has approximately 1000 professionals. The survey was sent via corporate email to all these professionals up to three times to obtain the maximum response rate. The period during which the survey was active was from 2 November 2022 to 31 January 2023. The survey was open to all PC medical and nursing professionals of the Catalan Institute of Health who were over 18 years of age and accepted the invitation they received by email. Responses from other professional categories have been excluded.

A minimum of 283 surveys, distributed in the study region were required, in order to estimate with a 95% confidence and a precision of 0.1 points, the values of the two dimensions of the questionnaire, assuming a standard deviation of 1 point. ²²

Sources

A validated survey known as Shinners Artificial Intelligence Perception (SHAIP) ³⁴ was used. The survey deals with two dimensions defined in order to describe Professionals impact of AI and Preparedness for AI with 10 questions using the Likert scale (1 means strongly disagree and 5 means strongly agree). Additionally, there are two descriptive multiple-choice questions about the challenges of incorporating AI and the training they would like to receive, and an initial part was added to obtain sociodemographic information (sex, age, professional category and years of professional experience), as well as questions related to knowledge of the concept of AI, one closed and the other open, training and/or preparation for it, and its impact on radiology.

Permission was requested from the author to use it and translate it into Catalan. Two researchers translated it separately and pooled it, and a third helped to reach consensus in cases of discrepancies in translation. The survey was sent to health professionals in digital format.

Statistical analysis

Categorical variables have been described with absolute frequency and percentage, and continuous variables with mean and standard deviation. Factor analysis (FA) and Cronbach's alpha was used to determine the validity and reliability of two dimensions. In general, Cronbach's alpha of 0.7 is taken as indication of good internal consistency. In some cases, an alpha of 0.5 or 0.6 can be acceptable.^35,36 For the bivariate analysis between the dimensions and the sociodemographic variables, the Student's t-test or analysis of variance with multiple comparisons was used. The analyses were performed with R statistical software version 4.2.1, and the significance level was set at 5%.

Ethical considerations

The study protocol was approved by the University Institute for Primary Care Research (IDIAP) Jordi Gol Health Care Ethics Committee (Code 20/177-PCV). The survey was anonymous, so the patient consent was not necessary.

Results

The survey was sent to 1068 health professionals, of whom 301 responded, obtaining a response rate of 28.2%.

Of the professionals who responded to the survey, the mean age was 46.0 years (SD  =  11.0) with 81.1% female, 47.8% were nursing professionals and 48.5% were medical professionals, the vast majority with more than 10 years of professional experience in their current job (62.5%) (Table 1).

Table 1.

Descriptive analysis of the sample.

	Total sample (N  =  301)
How old are you, in years (N  =  292)**	46.0 (11.0)*
Categorised age (N  =  292)
[18–30)	22 (7.53%)
[30–40)	62 (21.2%)
[40–50)	101 (34.6%)
[50–60)	70 (24.0%)
> = 60	37 (12.7%)
What is your gender?
Female	244 (81.1%)
Male	57 (18.9%)
What health discipline do you work in?
Nursing	144 (47.8%)
Medicine	146 (48.5%)
Others	11 (3.65%)
How many years of experience do you have in your job?
Less than 1 year	11 (3.65%)
Between 1 and 2 years	24 (7.97%)
Between 3 and 4 years	43 (14.3%)
Between 5 and 10 years	35 (11.6%)
More than 10 years	188 (62.5%)
Do you understand what the concept of AI means?
No	43 (14.3%)
Yes	258 (85.7%)
In your work do you use AI to perform or plan care?
No	99 (32.9%)
I do not know	71 (23.6%)
Yes	131 (43.5%)
AI education or training:
None	198 (65.8%)
Self-starting online courses, webinars, conferences, etc.	63 (20.9%)
Job training	30 (9.97%)
University education	10 (3.32%)
Would you like to receive education on AI in health?
No	26 (8.64%)
Yes	275 (91.4%)

Absolute frequency and percentage; n (%).

* Mean and standard deviation. ** Only 292 professionals answered the age correctly.

AI: artificial intelligence.

With regard to the use and knowledge of AI, 85.7% indicated that they understood its concept but saw discrepancies in its use due to the execution and/or planning in their work; 43.5% indicated that they used it, 32.9% that they did not and 23.6% that they did not know. More than half of the respondents indicated that they had not received any training on AI (65.8%) and the vast majority responded that they would like to receive it (91.4%) (Table 1).

Figure 1 describes the main challenges of incorporating AI into the workplace and topics they would like to know more about. On the one hand, the main incorporation challenges were the knowledge and skills of the professional staff (58.5%), interoperability with current systems (48.2%), infrastructure (46.8%), implementation cost (44.2%) and organisational support (43.2%). On the other hand, the main training topics were AI application in healthcare (85.0%), AI ethics in healthcare (50.8%) and general education on AI capabilities (36.2%).

Figure 1.

Challenges of incorporating artificial intelligence into the workplace and topics they would like to know more about.

Across the SHAIP questionnaire items (1: strongly disagree and 5: strongly agree), two dimensions were obtained; professional impact of AI as the mean of the individual scores of questions 1 to 6, and the preparedness for AI as the mean of the individual scores of questions 7 to 10. In a preliminary analysis, we examined the Cronbach's alpha estimation of internal consistency and reliability of two dimensions. The Cronbach's alpha estimation of professional impact of AI was 0.84 (95% confidence interval (CI) 0.79–0.88) which suggests that items had very little variance specific to individual items. The Cronbach's alpha estimation of preparedness for AI was a 0.56 (95% CI 0.46–0.65). Figure 2 shows the results of the FA to validate the two dimensions of survey with study data and study language.

Figure 2.

Factor analysis to validate dimensions of questionnaire.

Table 2 describes the two dimensions related to the professional impact of AI and the preparedness for AI. Professionals scored different items (on a Likert scale) related to each dimensions. The mean score of responses on the professional impact of AI was 3.62 points (standard deviation (SD)  =  0.72) out of 5 on the Likert scale. Thus, they consider that AI will improve their workflow, population health indicators and the sustainability of the healthcare system. Specifically, the two items with the highest scores meaning that they agree were ‘I believe that the use of AI in my specialty could improve clinical decision making’ (3.87 out of 5 mean score with SD  =  0.87) and ‘I believe that AI can improve population health indicators’ (3.81 out of 5 mean score with SD  =  0.86). In contrast, the two lowest scoring items indicated that they were unsure were ‘I believe that one day AI will be able to be part of my job as a healthcare professional’ (3.40 out of 5 mean score with SD  =  1.10) and ‘I believe that the introduction of AI can reduce the economic cost associated with my work’ (3.37 out of 5 mean score with SD  =  097) (Table 2).

Table 2.

Descriptive analysis of the items and dimensions.

	Total sample (N  =  301)
Professional impact of AI	3.62 (0.72)
(1) I believe that the use of AI in my specialty could improve patient care.	3.78 (0.95)
(2) I believe that the use of AI in my specialty could improve clinical decision making.	3.87 (0.87)
(3) I believe that AI can improve population health indicators.	3.81 (0.86)
(4) I believe that AI may change my role as a healthcare professional in the future.	3.51 (0.98)
(5) I believe that the introduction of AI can reduce the economic cost associated with my work.	3.37 (0.97)
(6) I believe that one day AI will be able to be part of my job as a healthcare professional.	3.40 (1.10)
Preparedness for AI	2.76 (0.70)
(7) I believe that most healthcare professionals are prepared for the introduction of AI technology.	2.65 (1.07)
(8) I believe that I have been adequately trained to use AI in my job.	2.02 (1.01)
(9) I believe there is an ethical framework for the use of AI technology in my workplace.	3.23 (1.05)
(10) I believe that if the AI technology makes a mistake, all the responsibility lies with the healthcare professional.	3.15 (1.14)

Mean of scores and standard deviation (SD); m (SD).

AI: artificial intelligence.

On the other hand, the mean score of the responses on the preparedness for AI was 2.76 points (SD  =  0.70) out of 5 on the Likert scale. Thus, professionals are unsure about if they are well prepared to use IA in their workflow, yet they assume their responsibility in the use of AI. Of the four items in the dimension, the one with the highest score meaning that they agree was ‘I believe there is an ethical framework for the use of AI technology in my workplace’ (3.23 out of 5 mean score with SD  =  1.05) and the one with the lowest score indicating that they were unsure was ‘I believe I have been adequately trained to use AI in my job’ (2.02 out of 5 mean score with SD  =  1.01) (Table 2).

Table 3 shows the results of the bivariate analysis between the two dimensions and the main explanatory variables. With regard to the professional impact of AI, healthcare professionals who understand the concept of the AI reported a greater impact of AI than those who did not. Those who have received training such as self-starting online courses, webinars, lectures or conferences have significantly different perceptions (p  =  0.012) of the AI impact to their professional role compared with those that have not had any training. In addition, professionals who want to receive training have reported a greater impact of the AI than those who did not. Otherwise, nursing professionals’ perception of preparedness for AI is significantly different than the ones from medical professionals, with nurses feeling more prepared than doctors (p  =  0.024). Furthermore, professionals who are using AI or did not know have reported that they feel more prepared for AI than those who did not.

Table 3.

Bivariate analysis between dimensions and sociodemographic variables.

	Professional impact of AI	p-value	Preparedness for AI	p-value
Categorised age (N  =  292)		0.628α		0.053α
[18–30]	3.74 (0.59)		3.16 (0.66)
[30–40]	3.63 (0.82)		2.67 (0.86)
[40–50]	3.54 (0.69)		2.72 (0.66)
[50–60]	3.65 (0.75)		2.74 (0.64)
> = 60	3.70 (0.57)		2.86 (0.59)
What is your gender?		0.205β		0.435β
Female	3.59 (0.67)		2.74 (0.66)
Male	3.75 (0.89)		2.84 (0.84)
What health discipline do you work in?		0.427α		0.024α,2
Others	3.86 (1.24)		2.93 (1.11)
Nursing	3.64 (0.69)		2.86 (0.69)
Medicine	3.59 (0.69)		2.65 (0.66)
How many years of experience do you have in your job?		0.758α		0.969α
Less than 1 year	3.52 (0.96)		3.0 (0.85)
Between 1 and 2 years	3.68 (0.91)		2.94 (0.69)
Between 3 and 4 years	3.77 (0.63)		2.81 (0.76)
Between 5 and 10 years	3.52 (0.84)		2.76 (0.81)
More than 10 years	3.61 (0.67)		2.71 (0.66)
Do you understand what the concept of AI means?		0.032β		0.128β
No	3.41 (0.69)		2.91 (0.69)
Yes	3.66 (0.72)		2.74 (0.70)
In your work do you use AI to perform or plan care?		0.194α		<0.001α,3
No	3.60 (0.82)		2.49 (0.66)
I do not know	3.51 (0.63)		2.80 (0.64)
Yes	3.70 (0.68)		2.94 (0.71)
AI education or training:		0.012α,1		0.349α
None	3.54 (0.72)		2.71 (0.67)
Self-starting online courses, webinars, lectures, conferences	3.86 (0.63)		2.80 (0.72)
Job training	3.74 (0.56)		2.96 (0.77)
University education	3.45 (1.27)		2.80 (0.86)
Would you like to receive education on AI in health?		0.006β		0.544β
No	3.21 (0.75)		2.68 (0.68)
Yes	3.66 (0.70)		2.77 (0.70)

^α p-Value of ANOVA contrast.

^β p-Value of the T student contrast.

¹ Post hoc comparison: only none vs self-starting online courses, webinars, lectures, conferences has obtained a significant p-value (p  =  0.011).

² Post hoc comparisons: only nursing vs medicine has obtained a significant p-value (p  =  0.028).

³ Post hoc comparisons: only I don’t know vs no (p  =  0.006) and yes vs no (p < 0.001) has obtained a significant p-value.

AI: artificial intelligence; ANOVA: analysis of variance.

At the end of the survey, they were asked about the priority level for three possible applications of AI in radiology. This was because radiology is one of the specialties where AI has made a significant impact and the region of Central Catalonia is currently in the process of validating AI algorithms for use in radiology. Of the three possible applications, the high percentage of professionals who consider it a high priority to implement AI for image interpretation to obtain immediate preliminary results (41.2%) and the high percentage of professionals who consider that AI should not be implemented for image interpretation to obtain final and definitive patient results (18.9%) stands out (Figure 3).

Figure 3.

Descriptive analysis of priority in radiology artificial intelligence applications.

Discussion

The aim of this study was to characterise the understanding and perception of PC healthcare professionals regarding the application of AI as a tool in healthcare, and to assess its potential impact on radiology. The findings of the present study provide insight into the perception and knowledge of AI as a healthcare tool among PC healthcare professionals in the region of Central Catalonia. Conducting such analyses is relevant for developing future strategies aimed at improving the acceptance and application of AI in a specific region.

Regarding the professional's knowledge of the concept of AI, most of them feel that they understand what it means, and the majority indicates that they do use it in their work. However, although the Catalan Institute of Health, a health provider for 5.8 million users in Catalonia, ³⁷ is working on the validation of AI models applied in imaging with the aim of implementing it in consultations^38–40 and is in an incipient development of different applications, ⁴¹ currently there are still none integrated in the patient's clinical history. Therefore, although the vast majority indicates that they are aware of the concept of AI, there is a general lack of knowledge about its applicability and its possible uses.

In fact, Chen found in the meta-analysis ⁴² that only 10% to 30% of all healthcare professionals used AI in their clinical practice. Moreover, these results are consistent with the results of the meta-analysis ⁴² where it was observed that 74.29% of the included studies indicated a lack of knowledge of the professionals and 15 studies suggested the urgent need to integrate AI into the training programmes of residents. Also, a survey of 3018 medical students in Turkey showed that only 2.8% felt well informed about the use of AI in medicine, ⁴³ and a survey answered by 484 UK medical students showed that 89% of students felt that receiving AI training would be beneficial to their career, as well as 78% believing that AI training was necessary as part of a university degree. ²⁹ Otherwise, various studies in the global literature suggest that incorporating AI education is crucial in overcoming barriers and limitations in the use and understanding of AI.^44–47

In terms of professionals’ perceptions, a positive impact of AI was reported in the survey (scored 3.62 out of 5) similar to that obtained in the Shinners et al.’s ²² study (scored 3.66 out of 5). This agreement was higher for professionals who had some prior knowledge and interest in AI, answering affirmatively that they understood the concept of AI, would like to be trained in AI, or had received some type of training in AI. Therefore, these results may suggest that receiving training may increase sensitivity to how AI can help in their clinical practice. Otherwise, with regard also to the professional's perceptions, over half of professionals considered that they were prepared for AI (scored 2.76 out of 5 in the Likert scale). This result is similar to the one reported in the Shinners et al.’s ²² study that reported a score on the Likert scale of 2.55 out of 5. It has been seen that the nursing category believes it is better prepared for the application of AI than the medical category. It is true that this category is one of the groups that can generate the most data in healthcare systems, as they play a fundamental role in data collection. ⁴⁸ In this context, data collection can become key in the creation of AI algorithms, and it is therefore interesting to take it into account when establishing strategies for adopting AI in the Catalan healthcare system.

Furthermore, it has also been observed that professionals have felt that using AI makes them feel more prepared. Although it has been mentioned that there is currently no integrated AI model in the management programme of the Catalan healthcare system, work is being done on the development of different applications, and there is therefore some awareness of the concept of AI among professionals. Sheng et al. also show that previous experiences with applied health technology and adequate support for professionals make them more interested in using it and training. ⁴⁹

Interestingly, professionals in the Central Catalonia region have considered AI as an ally for improving the quality of care they can provide and have not been so concerned about the possibility of being replaced in the future by AI. This could be due to the long tradition of e-health innovation in this region, being one of the pioneers in the use and application of telemedicine. ⁵⁰ In contrast are the responses from a survey of 3018 medical students from 103 medical schools in Turkey, where the results showed that 58.6% agreed that AI would devalue the medical profession and 45.5% were concerned that the application of AI would damage the fundamental value of trust in medicine. ⁴³

It should be mentioned that the region of Central Catalonia is currently working to validate an AI algorithm for chest X-rays. ³⁸ This region is unique in that it has a shortage of radiologists, which means that PC physicians are increasingly responsible for interpreting X-rays. Given this context, the development of AI algorithms to support X-ray diagnosis, as well as the knowledge and acceptance of healthcare professionals, is essential to integrate and improve workflow for professionals and enhance patient safety. In this context, a question was asked about the priority of incorporating different forms of AI in radiology. It should be taken into account that several studies have mentioned a certain reluctance to adopt AI in radiology due to the fact that in the future there will be a decrease in the demand for radiologists to interpret radiographs, as well as the definitive replacement of radiologists in the future.^29,51 However, both in our study and in other articles in the literature it has been observed that health professionals see clearly that AI should not give a definitive diagnosis to the patient but should only be a tool for quick help and support.^52,53

In this context, it seems that professionals in the territory understand AI as an ally to support diagnosis, rather than as a threat, and therefore strategies for the widespread implementation of AI in this territory should be aimed more at training and infrastructures than at resistance to change.

Limitations and strengths

It is possible that there is a bias in the representativeness of the sample, since 81.1% of the professionals who responded to the survey were women. ⁵⁴ It should be taken into account, however, that this study was carried out in the PC setting of the Catalan Health Institute of Central Catalonia, where 89.55% of the nursing category and 65.72% of the family medicine category are women. ⁵⁵

Furthermore, this is a study with little external validity of the results given the particularities of the territory, which is mostly made up of rural areas. However, this fact brings strength and innovation, as there are few studies on the perception and knowledge of AI in rural areas.

Finally, the SHAIP tool has only be validated in Australian healthcare settings. To control this limitation, the questionnaire was validated with the study sample and language using Cronbach's alpha and factor analysis.

Conclusion

The results of this study show that most professionals indicated that they had a good understanding of the concept, considered AI to have a positive impact, and felt prepared for its implementation. Additionally, although only as a diagnostic aid, the implementation of AI in radiology was considered a high priority by professionals. Observational studies in selected regions are relevant in order to create strategies that can offer training to professionals to increase their acceptance, reduce resistance to change and prepare them for the near future.