Table 1.
Ranking results of the survey.
| Rank | Barrier | Mean Likert score | Barrier group |
|---|---|---|---|
| 1 | Lack of decision-makers’ expertise about the methods and use of AI driven scientific evidence | 4.03 | H |
| 2 | Lack of appropriate skills for applying AI methods (natural language processing, machine learning etc.) in outcomes research | 3.90 | H |
| 3 | Lack of adequate education to generate AI driven scientific evidence | 3.90 | H |
| 4 | Issues with reliability, validity and accuracy of data (e.g., due to the lack of quality assessment of data entry or self-reporting) | 3.88 | D |
| 5 | Lack of awareness and openness on the part of decision-makers to rely on AI-based real-world evidence | 3.85 | R |
| 6 | Lack of political commitment (e.g., no health digitization strategy in the country to establish relevant databases) | 3.81 | R |
| 7 | Multinational data collection and analysis is limited due to differences in the coding system across countries, and the lack of mapping methods to standardize the vocabulary | 3.68 | D |
| 8 | Lack of resources to build and maintain IT infrastructure to support AI process | 3.68 | T |
| 9 | Regulatory compliance issues in the process of managing a high volume of sensitive information | 3.67 | R |
| 10 | Analysis of multicentre data is limited due to differences in database structures across systems (e.g., electronic medical records database of different service providers) | 3.63 | D |
| 11 | Raw fragmented or unstructured data (e.g., electronic medical records, imaging reports), which are difficult to aggregate and analyse | 3.62 | D |
| 12 | High cost of improving data validity (e.g., data abstracters to evaluate unstructured data) | 3.62 | T |
| 13 | Systemic bias in the data (e.g., due to upcoding) | 3.59 | D |
| 14 | Lack of well-described patient-level health databases | 3.59 | D |
| 15 | Data that are relevant for research purposes (e.g., important clinical endpoints) are missing from databases or are available only on paper. | 3.54 | D |
| 16 | The database is incomplete to fully track patient pathways, leading to inconsistent, unreliable findings | 3.49 | D |
| 17 | High costs associated with securing and storing data for research purposes | 3.47 | T |
| 18 | Data is not transferable across countries for multinational analyses | 3.46 | D |
| 19 | Lack of access to patient-level databases due to data protection regulations | 3.42 | R |
| 20 | Lack of knowledge in data governance: data ownership and data stewardship | 3.42 | H |
| 21 | Lack of transparency of protocols for data collection methods | 3.37 | M |
| 22 | Data cleansing is not feasible | 3.28 | D |
| 23 | Lack of methodological transparency of deep learning models (“black box” phenomenon) | 3.18 | M |
| 24 | Potential bias of AI to favour some subgroups based on having more or better information | 3.15 | M |
| 25 | Sample size of the available databases are low (e.g., databases of health care providers are not linked) | 3.13 | D |
| 26 | Acceptance and consent by patients and medical professionals | 3.13 | R |
| 27 | Text mining and natural language processing algorithms cannot be applied due to the lack of standardized medical terms in the local language | 3.12 | M |
| 28 | The result of analysing complex diseases with AI is difficult to use in health economic models | 3.10 | M |
| 29 | Limited reproducibility due to the complexity of AI methods | 3.06 | M |
D, data related barriers; H, human factor related barriers; M, methodological barriers; R, regulatory and policy related barriers; T, technological barriers.