Skip to main content
. 2024 May 22;26:e54705. doi: 10.2196/54705

Table 3.

Summary of quality standard–related issues in the reviews.

Key themes Quality issues Reviews
Ethics Guidelines needed, 10 issues—prudence, equity, privacy and intimacy, democratic participation, solidarity, responsibility, diversity inclusion, well-being, respect for autonomy, and sustainable development (Mörch et al [55]); the individual, organizational and society levels of the ethical framework by Asadi et al (Zhao et al [66])

  • Abd-Alrazaq et al [36]

  • Bertini et al [40]

  • Buchanan et al [43]

  • Le Glaz et al [48]

  • Loveys et al [54]

  • Mörch et al [55]

  • Abbasgholizadeh Rahimi et al [58]

  • Seibert et al [61]

  • Zhao et al [66]

  • Zidaru et al 69]
Benefits, cost-effective, economic, external and clinical validation, and clinical utility Need clinical validation and demonstration of economic benefits and clinical utility in real-world settings

  • Bertini et al [40]

  • Eldaly et al [47]

  • Kirk et al [53]

  • Abbasgholizadeh Rahimi et al [58]

  • Sahu et al [59]

  • Seibert et al [61]

  • Syeda et al [62]

  • Vélez-Guerrero et al [64]

  • Welch et al [65]

  • Zidaru et al [68]
Benchmarks—models, data, and performance Need standardized and comparable AIa models, parameters, evaluation measures, and gold standards

  • Barboi et al [38]

  • Bertini et al [40]

  • Choudhury et al [45]

  • Choudhury and Asan [46]

  • Hassan et al [50]

  • Kirk et al [53]

  • Syeda et al [62]
Integration, federated learning, decision fusion, and ability to aggregate results Need to integrate heterogeneous data from multiple sources and combine multiple classifier outputs into a common decision

  • Abd-Alrazaq et al [36]

  • Adamidi et al [37]

  • Bhatt et al [41]

  • Guo et al [49]

  • Kirk et al [53]

  • Popescu et al [57]

  • Vélez-Guerrero et al [64]

  • Zheng et al [67]
Privacy, security, open data, access, and protection Need agreements and processes on privacy, security, access, open data, and data protection

  • Buchanan et al [43]

  • Choudhury and Asan [46]

  • Le Glaz et al [48]

  • Guo et al [49]

  • Loveys et al [54]

  • Seibert et al [61]
Education, web-based learning, learning experience, and competencies Need education for the public, patients, students, and providers, including web-based learning and building competencies and as part of formal curricula

  • Abd-Alrazaq et al [36]

  • Chew and Achananuparp [44]

  • Choudhury and Asan [46]

  • Kirk et al [53]

  • Sapci and Sapci [60]

  • Seibert et al [61]
Explainability Enhance acceptability, understandability, and interpretability of solutions and ability to convey them to patients

  • Abd-Alrazaq et al [36]

  • Adamidi et al [37]

  • Bhatt et al [41]

  • Kirk et al [53]

  • Syeda et al [62]

  • Vélez-Guerrero et al [64]
Reporting standards Standardized reporting of study details to allow for comparison and replication

  • Abd-Alrazaq et al [36]

  • Barboi et al [38]

  • Bertini et al [40]

  • Choudhury and Asan [46]

  • Abbasgholizadeh Rahimi et al [58]

  • Zheng et al [67]
Transparency Need openness and being accountable through the entire AI life cycle

  • Adamidi et al [37]

  • Barboi et al [38]

  • Bhatt et al [41]

  • Chew and Achananuparp [44]

  • Le Glaz et al [48]

  • Huang et al [51]
Trust and trustworthiness Need ethical guidelines to ensure confidence, truthfulness, and honesty with the design, use, and impact of AI systems

  • Adamidi et al [37]

  • Bhatt et al [41]

  • Chew and Achananuparp [44]

  • Huang et al [51]

  • Abbasgholizadeh Rahimi et al [58]

  • Zidaru et al [68]
Safety Need to ensure patient safety from harm

  • Chew and Achananuparp [44]

  • Choudhury et al [45]

  • Choudhury and Asan [46]

  • Seibert et al [61]

  • Zidaru et al [68]
Bias—SDOHb and assessment Need to consider sociodemographic variables and adequate sample sizes

  • Adamidi et al [37]

  • Le Glaz et al [48]

  • Abbasgholizadeh Rahimi et al [58]

  • Syeda et al [62]

  • Vélez-Guerrero et al [64]
Co-design and engagement—user, provider, and public Meaningful participation at all life cycle stages

  • Buchanan et al [42]

  • Buchanan et al [43]

  • Huang et al [51]

  • Abbasgholizadeh Rahimi et al [58]

  • Zidaru et al [68]
Technology maturity or feasibility, acceptance, and usability Need user-friendly and mature AI systems with proven benefits to increase adoption

  • Chew and Achananuparp [44]

  • Eldaly et al [47]

  • Seibert et al [61]
Regulation and legal Need legal framework and laws to ensure appropriate safe use and liability protection

  • Bhatt et al [41]

  • Choudhury and Asan [46]

  • Abbasgholizadeh Rahimi et al [58]

  • Seibert et al [61]
Context and time dependency Purpose of AI models, health care context, and time lags have mediating effect

  • Choudhury et al [45]

  • Kaelin et al [52]

  • Payedimarri et al [56]
Data integration and preprocessing Need to integrate different variables and include multilevel data preprocessing to reduce dimensionality

  • Guo et al [49]

  • Kirk et al [53]
Design justice, equity, and fairness Need design justice principles to engage the public and ensure a fair and equitable AI system

  • Buchanan et al [43]

  • Zidaru et al [68]
Personalized care and targeted interventions Select the best AI algorithms and outputs to customize care for specific individuals

  • Battineni et al [39]

  • Kaelin et al [52]
Quality—data and study Need well-designed studies and quality data to conduct AI studies

  • Talpur et al [63]

  • Welch et al [65]
Social justice and social implications Need to balance human caring needs with AI advances, understanding the societal impact of AI interventions

  • Buchanan et al [43]
Governance Need governance on the collection, storage, use, and transfer of data; being accountable and transparent with the process

  • Choudhury et al [45]
Self-adaptability Need adaptable and flexible AI systems that can improve over time

  • Vélez-Guerrero et al [64]

aAI: artificial intelligence.

bSDOH: social determinants of health.