Table 1.

Overview of the alternative study designs and their application to AI technologies

Study Design	Deviation from traditional RCT design	Benefits for AI technologies
Cluster randomized study	Cluster RCTs randomize groups (e.g. clinics, hospitals, communities) instead of individual participants to different intervention arms.	• Suited for AI technologies supporting group efforts. • Useful when the intervention consists of multiple individual elements that interact with each other • Direct and indirect effects of an intervention can be evaluated.
Micro-randomized study	Micro-randomized studies randomize interventions multiple times per participant (e.g., daily or hourly) to evaluate the short-term effects of just-in-time adaptive interventions.	• Suited to determine the efficacy of a specific component of AI technologies. • Usable especially in the early stages of a product development.
Pragmatic trial	A pragmatic trial is an RCT conducted under typical real-world conditions rather than tightly controlled settings. Pragmatic trials use broad inclusion criteria and embed the intervention into routine care with minimal extra monitoring or protocol-mandated restrictions. Blinding may be relaxed, and multiple real-world sites may be involved.	•Yields high external validity and real-world clinical effectiveness data. •Demonstrates how AI technologies perform in diverse patient populations and workflows. •Supports cost-effectiveness through the use of existing infrastructure and representative outcomes.
Observational trial	No randomization; researchers observe outcomes as they occur in practice. This includes prospective cohorts, registries, or retrospective EHR analyses. Higher risk of confounding compared to RCTs.	• Provides real-world evidence of effectiveness and cost-effectiveness at scale. • Useful for assessing AI technologies using EHR or claims data. • Reflects actual use and outcomes in broad populations.
Target trial emulation	Emulates a hypothetical RCT using observational data. Defines eligibility, interventions, outcomes, and follow-up as if conducting an RCT, but without randomization. Reduces bias through protocol-like structure and confounding adjustment.	• Enhances credibility of real-world evidence. • Allows assessment of AI technologies post-deployment using large-scale datasets. • Faster and cheaper than prospective trials while approximating causal inference.
Zelen’s design	Randomizes patients before consent. In the single-consent version, only the intervention group is informed and consents. The control group gets standard care, often without knowing they were in a study. Intention-to-treat analysis applied.	• Simplifies recruitment and reflects the routine care offer of digital tools. • Minimizes disappointment and contamination bias. • Suitable for low-risk interventions with variable adherence. May support generalizability and policy relevance.
Silent trial	Silent trials refer to prospective, noninterventional assessments of AI models in the intended setting without affecting patient care or institutional workflow.	• Suitable for assessing whether the model will maintain its performance in a live context using local evidence. • Allows testing for potential utility (e.g., pipeline stability, model drifts), as well as financial sustainability of models in real world evaluations without affecting the clinical workflow.