Generalization of AI-Based Gestational Age Assessment Using Blind Sweep Ultrasonography

Key Points

Question

Can artificial intelligence (AI) models trained on blind sweep ultrasonography scans generalize to new clinical settings and perform as well as traditional sonographers on estimating gestational age?

Findings

In this diagnostic study of 385 participants, the AI model effectively generalized to new clinical environments and institutions, achieving a mean absolute error of 4.2 days, which was noninferior to the clinical standard.

Meaning

This AI system demonstrated strong potential to expand access to diagnostic tasks such as gestational age estimation, particularly in low-resource settings, by enabling novice operators to perform accurate ultrasonography assessments.

Abstract

Importance

Artificial intelligence (AI) systems have been developed to estimate gestational age (GA) based on blind sweep ultrasonography scans performed by novice operators. However, their generalizability to new clinical contexts outside original training settings requires evaluation.

Objective

To investigate the ability of a previously trained AI system to interpret blind sweep ultrasonography scans for GA estimation to generalize to new clinical environments.

Design, Setting, and Participants

This prospective diagnostic study compares AI model performance against the clinical standard for GA determination. The model was primarily accessed following lightweight adaptation using data from 1 setting. From a broader cohort of 2043 participants (1008 in Chicago and 1035 in Nairobi), datasets included an adaptation set of 120 Chicago participants (split evenly for training and validation) and a primary evaluation set of 385 participants (192 in Chicago and 193 in Nairobi) with GAs of 16 to 36 weeks. The study took place in 2 urban medical institutions, 1 in Chicago, Illinois (March 7, 2022, to May 28, 2024) and 1 in Nairobi, Kenya (January 8 to September 2, 2024). Data were analyzed April 21 to June 4, 2025.

Intervention

A mobile application containing the AI system for GA estimation, with data collected via a protocolized blind sweep procedure by novice operators.

Main Outcomes and Measures

The primary outcome was the mean absolute error (MAE) of the fine-tuned AI’s GA estimation compared with the clinical standard (expert sonographer estimate), using a 1-day noninferiority margin.

Results

The primary evaluation set included 385 participants (mean [SD] maternal age, 31.3 [5.6] years). The adapted model achieved an MAE of 4.1 days (95% CI, 3.5-4.7) in Chicago and 4.3 days (95% CI, 3.8-4.8 days) in Nairobi. Overall, the AI model demonstrated noninferiority to clinical standard estimates (MAE, 4.2 days [95% CI, 3.8-4.6 days] vs 4.5 days [95% CI, 4.1-4.9 days]; P < .001).

Conclusions and Relevance

These findings suggest that the AI model for GA estimation, using blind sweep data collected by novice operators, demonstrated generalization to new clinical settings and a different ultrasonography probe. This study highlighted the potential of AI-based ultrasonography assessment to expand access to care through scalable cross-setting deployment, particularly in low-resource settings.

This diagnostic study investigates the ability of a previously trained artificial intelligence (AI) system to interpret blind sweep ultrasonography scans for gestational age estimation to generalize to new clinical environments.

Introduction

Access to prenatal ultrasonography is essential for monitoring fetal growth and development, enabling early detection of potential complications and ultimately improving maternal and neonatal outcomes.¹ While recommended by the World Health Organization,² traditional ultrasonography requires skilled sonographers and expensive equipment, limiting its availability, particularly in low-resource settings.³ Artificial intelligence (AI) systems that use low-cost portable US devices and require minimal operator expertise have the potential to improve maternal health outcomes by bridging this gap. Among these, blind sweep ultrasonography—a set of protocolized sweeps that does not rely on real-time imaging interpretation—has emerged as a particularly promising approach.^4,5,6

The successful deployment of such systems across diverse clinical environments hinges on their ability to generalize to new settings.⁷ Systems must be able to adapt to different patient populations, variations in image quality due to use of probes of different types or from different manufacturers, and integration with diverse clinical environments and workflows. Robust generalization is paramount to ensure the delivery of consistent and accurate results.

In this study, we focus on the crucial task of gestational age (GA) estimation, a fundamental component of prenatal care. Accurate GA determination is essential for timing obstetric care, including life-saving interventions to prevent preterm and postterm births (and related morbidities), managing high-risk conditions such as preeclampsia, interpretation of specific antepartum tests, and identifying potential growth abnormalities.^8,9 However, existing studies have examined AI performance primarily within the same network of hospitals as the development dataset,^4,10 including our own work to evaluate models for the expert¹¹ and nonexpert⁴ use cases. For evaluation of the nonexpert model, we used a held-out test set, including a fraction of sweeps acquired by novice operators. Building on this work, we prospectively investigate whether the model can generalize to new clinical settings when used by novice operators with a different ultrasonography device.

Methods

Study Design

This prospective, multicenter observational diagnostic study was conducted at 2 clinical sites: Northwestern Medicine (Chicago, Illinois) and Jacaranda Health (Nairobi, Kenya). The study aimed to (1) evaluate the generalizability and performance of AI models for GA estimation from blind sweep ultrasonography scans and (2) develop new AI models for identifying point-of-care ultrasonography findings relevant to pregnancy care (planned future work, not part of this GA validation). All study participants provided written informed consent. This study was approved by the respective institutional review boards. This diagnostic study follows the Standards for Reporting of Diagnostic Accuracy (STARD) reporting guideline. The cohort size was determined by a resource-defined enrollment period to capture a diverse distribution of GAs for secondary and future exploratory analyses.

Participants with GA between 16 weeks 0 days and 39 weeks 6 days (based on the best obstetric estimate derived during an early dating scan, as recommended by the American College of Obstetricians and Gynecologists⁸) were recruited at both clinical sites (Figure 1). To be eligible for this study, individuals were required to be 18 years or older, pregnant (clinically confirmed), and not deemed medically unsafe to participate.

Figure 1. Flow Diagram of Patient-Level Data Flow for Primary Analysis.

GA indicates gestational age.

Primary analysis inclusion (n = 385) required a GA of 16 to 36 weeks at the time of the blind sweep examination, confirmed by early dating scans (Figure 1). This GA range matched the original model’s training distribution. Per American College of Obstetricians and Gynecologists guidelines, expert dating scans occurred before 22 weeks 0 days.⁸ Chicago scans were performed before 14 weeks, which American College of Obstetricians and Gynecologists guidelines define as most accurate for dating, using crown-rump length. In Nairobi, 74 cases (approximately 38%) presented after 14 weeks, necessitating the use of later-term fetal biometry measurements: head or abdominal circumference, femur length, and biparietal diameter.

In addition to the early dating scan, each participant received a standard-of-care ultrasonography within 1 week of the blind sweep. Local experts used traditional machines to derive a Hadlock formula-based “standard procedure estimate” of GA¹² for model comparison. Data collection spanned March 7, 2022, to May 28, 2024, in Chicago and January 8 to September 2, 2024, in Nairobi. Follow-up tracked ultrasonography findings and birth outcomes through delivery, concluding in December 2024.

Previously Trained AI System: Original Clinical Setting and Training Data

The original neural network model was trained on the Fetal Age Machine Learning Initiative (FAMLI) cohort,¹³ including sites in Chapel Hill, North Carolina, and Lusaka, Zambia. Data were collected using standard ultrasonography devices (SonoSite M-Turbo or GE Voluson) and the low-cost, portable ButterflyIQ device, with standard clinical assessments⁸ as ground truth.

The model predicts GA from ultrasonography videos,⁴ using sonographer-acquired “fly-to” cines and blind sweeps without intermediate biometrics. The protocol involves 6 freehand sweeps (3 vertical, 3 horizontal). Twenty-four–frame clips are sampled every other frame from 49-frame blocks. The system outputs a GA (in days) prediction and a variance-based confidence estimate to reject low-quality sweeps. Inverse variance weighting combines clip-level predictions.

Model Fine-Tuning

An initial pilot phase of the study was conducted at the Chicago site to collect a small fine-tuning set of 180 examinations from 120 participants (split into 50% training and 50% internal validation sets). Fine-tuning was performed to adapt the model to the video output of the Clarius C3 transabdominal probes (not seen during original model training) and to include a small amount of data from GAs beyond 36 weeks, to explore its potential performance with a new GA distribution. We chose to fine-tune exclusively on Chicago data to demonstrate the model’s ability to generalize to a new population (Nairobi) without any additional site-specific fine-tuning. The size of the fine-tuning set was chosen to demonstrate the feasibility of a lightweight adaptation of the base model to new hardware, reflecting a realistic deployment scenario in which large-scale local datasets may not be available.

During this fine-tuning process, training batches were constructed with an equal split of the fine-tuning data and the original FAMLI training data. Model optimization was the same as that of the original model, except that the learning rate was scheduled to linearly decay from an initial value of 3.47 × 10⁻⁴ to 4.58 × 10⁻⁷ over the entire fine-tuning period to ensure stable convergence until the tune-set performance was saturated (300 000 optimization steps). The learning rate was selected based on the Vizier optimization system.¹⁴ The fine-tuned model was then deployed in both new clinical settings before primary analysis data collection.

Two operators in Chicago (S.C. and C.A.) and 3 in Nairobi, with no prior ultrasonography experience, acquired B-mode blind sweep scans using Clarius probes. Chicago operators received no formal training, relying instead on a protocol handout, verbal instructions, and a brief demonstration. Based on qualitative feedback from operators at the Chicago site, the training protocol was revised for the Nairobi site: before data collection, operators received 6 hours of hands-on training. Sessions included sweeping demonstrations and application practice on phantoms and volunteers. Training also covered general study protocols (data collection, technical, and recruitment) and shadowing expert sonographers. Both sites used the original protocol’s sweeping procedure: up to 6 predefined ultrasonography sweeps over the gravid abdomen⁴ (eFigure 1 in Supplement 1).

The GA model was installed onto tablet devices via a mobile application developed for this research. This application captured the raw ultrasonography video by interfacing with the Clarius mobile application. The video was processed, and GA predictions were made on the device in real time. Although operators could see the live ultrasonography video, diagnostic results were hidden to prevent clinical influence (see the interaction diagram in eFigure 2 in Supplement 1). Clarius imaging depth was programmatically set based on fundal height: 11 cm for nonpalpable fundi (eg, due to early GA or body habitus), 13 cm for heights below 25 cm, and 15 cm for heights above 25 cm. Clinical leads selected these settings based on expert judgment and Chicago pilot data.

Statistical Analysis

The mean absolute error (MAE) and the mean error (ME) with 95% CIs were calculated for the AI and standard procedure against ground truth GA. For participants with multiple visits, measurements were weighted equally per patient to mitigate correlation. The primary outcome was an MAE comparison via a 1-sided noninferiority test (1-day margin; α = .05), consistent with prior investigations.⁴ We inspected the data for equality of variance, to decide the use of a t test or a Welch t test, and for nonnormal distributions, a Wilcoxon or Mann-Whitney U test was used. Secondary analyses (2-sided P values) evaluated ME bias and subgroups based on dating scan timing (<14 weeks), late GA (37-40 weeks), and study site.

Analysis and inference used Python, version 3.12.11, with NumPy version 2.3.5; SciPy, version 1.16.3; and Pandas, version 2.3.3 (Python Software Foundation). Machine learning models were developed and optimized using Google’s TensorFlow and Vizier. Statistical testing, including noninferiority and Mann-Whitney U tests, was performed through these libraries. Data were analyzed from April 21 to June 4, 2025.

Results

Primary Analysis

The primary analysis included 385 participants (192 in Chicago and 193 in Nairobi). The overall mean (SD) maternal age was 31.3 (5.6) years, and the mean (SD) GA at enrollment was 26.8 (6.5) weeks. Specifically, the mean (SD) maternal age was 33.9 (4.9) years in Chicago and 28.5 (4.8) years in Nairobi; the mean (SD) GA at enrollment was 26.3 (6.3) weeks in Chicago and 27.3 (6.6) weeks in Nairobi (Table 1). Within the broader cohort, Chicago enrolled 1008 participants (1401 scans) with a mean of 1.5 examinations per patient and a maximum of 7.0 (eFigure 3 in Supplement 1), whereas the 1035 participants in Nairobi each completed a single examination. Examinations averaged a mean (SD) of 7.2 (4.1) sweeps, with a 4.9% rejection rate and a mean (SD) of 5.0 (1.3) clips per sweep (Table 1).

Table 1. Per-Site Characteristics and Demographic Characteristics of the Primary Analysis Set^a.

Characteristic	Chicago site (n = 192 participants)	Nairobi site (n = 193 participants)	Overall (n = 385 participants)
Participant characteristics
Maternal age, mean (SD), y	33.9 (4.9)	28.5 (4.8)	31.3 (5.6)
Maternal age, median (IQR), y	34.2 (31.0-37.1)	28.0 (25.0-31.0)	31.2 (27.0-35.0)
GA at enrollment, mean (SD), wk	26.3 (6.3)	27.3 (6.6)	26.8 (6.5)
Participants with multiple examinations, No. (%)	28 (13.0)	0	28 (7.2)
Early dating scan <14 wk, No. (%)	192 (100)	119 (61.7)	311 (80.8)
Early dating scan >14 wk and <22 wk, No. (%)	0	74 (38.3)	74 (19.2)
Operator characteristics
No. of novice operators	2	3	5
Operator training	Handout, verbal instruction, demo	6 h of Formal training	NA
Examination characteristics
Attempted sweeps per examination, mean (SD), No.	8.0 (5.5)	6.4 (1.5)	7.2 (4.1)
Model rejected sweeps, %	7.9	1.8	4.9
No. of clips per sweep, mean (SD)	4.2 (1.0)	5.8 (1.1)	5.0 (1.3)

Abbreviations: GA, gestational age; NA, not applicable.

^a Key baseline demographic, clinical, and procedural characteristics of the study participants and novice operators included in the primary analysis, broken down by the 2 distinct clinical sites in Chicago and Nairobi.

Across Chicago and Nairobi, the model MAE was noninferior to the clinical standard within the predefined 1-day margin (Figure 2). The overall model MAE was 4.2 days (95% CI, 3.8-4.6 days) vs 4.5 days (95% CI, 4.1-4.9 days) for the Hadlock procedure (P < .001). Site-specific analyses also confirmed noninferiority: Chicago (model MAE, 4.1 days [95% CI, 3.5-4.7 days]; Hadlock: 4.6 days [95% CI, 4.1-5.3 days]) and Nairobi (model: 4.3 days [95% CI, 3.8-4.8 days]; Hadlock: 4.3 days [95% CI, 3.8-4.9 days]) (P < .001) (Table 2). The clinical standard demonstrated a statistically significant overestimation of GA (ME, −0.7 days [95% CI, −1.3 to −0.1 days]), whereas the AI model’s ME was not statistically significant (ME, −0.2 days [95% CI, −0.8 to 0.4 days]).

Figure 2. Scatterplots Comparing Artificial Intelligence Model Predictions (A) and the Standard Clinical Procedure (Hadlock) (B) With the Ground Truth Gestational Age (GA).

Table 2. Comparison of Model Performance Against Clinical Standard Procedure (Hadlock Formula) With or Without Fine-Tuning.

Clinical setting	Mean absolute error (95% CI), d		Mean error (95% CI), d		Unique participants, No.
	Model	Hadlock	Model	Hadlock
Primary end point
Nairobi and Chicago, with fine-tuning	4.2 (3.8 to 4.6)	4.5 (4.1 to 4.9)	−0.2 (−0.8 to 0.4)	−0.7 (−1.3 to −0.1)	385
Chicago with fine-tuning	4.1 (3.5 to 4.7)	4.6 (4.1 to 5.3)	−0.7 (−1.5 to 0.1)	−1.0 (−1.9 to −0.1)	192
Nairobi with fine-tuning	4.3 (3.8 to 4.8)	4.3 (3.8 to 4.9)	0.3 (−0.5 to 1.1)	−0.4 (−1.1 to 0.5)	193
Chicago without fine-tuning	5.1 (2.9 to 8.6)	4.2 (3.1 to 5.5)	0.4 (−3.8 to 3.0)	−2.4 (−3.9 to −0.9)	53

Secondary Analysis

Performance by Dating Scan Timing

Because Hadlock-based ground truth is most accurate before 14 weeks,⁸ we analyzed the Nairobi cohort by dating window (119 participants before 14 weeks; 74 participants for 14-22 weeks). For the participants before 14 weeks, the model MAE was 4.2 days (95% CI, 3.6-4.9 days) vs 4.2 days (95% CI, 3.6-4.9 days) for the standard procedure. Performance remained consistent for the participants in the 14- to 22-week group, with a model MAE of 4.4 days (95% CI, 3.4-5.6 days) vs 4.4 days (95% CI, 3.5-5.3 days) (eTable in Supplement 1). Mann-Whitney U testing confirmed no significant performance difference between these early and later dating windows (P = .98).

Performance Without Fine-Tuning in the Chicago Setting

Original FAMLI-only model performance (before fine-tuning) was assessed on an independent Chicago pilot subset (n = 53). The nontuned MAE was 5.1 days (95% CI, 2.9-8.6 days) compared with 4.2 days (95% CI, 3.1-5.5 days) for the standard procedure (Table 2).

Performance Beyond 36 Weeks

Performance was evaluated for cases beyond 36 weeks, which were absent from the original training data. In Chicago, the model MAE was 9.6 days (95% CI, 6.0-13.5 days), and the clinical standard MAE was 10.9 days (95% CI, 7.8-14.0 days) (n = 9); standard-of-care GA assessments were not commonly performed for this subpopulation in this setting. Due to the small paired sample size, we also report the model MAE for a larger Chicago superset (n = 114), including cases without a same-week standard procedure estimate: 7.8 days (95% CI, 6.8-8.7 days). In Nairobi (n = 54), the model MAE was 12.0 days (95% CI, 10.8-13.2 days) vs a clinical standard MAE of 5.0 days (95% CI, 3.0-8.2 days). Notably, the standard procedure MAE in Nairobi was significantly lower than that of the corresponding subgroup in Chicago. This difference likely reflects a lack of clinical blinding in Nairobi, where uninterrupted standard-of-care workflows allowed clinicians prior knowledge of the ground truth GA.

Exploratory Analysis and Performance by Operator

In exploratory analyses of multiple gestations and fetal weight extremes, the model maintained performance comparable to or better than the clinical standard (eMethods and eTable in Supplement 1). The model was more accurate for fetuses above the 90th weight percentile, suggesting AI discerns subtle video features that remain robust against variations in fetal size.

Exploratory analysis evaluated consistency across novice operators (Figure 3). The lowest model MAE was achieved by operator A (n = 55): 3.7 days (95% CI, 2.8-4.5 days) vs 3.6 days (95% CI, 2.7-4.5 days) for standard care. Operator D showed the largest absolute divergence: 4.0 days (95% CI, 3.4-4.7 days) for the model MAE vs 5.2 days (95% CI, 4.4-5.9 days) for the clinical standard MAE.

Figure 3. Bar Graph Showing Clinical Standard and Model Mean Absolute Error (MAE) by Ultrasonography Operator Across Both Clinical Settings.

Discussion

This multisite prospective diagnostic study showed that our AI-based system for GA estimation successfully generalized and maintained reasonable accuracy when deployed by novice operators in new clinical environments with minimal fine-tuning. Originally trained on suburban North Carolina and urban Zambian data, the model was validated in a Chicago urban academic center and a Kenyan urban clinic using a different portable probe. The fine-tuned model’s cross-site MAE of approximately 4.0 days was similar to standard care. This generalizable accuracy, achieved with low-cost probes, represents an important step toward World Health Organization–recommended scalable prenatal implementation.²

Fine-tuning (180 examinations from 120 Chicago participants; approximately 6% of original training size) targeted generalization to new hardware and GA distributions (Methods). The model achieved a 4.2-day cross-site MAE, which was comparable to the original study’s MAE of 4.0 days. Robust Nairobi performance (4.3 days), without local-tuning, mirrored Chicago results (4.1 days) and underscored the model’s inherent adaptability and transfer-learning efficacy.

The lower sweep rejection rate in the Nairobi setting (1.8% vs 7.9% in Chicago) may suggest that approximately 6 hours of formal, interactive, hands-on training improves acquisition quality compared with informal and written instruction. Such structured training optimizes clinical workflows by minimizing model-initiated retries.

The generalization observed in this study is critical for assessing large-scale deployment potential. Clinical sites vary significantly in workflow, staffing, patient demographics, and equipment, alongside a global shortage of expert sonographers in low-resource settings. Our findings address these challenges, demonstrating that AI paired with low-cost, portable ultrasonography and minimally trained operators can achieve clinically acceptable accuracy rates across diverse geographical and infrastructural contexts. Minimal operator training requirements, as detailed in our Methods, underscore deployment feasibility. This system could mitigate disproportionate maternal or fetal comorbidity in low-resource areas by providing access to an essential clinical tool, serving as a template for automating and democratizing other ultrasonography-based diagnostics in future obstetric work.

The deployment of blind sweep AI models presents a significant opportunity to improve diagnostic accuracy in clinically complex situations where conventional methods may be less reliable. For example, the model was substantially more accurate for fetuses above the 90th weight percentile when compared with standard clinical formulas (MAE, 4.4 vs 7.2 days; P = .004), suggesting AI discerns subtle video features that remain robust against variations in fetal size.

Despite this promise, the practical implementation of blind sweep AI has challenges, particularly in low-resource environments, as we learned as part of this prospective diagnostic study. Infrastructural hurdles, such as unreliable power for charging devices and the need for durable hardware capable of withstanding demanding conditions, are significant barriers.¹⁵ Operators struggled with device charging during consecutive examinations, favoring swappable batteries over models requiring extended docking. Furthermore, the system’s interconnected hardware and software necessitate ongoing maintenance and regression testing to ensure postupdate compatibility. Furthermore, these advanced AI models require building trust with clinicians and patients, via transparency, such as implementing model confidence metrics to prevent predictions on low-quality data, and by ensuring the model’s fairness and equity through ethically sourced, diverse training data and validation across multiple sites.

Limitations

This study has several limitations. Although Chicago-based fine-tuning generalized well to Nairobi, the lack of site-specific Nairobi tuning precluded potential local optimization but ensured independent external validation. Furthermore, novice operator training differed between sites, and, whereas generalization was demonstrated for 1 new manufacturer (Clarius), further research is required to confirm performance across a wider range of hardware.

Performance beyond 36 weeks varied notably. Nairobi’s standard procedure MAE was significantly lower than both the model’s MAE and the corresponding clinical MAE at the Chicago site. This disparity suggests potential ground-truth inconsistencies because sonographers were not blinded to dating scan age to preserve standard practice. Alternatively, these findings may indicate a need for model refinement for late-term pregnancies in specific contexts. Additionally, exploratory insights into subgroups such as multiple gestations and fetal weight extremes were limited by small sample sizes, warranting larger dedicated studies.

Although the model showed no statistically significant performance difference from the clinical standard across operators—suggesting robustness to novice variability—the relatively large 95% CIs require further validation. Future research should incorporate a more diverse range of operators to fully assess the system’s robustness and reliability.

Conclusions

In this diagnostic study of AI for GA estimation using blind sweep ultrasonography, we effectively demonstrated generalization to new clinical settings with new novice operators and different probes, and we achieved noninferior performance to the clinical standard, thereby highlighting its potential for expanding access to essential prenatal care. Our findings offer critical insights into the challenges and opportunities for deploying AI in clinical health care settings.

Supplement 1.

eMethods. Exploratory analysis

eTable. Secondary and exploratory comparisons of model performance against clinical standard procedure (Hadlock formula)

eFigure 1. Sweep diagram

eFigure 2. Component interaction diagram of the study exam system

eFigure 3. Distribution of participants per number of blind sweep examinations in Chicago

Supplement 2.

Data Sharing Statement