Cross-Domain Transfer of Depression Voice Biomarkers Depends on the Outcome Instrument: Leakage-Controlled Cross-Sectional Evaluation Study

Abstract

Background:

Voice-based depression screening is commercializing rapidly, yet whether voice biomarkers generalize across clinical settings is largely untested. Generalization is usually framed as a question about populations, but it is also a question about the outcome instrument a model is scored against: reported accuracy varies with the diagnostic criteria a study adopts, but confounded with everything else differing between studies.

Objective:

We tested whether depression voice biomarkers transfer between perinatal and general-psychiatric speech in both directions under strict leakage control, and whether transfer depends on which self-report instrument defines the outcome.

Methods:

In a cross-sectional US-nationwide online study, 446 sessions from 390 pregnant participants at 22±1 weeks’ gestation (analytical sample N=316, one per participant) each provided four voice recordings, the eight-item Patient Health Questionnaire (PHQ-8), and a modified nine-item Edinburgh Postnatal Depression Scale (mEPDS-9). We extracted 375 acoustic descriptors and 330-dimensional Population Dynamics Foundation Model (PDFM) community-health embeddings. Discrimination (area under the curve, AUC) was assessed under leakage-controlled cross-validation, with feature and classifier selection inside training folds only, gated by a permutation negative-control harness, across a pre-specified grid of 4 speech tasks × 4 outcome definitions. The prenatal-trained model was applied to two general-psychiatric corpora (DAIC-WOZ, N=189; E-DAIC, N=219) under pure held-out inference, and an open-source model (DAM) trained on ~35,000 individuals was applied to all three cohorts without refitting. Multiplicity was controlled by the Benjamini–Hochberg procedure.

Results:

The pre-registered within-cohort outcome performed at chance (AUC 0.494, 95% CI 0.431–0.560), and no cell of the 16-cell grid survived multiplicity adjustment under either modeling paradigm. The prenatal-trained model did not transfer to either general-psychiatric corpus (AUC 0.505, 0.478). Transfer measured in the reverse direction, however, varied with the outcome instrument: DAM reached AUC 0.706–0.708 on general-psychiatric speech, 0.510 (0.411–0.609) against the PHQ-8 in pregnant women, and 0.645 (0.541–0.744) against the mEPDS-9 in the same women; paired difference 0.135 (0.019–0.248; unadjusted post-hoc p=0.021). An exploratory item-level analysis suggests a mechanism: DAM tracked affective-distress items (crying r=0.26, sadness r=0.21, overwhelmed r=0.20) but not the somatic PHQ-8 items, though only crying survived adjustment across 17 tests. Transfer also varied descriptively by elicitation task (pregnancy narrative 0.653, 95% CI 0.550–0.753; open-ended speech 0.597, 0.486–0.699; scripted reading 0.544, 0.434–0.651), not compared formally.

Conclusions:

Whether a depression voice biomarker transfers may depend not only on the population but on whether the outcome instrument captures what the model can detect: in the same pregnant participants, a general-population speech model discriminated better against the mEPDS-9 than against the PHQ-8. Validation of voice-based mental-health tools should state the population, the elicitation task, and the outcome instrument together, and a tool validated against one instrument should not be assumed to work against another.

Trial Registration:

Preregistered on the Open Science Framework.[1]

Introduction

Perinatal depression affects up to 20% of pregnant individuals and is a leading cause of maternal morbidity and pregnancy-related death in the United States, yet only about half of antenatal cases are identified during pregnancy and fewer than 10% receive adequate treatment.[2–4] Postpartum-depression prevalence roughly doubled over the past decade in large US cohorts (9.4% in 2010 to 19.0% in 2021 in a Kaiser Permanente Southern California cohort of 442,308 pregnancies), with the steepest increases among Asian/Pacific-Islander and non-Hispanic Black participants.[5, 6] Current screening commonly relies on standardized self-report questionnaires such as the EPDS and PHQ-9, administered or reviewed within clinical care and recommended during perinatal care,[7] but real-world adherence is variable and even adherent screening can miss interval mood changes. Scalable, low-burden adjuncts that integrate into existing workflows are needed.

Voice carries a biological signal of depressive state: major depression produces psychomotor retardation and altered laryngeal tone, manifesting as reduced fundamental-frequency variability, longer pauses, reduced speech rate, and elevated jitter/shimmer.[8–10] Pregnancy itself produces overlapping but distinct phonatory changes via progesterone-mediated mucosal effects and altered respiratory mechanics.[11–13] This overlay motivates the usual framing of the question: are depression voice biomarkers population-specific: do models trained on general-population speech generalize to perinatal speech, where the phonatory baseline is itself perturbed? We test that question in both directions, and add one the literature has left confounded. A systematic review of 25 speech-based depression studies reports that diagnostic performance varied with the diagnostic criteria used, alongside sample size, validation strategy and language.[14] Because those criteria differ between studies at the same time as the cohorts, the recordings and the models do, the instrument’s contribution cannot be separated from the rest; we are not aware of a test that holds participants, recordings and model fixed and varies only the instrument. Every such model is scored against a self-report instrument, and instruments differ in what they measure; in pregnancy in particular, somatic-weighted instruments capture symptoms that gestation elevates independently of mood. Generalization may therefore depend not only on who is speaking but on what the outcome measure is asking. Our cohort carries two instruments on the same participants, which allows population and instrument to be separated in one direction.

We address this and a second, methodological question. This cohort was recruited cross-sectionally at the single mid-gestation timepoint of 22±1 weeks, so the findings concern antenatal mid-gestation depression specifically. To enrich acoustic screening without added patient burden, we incorporated community-level health embeddings from Google’s Population Dynamics Foundation Model (PDFM),[15] which summarizes socioeconomic conditions, healthcare access, and community-health patterns for US ZIP codes already captured in medical records.

We tested generalizability against two independent general-psychiatric datasets (DAIC-WOZ, E-DAIC) and an open-source deep-learning model trained on ~35,000 individuals.[16] We had two hypotheses. First, that depression voice biomarkers would prove population-specific, transferring poorly between prenatal and general-psychiatric contexts. Second, that within a prenatal cohort, acoustic features fused with PDFM embeddings would yield a signal strong enough to be deployable. Both were tested under a strict leakage-control protocol: an sklearn Pipeline fitted inside cross-validation, no pre-split class balancing, no operating-point selection on reported predictions, and a 20-permutation negative-control harness. We offer that protocol as a methodological template.

Methods

Study Design and Participants

This was a cross-sectional US-nationwide online study of pregnant individuals at 22±1 weeks’ gestation. The study was pre-registered on the Open Science Framework (OSF).[1] Participants were recruited via Cint, an online survey-recruitment platform that maintains a verified panel of US-based participants,[17] and directed to the secure Databoard platform to consent and complete the study. The study was deemed “not human subjects research” by the University of California San Diego Institutional Review Board (IRB#812757) as it collected no identifiable information.

Participants were included if female, aged 18–45, pregnant at 22±1 weeks, able to respond via a computing device with internet access, and without current suicidal ideation.

Voice Recording Protocol

Participants completed four vocal tasks via internet browser: (1) a 10-second sound check, (2) a 1-minute free speech prompt on pregnancy experiences, (3) a 2-minute structured reading passage, and (4) a 1-minute free speech prompt on a non-pregnancy topic. Tasks 2 and 4 were counterbalanced across participants to control for order effects (Multimedia Appendix 2).

Depression Assessment

Depression was measured at the single 22±1-week timepoint using the eight-item Patient Health Questionnaire (PHQ-8) and the Edinburgh Postnatal Depression Scale[18] with item 10 (suicidality) removed (IRB-mandated).[19, 20] Binary classification used ≥10 and ≥13 thresholds for both instruments. The pre-registered primary outcome is Reading Passage with PHQ-8≥10; all mEPDS-9 analyses are secondary or exploratory, and all four thresholds are reported.

A caution about the mEPDS-9 thresholds is required. The instrument as administered used a generic four-point agreement scale, not the item-specific response anchors of the published EPDS, and omitted item 10. Correcting the scoring restores a monotone symptom score, but it does not establish that the psychometric properties of the published EPDS (including the validation of its ≥10 and ≥13 cutoffs) transfer to a nine-item instrument administered with different response options. We therefore treat mEPDS-9 scores as a reconstructed symptom score, and the ≥10 and ≥13 values as operational thresholds mapped from EPDS convention, not as validated diagnostic cutoffs for this administration.

Rationale for a Single-Timepoint Screening Outcome

The study evaluates voice analysis as a screening adjunct, and its real-world comparator is exactly what was administered here: a single self-report screening instrument at a single clinical touchpoint, which is how perinatal depression screening is delivered in practice.[7] A screen-positive result is not a diagnosis. Individual-participant-data meta-analyses report that against semi-structured diagnostic interview the PHQ-9 at ≥10 achieves sensitivity 0.85 (95% CI 0.79–0.89) and specificity 0.85 (0.82–0.87),[21] and the EPDS at ≥10 sensitivity 0.85 (0.79–0.90) and specificity 0.84 (0.79–0.88).[22] That is the performance level a screening adjunct would have to approach to be useful, and it is far above anything observed here.

Two caveats attach to the EPDS figure. First, the same meta-analysis found that combined sensitivity and specificity is maximised not at ≥10 but at ≥11; we use ≥10 because itis the conventional threshold and the one specified in our pre-registration, not because itis the optimal operating point in that analysis. Second, and more importantly, these figures apply to the published instruments as validated, and cannot be assumed to transfer to the modified administration used here (see below). Throughout, the outcome is described as screen-positive depressive symptomatology rather than depression as a diagnostic entity.

Both instruments were self-administered online, unassisted, in the same session as the voice recordings; no clinician was involved in administration or scoring. The platform presented, for all nine mEPDS-9 items, a single generic four-point agreement scale (“Yes, all of the time” / “Yes, most of the time” / “No, not very often” / “No, not at all”) instead of the item-specific anchors of the published EPDS.

Acoustic Feature Extraction (the “privacy tier”)

We extracted 375 audio-only acoustic descriptors per recording, composed of four feature blocks. Block sizes and the open-source toolboxes used are stated explicitly:

The 375 descriptors comprise four blocks: a 325-feature base-acoustic block (13 MFCCs, 26 log-Mel-filterbank energies, 26 spectral subband centroids, each summarized by five statistics) adapted from the open-source Allie ML framework’s tier-1 audio pipeline;[23, 24] 22 original voice-quality features (F0, jitter, shimmer, HNR, formants) via parselmouth/Praat;[25, 26] 12 extended voice-quality distributional moments; and 16 talking-interval/timing features. Per-block extraction detail is in Multimedia Appendix 3.

These features measure how speech is produced; spectral shape, voice-source characteristics, and timing, and require no speech transcription, no language model, and no semantic content. We refer to this block collectively as the “privacy tier” throughout the main-text Results, the calibration grid, the negative-control harness, and the leakage-controlled TPOT-CV sensitivity analysis (Multimedia Appendix 8). Audio loading uses soundfile via a thin wrapper (pipeline_a/audio_loader.py).

Within each cross-validation training fold, the 375-feature vector was reduced to 20 features by univariate filter feature selection (using SelectKBest with the ANOVA F-statistic from scikit-learn), with no information from the held-out test fold used in the selection. Technical details of the per-block extraction pipelines, together with documentation of two additional feature sources (TELL linguistic features and Kintsugi DAM scores) that are deliberately excluded from the headline analysis, are in Multimedia Appendix 3.

Community Health Embeddings

PDFM provides 330-dimensional embeddings per US ZIP Code Tabulation Area (ZCTA), summarizing community-level socioeconomic conditions, healthcare access, environmental factors, and health patterns from publicly available data.[15] We merged pre-trained embeddings for 32,069 US ZCTAs with participant data; 88.9% of participants had a matching ZCTA embedding, and unmatched participants were excluded from the headline analytical sample (see Figure 1). As with the acoustic features, the 330-dimensional embedding was reduced to 20 features by SelectKBest fit on the training fold only.

Figure 1: CONSORT-style flow diagram of cohort assembly for the primary (Reading-Passage, PHQ-8 ≥ 10) analysis.

The participant-linkage line in the top box is descriptive, not a filtering step. Exclusions are shown at the right of each step and are mutually exclusive, so the counts are additive down the chain (446 − 22 − 47 − 61 = 316). Within the analytical sample, 312 sessions have a complete item-level PHQ-8 and 313 a complete item-level modified EPDS-9; the small shortfall is reported with each analysis rather than applied as a further cohort filter. The dashed box is the comparator-model analysis set, a parallel branch rather than a nested subset: that model requires at least 30 seconds of audio from any eligible task but needs neither a ZIP-code match nor the reading-passage recording, so the same participant-grouping exclusions are applied to a different audio requirement; 115 of its 136 sessions also appear in the N = 316 sample. Pertask sample sizes for the secondary analyses are in Multimedia Appendix 7.

Critically, PDFM uses only the ZIP code already present in medical records. It requires no additional demographic questionnaires, no individual-level data collection, and no patient interaction; making it a zero-burden enhancement applicable in any clinical setting with access to a patient’s address.

We retained ZCTA-level instead of coarser (e.g. urban/rural) granularity; a parallel large-cohort PRAMS analysis indicates binary urban/rural stratification of PDFM underperforms finer schemes, so coarse aggregation discards signal.[27]

Model Development under Leakage-Controlled Cross-Validation

Acoustic features and PDFM embeddings were combined by late fusion: a separate StandardScaler → SelectKBest → classifier pipeline was refit per cross-validation fold on each modality’s features, and the two arms’ out-of-fold positive-class probabilities were averaged with equal weight to form the fused score (no meta-learner, no fitted fusion weights). The analysis uses a strictly leakage-free protocol (Multimedia Appendix 1): an sklearn Pipeline containing StandardScaler → SelectKBest → classifier was passed to cross_val_predict with 5-fold stratified cross-validation, ensuring scaling and feature selection are refit on each training fold. Class imbalance was handled by setting the classifier’s class_weight parameter to ‘balanced’; no pre-split downsampling was applied. Three evaluation protocols are reported. (a) A parsimonious single-classifier model using L2-regularized logistic regression, which provides the headline estimate. (b) A sensitivity analysis using nested cross-validation that selects between logistic-regression, gradient-boosting, and random-forest candidates inside an inner cross-validation loop on training data only. (c) An automated machine-learning (AutoML) sensitivity analysis using TPOT,[28] in which the pipeline itself; preprocessing, feature transformation and classifier; is searched rather than fixed in advance. In (c), outer folds are participant-grouped (StratifiedGroupKFold), StandardScaler and SelectKBest(k=20) are fitted on outer-training rows only, and TPOT runs its own internal cross-validation on those rows to select a pipeline, leaving the outer-test partition untouched until prediction; the search budget was three minutes per outer fold with the linear-light search space. Protocols (b) and (c) are the two modeling paradigms across which the pre-specified grid is reported; the full AutoML specification is in Multimedia Appendix 8. Because the analytical sample retains at most one session per participant (dual-session-artifact participants were removed entirely; Figure 1), cross-validation folds contain no within-participant repeated measures; with one session per participant, grouping by participant identifier (GroupKFold) is equivalent to the stratified folds used here, so no participant-level leakage is possible.

Operating-point metrics used the fixed 0.5 threshold and a sensitivity-targeted threshold. No threshold was tuned to improve reported performance. The sensitivity-targeted threshold was selected inside training folds only: the threshold reaching training-fold sensitivity ≥80% was chosen on training rows and applied unchanged to held-out rows. This is a leakage control, not an optimisation step; choosing an operating point on the same predictions that are then reported would inflate every metric derived from it. Discrimination (AUC) is threshold-independent in any case, so the reported AUCs do not depend on either choice. PPV/NPV were Bayes-recalibrated to 12–30% prevalence (Table 1). Bootstrap 95% confidence intervals (1,000 resamples) were computed for all AUC estimates, with pairwise contrasts by DeLong’s test.[29]

Table 1:

Operating-point metrics for the pre-registered configuration, with deployment-prevalence Bayes recalibration.

Config	Prev	Sens	Spec	PPV	NPV	NNS	LR+	LR−
RP / privacy + PDFM / PHQ-8≥10	55.8%	0.638	0.377	0.563	0.452	1.8	1.02	0.96
RP / privacy + PDFM / PHQ-8≥10	12%	0.638	0.377	0.122	0.884	8.2	1.02	0.96
RP / privacy + PDFM / PHQ-8≥10	15%	0.638	0.377	0.153	0.855	6.5	1.02	0.96
RP / privacy + PDFM / PHQ-8≥10	20%	0.638	0.377	0.204	0.806	4.9	1.02	0.96
RP / privacy + PDFM / PHQ-8≥10	25%	0.638	0.377	0.254	0.757	3.9	1.02	0.96
RP / privacy + PDFM / PHQ-8≥10	30%	0.638	0.377	0.305	0.708	3.3	1.02	0.96

LR+ = 1.02 and LR− = 0.96; essentially non-informative, so PPV approximates prevalence and NPV approximates 1-prevalence at every deployment prevalence, consistent with an AUC of 0.494. Sens, Spec, LR+ and LR− are prevalence-independent; PPV, NPV and NNS are recomputed by Bayes’ theorem across the 12–30% deployment range (first row shows in-cohort prevalence). Row format is task / feature set / outcome. Abbreviations: RP, reading passage; privacy + PDFM, privacy-tier acoustic features fused with Population Dynamics Foundation Model community-health embeddings; Prev, prevalence; Sens, sensitivity; Spec, specificity; PPV/NPV, positive/negative predictive value; NNS, number needed to screen; LR+/LR−, positive/negative likelihood ratio.

Negative-Control Harness

Before reading the AUC for any pipeline we ran a 20-permutation negative-control harness (shuffled-label, random-feature, and joint-shuffle controls); a clean pipeline must produce AUC ≈0.5 in all three conditions, and all pre-registered and exploratory configurations passed. Full harness specification is in Multimedia Appendix 1.

External Validation

We evaluated prenatal-trained models on two independent general-psychiatric datasets: DAIC-WOZ (N = 189) and E-DAIC (N = 219), both semi-structured clinical interviews of mixed-gender adults, and benchmarked against the Kintsugi Depression/Anxiety Model (DAM), an open-source model fine-tuned from Whisper on ~863 hours of calls from ~35,000 individuals (Multimedia Appendix 4).[16, 30, 31] All cross-domain AUCs reflect pure inference: no refitting, recalibration or threshold selection was performed on the target cohorts, and cross-domain operating points use the fixed 0.5 threshold. For the prenatal→external direction, the privacy-tier acoustic block with a leakage-controlled logistic-regression pipeline (StandardScaler → SelectKBest(20) → class-weighted L2 LR) was fitted on the Reading-Passage rows of the analytical sample carrying PHQ-8 labels (N=312; Figure 1) and applied to native DAIC-WOZ and E-DAIC features extracted by the identical pipeline (375/375 feature alignment); PDFM is unavailable externally, so the acoustic arm is used (implemented in the external-validation scripts; e.g. run_prenatal_to_edaic.py).

Statistical Analysis

Data were tested for normality with appropriate two-tailed group comparisons at α = 0.05; demographic confounding was assessed with effect sizes and inverse probability weighting (Multimedia Appendix 5).

Multiplicity.

Two families of tests are adjusted for multiple comparisons: the pre-specified 16-cell task × outcome grid, and the comparator-model cells evaluated on the prenatal cohort. Within each family, one-sided p values for H₀: AUC = 0.5 were obtained from the Mann–Whitney statistic on out-of-fold predictions and adjusted by the Benjamini–Hochberg procedure.[32] We report one adjustment procedure, not several. Benjamini–Hochberg controls the false discovery rate instead of the family-wise error rate and is the less conservative of the two procedures in common use; at this sample size a family-wise correction would have little power against effects of the size at issue, and reporting both procedures would invite a reader to adopt whichever is convenient once the result is known. The two families are adjusted separately because they address distinct analytic questions. Pipeline scripts, the negative-control harness, and per-(task×threshold) result JSONs are available on reasonable request (see Data Availability).

Ethical Considerations

The University of California San Diego Institutional Review Board determined that the study did not constitute human-subjects research (determination IRB#812757). We state the determination without restating the rationale for it; we note that ZIP code, which this study uses, can be identifying in combination with other attributes. Participants reviewed and agreed to a digital consent statement on the secure Databoard platform before any data collection.

Current suicidal ideation was an exclusion criterion applied by the recruitment panel at eligibility screening, before enrolment and before any study instrument was administered; the study team did not administer a suicidality item at any point, and the suicidality item (EPDS item 10) was omitted from both administered instruments per the IRB determination. Because no suicidality item was administered within the study, no in-study endorsement could occur and collection, and a deployed system would require a separate suicidality-management pathway. No compensation details, identifiable demographics, or contact information were retained, and the terms of consent did not authorize sharing of the prenatal voice data.

Results

Participants

The full data assembly comprised 446 sessions from 390 unique participants. Two participant-grouping artifacts defined the analytical sample: 31 participants contributed duplicate sessions submitted 6–24 minutes apart (often with scores differing by 10+ points and crossing the ≥10 cutoff in opposite directions), treated as platform artifacts and excluded; and 16 sessions missing protocol-specified audio were excluded. The analytical sample is N = 316 (Figure 1). Participants were aged ~30±7 years with no age difference between groups. Cohort prevalence was elevated relative to general perinatal populations (PHQ-8≥10 55.8%, modified-EPDS-9≥10 73.5%), reflecting online-recruitment selection toward depressive symptomatology. Both instruments were administered in IRB-approved truncated form (suicidality item omitted). Both instruments show good internal consistency (Cronbach’s α = 0.882 for the PHQ-8 and 0.881 for the mEPDS-9; McDonald’s ω = 0.883 and 0.879) and agree with one another at Pearson r = 0.660 (Spearman ρ = 0.639; Cohen’s κ at ≥10 = 0.41), comparable to the median PHQ-9–EPDS correlation of r = 0.59 reported across perinatal validation studies.[33] One instrument’s continuous score predicts the other’s ≥10 binary at AUC 0.80–0.84.

Within-Cohort Screening Performance

Under the leakage-controlled protocol on the analytical sample, the pre-registered outcome; PHQ-8≥10 on the Reading Passage task, privacy-tier acoustic features fused with PDFM: performed at chance: AUC 0.494 (95% CI 0.431–0.560). The acoustic arm alone reached 0.452 and the PDFM arm alone 0.541, so the voice increment over community context alone was negative (ΔAUC = −0.047). At the fixed 0.5 threshold, sensitivity was 0.64 and specificity 0.38, giving LR+ = 1.02 and LR− = 0.96. Likelihood ratios this close to 1 mean a positive or negative result leaves a patient’s estimated probability of depression essentially where it started, and Bayes-recalibrated PPV accordingly approximates prevalence at every deployment prevalence examined (Table 1). The same cell under the AutoML pipeline-search protocol was also at chance (AUC 0.540, 95% CI 0.477–0.603), so the null does not depend on the choice of classifier or on fixing the pipeline in advance. The configuration passed the permutation negative-control harness (real 0.505, shuffled labels 0.508, random features 0.504), confirming that the null reflects absent signal and not a broken pipeline. The harness applies a single fixed logistic-regression pipeline to the concatenated feature matrix, and not the nested classifier-selection and late-fusion procedure used for the reported estimate, which is why its “real” value (0.505) differs slightly from the headline AUC (0.494); the comparison of interest is between the real run and its own shuffled controls, not against the headline.

The full pre-specified grid, with multiplicity adjustment.

Across all 16 task × outcome cells, no cell reached nominal significance under nested cross-validation (maximum AUC 0.550); one cell reached the nominal level under AutoML pipeline search (maximum AUC 0.569, raw p = 0.020) but did not survive adjustment. Every confidence interval covered 0.5 under nested cross-validation. After Benjamini–Hochberg adjustment across the grid, no cell survives under either modeling paradigm (Table 2). Under a global null the expected number of cells clearing the nominal 5% level is 0.8, which is what we observe.

Table 2:

Multiplicity across the pre-specified 16-cell grid (4 speech tasks × 4 outcome definitions), under two modeling paradigms.

Paradigm	Max AUC Cells	p < 0.05	Surviving BH
AutoML pipeline search	0.569	1	0
Nested cross-validation	0.550	0	0

p is one-sided for H₀: AUC = 0.5, from the Mann–Whitney statistic on out-of-fold predictions; adjustment is by the Benjamini–Hochberg procedure across the 16 cells within each row. Under a global null the expected number of cells clearing the nominal 5% level is 0.8. Abbreviation: BH, Benjamini–Hochberg.

Cell-level estimates in this grid are unstable.

Repeating the AutoML search under protocol variations reproduces individual cell values only loosely: mean absolute difference 0.036, maximum 0.081, and correlation across the 16 cells r = 0.376. The search is stochastic under a wall-clock budget, and cell values additionally shift with the fold-grouping scheme and with whether item-complete subsets are used, so no single cell of this grid should be interpreted on its own. This is why the grid is reported in full with multiplicity adjustment rather than as a single best-performing configuration; selecting a primary outcome by choosing the highest cell would be reading noise.

Sensitivity analyses.

A reduced 325-feature acoustic tier fused with PDFM performed no better (AUC 0.501 at the pre-registered outcome). Removing the participant-grouping filter did not change the conclusion (N=372, AUC 0.540). On the subset with acoustic, DAM and PDFM features jointly available (N=115), no block among seven combinations reached significance (maximum 0.576), and adding DAM to voice+PDFM produced no increment in any cell (largest ΔAUC = +0.034, 95% CI −0.012 to +0.083). Adding the individually collected sociodemographic variables (age, education, employment, race, household income, relationship status and smoking) did not improve performance (ΔAUC = +0.009 over voice + PDFM; the sociodemographic block alone reached 0.508), nor did semantic transcript features.

Transfer Depends on the Outcome Instrument, Not Only the Population

Cross-domain evaluation is a compound shift: population, elicitation task and outcome instrument all differ at once, and a design that varies them together cannot say which one matters. In one direction, however, we could hold population and task fixed and vary only the instrument, because our cohort carries two outcome measures on the same participants. That comparison is the central result of this study.

Prenatal → general-psychiatric population: no transfer.

Under pure held-out inference (prenatal-trained leakage-controlled acoustic model; no refitting on target), the model did not separate from chance on either corpus: DAIC-WOZ AUC 0.505 [0.417, 0.593], E-DAIC 0.478 [0.398, 0.564]. This held across feature-extraction toolkits: our features, OpenSMILE eGeMAPS (0.483) and OpenSMILE MFCCs (0.546) were all near chance on E-DAIC (Multimedia Appendix 6), so it is not a toolkit artifact.

General-psychiatric → prenatal population: measured transfer differed substantially by outcome instrument.

DAM, trained on ~35,000 individuals and applied without refitting, reached AUC 0.706–0.708 on the two general-psychiatric corpora. Applied to pregnant women it reached 0.510 [0.411, 0.609] against the PHQ-8 and 0.645 [0.541, 0.744] against the mEPDS-9: in the same participants, on the same recordings, differing only in which questionnaire defined the outcome (Table 3, Figure 2). The two estimates differ from one another on a direct paired test, which the design permits because both are computed on the same participants and recordings: paired bootstrap ΔAUC = +0.135 (95% CI +0.019 to +0.248; unadjusted p = 0.021; 10,000 resamples, participants resampled once per iteration and both AUCs recomputed on that draw so their correlation is preserved). The contrast is significant on all three cohort definitions (+0.138, p = 0.011 at N=149; +0.127, p = 0.037 at N=115). This contrast was not pre-specified, is not part of the 16-cell multiplicity family reported above, and the p value is unadjusted; it requires independent replication. The pre-specified 16-cell task × outcome grid and this comparator-model family are treated as separate multiplicity families because they address distinct analytic questions. The mEPDS-9 estimate survives Benjamini–Hochberg adjustment across the 10 comparator-model-on-prenatal cells examined (four outcome definitions plus per-task breakdowns) at adjusted p = 0.005, and is stable across cohort definitions: 0.651 [0.549, 0.746] on all N=149 participants with DAM-processable audio, 0.645 [0.541, 0.744] after applying the same participant-grouping exclusions used for the main analysis (N=136, reported here), and 0.642 [0.528, 0.749] on the N=115 who additionally meet the reading-passage and ZIP-matching requirements of the main analytical sample. The PHQ-8 estimate is 0.510–0.515 in all three.

Table 3:

Transfer matrix (AUC, 95% CI).

Model	Prenatal cohort		DAIC-WOZ	E-DAIC
	vs PHQ-8≥10	vs mEPDS-9≥10
Prenatal, acoustic + PDFMa	0.494 [0.431, 0.560]	0.456 [0.384, 0.525]	—	—
Prenatal, acoustic onlyb	0.452 [0.389, 0.511]	0.474 [0.402, 0.545]	0.505 [0.417, 0.593]	0.478 [0.398, 0.564]
Kintsugi DAMc	0.510 [0.411, 0.609]	0.645 [0.541, 0.744]	0.706 [0.620, 0.785]	0.708 [0.627, 0.789]

Bold marks the three estimates whose confidence interval excludes chance. All cross-domain cells are pure inference, with no refitting, recalibration or threshold selection on the target cohort; the two prenatal cells for the prenatal-trained model are leakage-controlled cross-validated estimates within its own cohort, since a model cannot be applied to its own training cohort without being fitted on it. The two prenatal columns differ only in which self-report instrument defines the outcome; the participants, recordings, and models are identical.

^aPrenatal-trained leakage-controlled model, acoustic features fused with community-health embeddings. Community-health embeddings do not exist for DAIC-WOZ or E-DAIC (no ZIP code), so this row has no external cells.

^bThe same prenatal-trained model using the acoustic block alone (Reading Passage, PHQ-8≥10 training labels). This row is the like-for-like comparison with DAM, which is acoustic-only in every column, and is the only row that can be read across all four columns.

^cDAM applied without refitting; prenatal values are aggregated per participant across tasks. The prenatal DAM set is N=136: participants with audio meeting DAM’s minimum-duration requirement, after the same participant-grouping exclusions applied to the main analysis. It is not nested within the N=316 analytical sample, because DAM requires neither a ZIP-code match nor the reading-passage recording; robustness across three cohort definitions is reported in the text. Abbreviations: PDFM, Population Dynamics Foundation Model; DAM, Kintsugi Depression/Anxiety Model; mEPDS-9, modified nine-item Edinburgh Postnatal Depression Scale; DAIC-WOZ, Distress Analysis Interview Corpus (Wizard-of-Oz); E-DAIC, Extended DAIC.

Figure 2:

Transfer of depression voice biomarkers across cohorts and outcome instruments. Points are AUC; bars are 95% bootstrap confidence intervals; the dashed line marks chance. All cross-domain estimates are pure held-out inference; the prenatal-trained model’s two prenatal conditions are leakage-controlled cross-validated estimates within its own cohort. The prenatal-trained model (blue circles) does not separate from chance in any condition, including on its own cohort. The general-population model (DAM, orange squares) is competent on both general-psychiatric corpora (0.706–0.708), at chance against the PHQ-8 in pregnant women (0.510), and above chance against the modified EPDS-9 in the same women (0.645). The two left-hand conditions share participants, recordings and model, and differ only in the questionnaire defining the outcome; that contrast is the study’s central finding. As in Table 3, the two prenatal conditions for the prenatal-trained model use the acoustic + community-context fusion while its two external conditions use the acoustic block alone, because community-health embeddings do not exist for DAIC-WOZ or E-DAIC; the comparator model is acoustic-only in all four. Confidence intervals are shown instead of a colour-coded matrix because several of these estimates are not distinguishable from one another, and a shaded matrix would imply a precision the data do not support. Series are distinguished by hue, marker shape and direct labels, so the figure remains readable in greyscale.

A general-population speech model therefore showed evidence of transfer into pregnancy when evaluated against the mEPDS-9, but not when evaluated against the PHQ-8.

Exploratory evidence consistent with construct mismatch

The analyses in this paragraph were not pre-specified, are based on correlations of about 0.2 in a subset of 136 participants, and are reported as hypothesis-generating. Across the full family of 17 items examined (nine mEPDS-9, eight PHQ-8), one item survives Benjamini–Hochberg adjustment (crying, r = 0.26, adjusted p = 0.025); the remaining associations are nominal only. The pattern is offered as a candidate explanation requiring replication, not an established mechanism. DAM’s score tracks the part of the mEPDS-9 that the PHQ-8 does not capture: partial r(DAM, mEPDS-9 | PHQ-8) = +0.279 [0.11, 0.43], while partial r(DAM, PHQ-8 | mEPDS-9) = −0.179 [−0.33, −0.01]. Item-level correlations locate that content precisely. DAM tracks affective distress; crying (r = 0.26), sadness (r = 0.21), feeling overwhelmed (r = 0.20), anxiety (r = 0.19), and is flat on the two positively-worded anhedonia items (r ≤ 0.03). On the PHQ-8 every item sits at or near zero except fatigue (r = 0.11), with slightly negative associations for appetite change and anhedonia (both −0.08). This is not an anxiety effect: the EPDS anxiety subscale (r = 0.185) and its depression subscale (r = 0.216) correlate with DAM equally. The PHQ-8 contains substantial somatic and neurovegetative content (sleep, appetite, energy, psychomotor change) and these symptoms are common in pregnancy for reasons unrelated to mood. If that content contributes disproportionately to the PHQ-8 total in this cohort, a model that responds to affective distress would fail against the PHQ-8 here while succeeding against the same instrument in general-psychiatric adults, where those items do track depression.

Transfer also varies with elicitation task.

Within the prenatal cohort and against the same instrument, DAM reached 0.653 [0.550, 0.753] on spontaneous pregnancy narrative, 0.597 [0.486, 0.699] on open-ended opinion speech and 0.544 [0.434, 0.651] on the scripted reading passage; an ordering that follows similarity to its training distribution of spontaneous speech about personal experience.

Sex composition explains none of it.

DAM performed comparably on female and male general-psychiatric speech (0.657 vs 0.708 on DAIC-WOZ; 0.662 vs 0.710 on E-DAIC; Δ ≈ −0.05, p ≈ 0.56), yet dropped to 0.510 against the PHQ-8 in pregnant women. A model that handles women in one setting and fails on women in another is not failing because of the sex of the speakers. In the forward direction, the prenatal model performed better on female than male external participants (0.587 vs 0.461 on DAIC-WOZ; 0.588 vs 0.417 on E-DAIC), and a probe confirms why: out of domain, its score predicts participant sex (AUC 0.62–0.63) better than depression (0.48–0.51). Restricted to women it still did not separate from chance.

How large is the shift the models face?

A classifier trained to distinguish prenatal from external recordings, using the same 20 features the transfer model itself selects, reaches AUC 0.89 (DAIC-WOZ) and 0.88 (E-DAIC). Domain identity is thus far more recoverable from this feature space than depression is, which is why a model fitted in one domain carries so little into another, and why no component of the compound shift can be isolated from these data alone.

Discussion

Principal Findings

This study yields four findings. First, and centrally: whether a depression voice biomarker transfers into a new clinical setting is substantially shaped by the outcome instrument, not by the population alone. A general-population speech model applied without refitting to the same pregnant women, on the same recordings, reached 0.510 against the PHQ-8 and 0.645 against the modified EPDS-9. Nothing about the model, the speakers or the audio differed between those two numbers: only the questionnaire that defined depression. Second, exploratory item-level associations are consistent with a construct mismatch: the model’s score was associated with selected affective-distress items and showed no association with the somatic PHQ-8 items examined, although only one of 17 item-level tests survived multiplicity adjustment. Third, transfer additionally varies with elicitation task, following similarity to the training distribution (pregnancy narrative > open-ended speech > scripted reading). Fourth, our own prenatal-trained model performed at chance both within the cohort, on the pre-registered outcome, and on two independent external corpora; no cell of a 16-cell pre-specified grid survived multiplicity adjustment under either of two modeling paradigms.

The practical consequence is that “does this voice biomarker generalize?” is not a well-posed question. It becomes well-posed only when the population, the elicitation task and the outcome instrument are named together. A tool validated against one instrument should not be assumed to work against another, even in the same people.

Neither Voice nor Community Context Discriminates Individuals

The voice increment over community context is ΔAUC = −0.047 at the pre-registered outcome, and community context itself does not discriminate either: the PDFM arm reaches a median of 0.519 across the 16 cells, a maximum of 0.546, and clears chance in none of them. Community context performs no worse than acoustics, but both are at chance, so neither carries individual-level signal.

The like-for-like comparison sharpens this. Restricted to the acoustic block alone, which is the only basis on which our model and the general-population comparator can be compared directly, the prenatal-trained model reaches 0.452 against the PHQ-8 and 0.474 against the mEPDS-9, while the comparator reaches 0.510 and 0.645 on the same recordings (Table 3). Both operate on the voice signal alone, without community context, transcripts or refitting. The difference between them is therefore not the information available in the audio but what each model has learned to extract from it, which is consistent with the null reported here being a property of this cohort’s features and fitting procedure, and not evidence that perinatal speech carries no recoverable signal at all.

A graph-based foundation model pretrained on a large national survey showed the same absence of individual-level transferability in this cohort (Multimedia Appendix 10). This is an individual-level result and should not be read more broadly. Population-level associations between geospatial context and perinatal depression are well documented, including in work using these same embeddings at the level of aggregated geographies. A representation can track prevalence across places and still fail to rank individuals within them; that is the ordinary distinction between ecological and individual inference, and our result speaks only to the latter.

Implications for Validation Practice

Our results argue for three changes in how voice-based mental-health tools are evaluated. First, validation claims should name the population, the elicitation task and the outcome instrument together; any one alone is insufficient, and our data show the instrument can flip a result from chance to above chance in the same participants. Second, a model deployed outside the setting it was validated in gives no signal that it has left that setting; DAM’s score distribution looks the same whether it is discriminating at 0.71 or at 0.51, which argues for explicit detection of the mismatch, not for trust in the output. In our data the information needed for such detection is present: domain identity is recoverable at AUC 0.88 from the same 20 features the model uses. Third, the instrument should be chosen for what the model can plausibly detect. In pregnancy specifically, somatic-weighted instruments are poor targets for acoustic models because gestation elevates the somatic items independently of mood.

We do not report a deployment scenario for this cohort. With the pre-registered outcome at chance and a non-informative operating point (LR+ = 1.02, LR− = 0.96), no configuration examined here supports screening use.

Cross-Domain Non-Transfer Is Robust to Audio Format

The cross-domain failure is not an audio-format artifact: format-matching Voice-AI audio to DAIC’s native 16 kHz PCM and re-running the comparator model leaves ΔAUC essentially centered on zero, and a symmetric reversal test does not rescue transfer in either direction. Browser-recorded audio in this cohort was, if anything, higher quality than the laboratory reference corpora. Full analyses, including the task-conditional partial-transfer pattern and the pregnancy-specific phonatory substrate, are in Multimedia Appendix 9.

Comparison With Prior Work

The voice-biomarker literature has repeatedly reported above-chance, sometimes high, within-study discrimination for depression, but external validation across clinical settings is rare and controls for data leakage are inconsistently applied.[8–10, 34] Our results add two correctives. First, generalization failure in this literature is routinely attributed to population differences; we show that in the one direction where the two can be separated, the outcome instrument determined whether transfer occurred, with the population held fixed. Reports of population-specific failure should therefore be read with the instrument in view, since studies comparing cohorts almost always change instruments at the same time. Second, our own within-cohort estimates collapse to chance under leakage-controlled evaluation with multiplicity adjustment, consistent with the view that some previously reported performance reflects analytic flexibility and leakage, not transportable signal.

Limitations

Self-report labels at a single timepoint. Both outcomes are self-administered screening instruments, not clinical diagnoses, collected once. This is the study’s most consequential limitation and it is compounded by (2). (2) The mEPDS-9 required item-level rescoring. The instrument as administered used a generic response scale in place of the published EPDS anchors, and the responses required rescoring at the item level before analysis. Because the resulting totals are indistinguishable from valid ones at the score-total level, we recommend item-level verification of any self-report label before modeling. (3) Cell-level instability. Individual cells of the task × instrument grid vary by up to 0.08 across protocol variations, so no single cell should be interpreted; this is why we report the whole grid with multiplicity adjustment and interpret no single best-performing configuration. (4) Post-hoc analyses. The instrument-specificity finding, the item-level mechanism, the task ordering and the sex-stratified analyses were not pre-specified. They are hypothesis-generating and require pre-registered replication. (5) The DAM subsample is a different sample. DAM analyses use the N=136 participants whose recordings meet its minimum-duration requirement and who pass the same participant-grouping exclusions. This set is not nested within the N=316 analytical sample: DAM requires neither ZIP matching nor the reading-passage recording, so the two analyses are not drawn from identical participants. The instrument contrast is internally consistent because both estimates come from the same participants; the comparison against the within-cohort analysis is not exact. (6) Compound shift. Population, task and instrument differ simultaneously between cohorts; we can isolate the instrument only in the direction where two instruments exist on the same participants, and cannot isolate population from task at all. (7) No depression-history data. Onset timing was not collected, so pre-existing and incident antenatal depression cannot be distinguished. (8) Residual demographic differences. Sex is addressed directly, but age could not be matched to the prenatal range within the external corpora without collapsing sample size. (9) Online recruitment selection. Cohort prevalence (53–74%) far exceeds general perinatal populations; PPV/NPV/NNS are recomputed at 12–30% deployment prevalence (Table 1). (10) Self-harm-item omission under the IRB determination. Additional limitations appear in Multimedia Appendix 7.

Scope of Inference

The supported claims are narrow and we state them explicitly. Within this cohort, under leakage-controlled evaluation, neither acoustic features nor community-health context discriminated individuals on the pre-registered outcome, and no cell of the pre-specified grid survived multiplicity adjustment. Across cohorts, a general-population speech model transferred into pregnancy against one instrument and not another, with an item-level mechanism consistent with that pattern. We do not claim that voice biomarkers cannot work in perinatal populations; we claim that this cohort, with these instruments and these tasks, provides no evidence that they do, and that a substantial part of what looks like population-specific failure in this literature may be instrument-specific.

Future Directions

Three directions follow directly. Modeling. Parameter-efficient fine-tuning of foundation speech models on target-domain data is the most direct test of whether the failures we observe are trainable domain gaps; the task-conditional partial transfer we report suggests adaptable representations exist. Unsupervised domain adaptation is motivated by the size of the shift we measure, and our domain probe sets its difficulty. Domain-invariant objectives should target the axes our analysis identifies (population, task and instrument) and the sex-tracking probe suggests speaker-attribute invariance deserves an explicit penalty. Measurement. Confirmatory work should adopt structured clinical interview as the primary outcome, and where self-report is used, should report item-level psychometrics instead of totals alone. In perinatal samples specifically, instruments should be chosen or scored so that pregnancy-confounded somatic content does not dominate the target. Design. Studies should vary population, task and instrument separately rather than together; a cohort carrying two instruments on the same participants, as ours did, is what made the central finding of this paper visible at all.

Conclusions

Whether a depression voice biomarker transfers to a new clinical setting may depend on what the outcome instrument measures, not on the population alone. A general-population speech model applied without refitting to the same pregnant women reached AUC 0.645 against a modified EPDS-9 and 0.510 against the PHQ-8 (identical participants, recordings and model) with exploratory item-level results consistent with the model responding to affective-distress content, not to the somatic PHQ-8 content that gestation itself elevates. Our own prenatal-trained model performed at chance within the cohort on its pre-registered outcome and on two independent external corpora, and no cell of a pre-specified 16-cell grid survived multiplicity adjustment. Validation of voice-based mental-health tools should therefore specify population, elicitation task and outcome instrument together, and should not assume that a tool validated against one instrument works against another in the same people.

Changes in this version.

This version supersedes both v1 (May 2025) and v2 (September 2026), and readers who cited either should treat it as replacing them in full.

Relative to v1.

During peer review we identified a scoring defect in the study’s depression labels: the survey platform presented a single generic response scale in place of the item-specific anchors of the Edinburgh Postnatal Depression Scale, and the scoring applied to it was non-monotone in symptom severity and did not reverse-score the two positively-worded items. All analyses were recomputed under corrected scoring. The correction changed the study’s conclusions. The pre-registered within-cohort outcome is at chance and no cell of the pre-specified analysis grid survives multiplicity adjustment; an inter-instrument “ceiling” argument made in v1 is withdrawn; and the central finding is now that measured cross-domain transfer varied with the outcome instrument rather than with the population alone. The analysis was also rebuilt under a leakage-controlled protocol with participant-grouped cross-validation.

Relative to v2.

Several reported values are corrected: one confidence interval in the transfer matrix, the node-coverage figure in the graph-model appendix, the description of the cohort used to train the forward-transfer model, and an effect size in the bias appendix that had been taken from a simulation rather than measured on the study cohort. Multiplicity is now reported under a single procedure (Benjamini–Hochberg) instead of two. Author order and ORCID identifiers are corrected, and results under the superseded scoring are no longer reported alongside the corrected ones.

Abbreviations

RID

respondent identifier, the participant identifier assigned by the recruitment panel

AUC

area under the receiver operating characteristic curve

BH

Benjamini–Hochberg

CI

confidence interval

CV

cross-validation

DAIC-WOZ

Distress Analysis Interview Corpus – Wizard-of-Oz

DAM

Depression/Anxiety Model (Kintsugi Health)

E-DAIC

Extended Distress Analysis Interview Corpus

EPDS

Edinburgh Postnatal Depression Scale

mEPDS-9

modified nine-item Edinburgh Postnatal Depression Scale (item 10 omitted)

F0

fundamental frequency

HNR

harmonics-to-noise ratio

IPW

inverse probability weighting

IRB

institutional review board

LR

logistic regression

LR+/LR−

positive/negative likelihood ratio

MFCC

mel-frequency cepstral coefficient

NNS

number needed to screen

NPV

negative predictive value

OSF

Open Science Framework

PDFM

Population Dynamics Foundation Model

PHQ-8

eight-item Patient Health Questionnaire

PPV

positive predictive value

TPOT

Tree-based Pipeline Optimization Tool

ZCTA

ZIP Code Tabulation Area

Data Availability

The prenatal voice dataset is not publicly available due to the terms of the IRB determination and participant consent, which did not authorize data sharing. The DAIC-WOZ and E-DAIC datasets are available through the University of Southern California’s Institute for Creative Technologies upon execution of a data use agreement. The Kintsugi DAM model is open-source (Apache 2.0) and available at the Kintsugi Health GitHub repository. PDFM embeddings are publicly available from Google Research. Analysis code, including the instrument scoring and an independent verification script, together with machine-readable result files, are available from the corresponding author on reasonable request. They are not deposited publicly because the result files derived from this cohort contain participant-level outcome values that participant consent does not authorise us to share.