Figure 1.

Evaluation of speech decoding from neural activity. (A) First, the optimal dynamic time warping alignment of BY2014 acoustic features onto P5 acoustic features is computed. Then this alignment is applied on articulatory trajectories of BY2014 to give an estimate of P5's articulatory trajectories. (B) Mel cepstral coefficients are decoded either 1. directly from neural activity or 2. from articulatory trajectories (EMA) decoded from neural activity. Mel cepstral coefficients directly decoded from neural activity or predicted from decoded articulatory trajectories with fine tuning are evaluated against the participant's original Mel cepstrum. Mel cepstral coefficients predicted from decoded articulatory trajectories without fine tuning are evaluated against BY2014's Mel cepstrum aligned on the participant's Mel cepstrum using DTW.