Skip to main content
[Preprint]. 2024 Aug 19:2024.08.14.607690. [Version 1] doi: 10.1101/2024.08.14.607690

Extended Fig. 8: Closed-loop pitch modulation.

Extended Fig. 8:

a. An example trial of closed-loop intonation modulation for speaking a sentence as a question. A separate binary decoder identified the change in intonation and sent a trigger to modulate the pitch feature output of the regular brain-to-voice decoder according to a predefined pitch profile for asking a question (low pitch to high pitch). Neural activity of an example trial with its synthesized voice output is shown along with the intonation decoder output, time of modulation trigger, originally predicted pitch feature and the modulated pitch feature used for voice synthesis. b. An example trial of closed-loop word emphasis where the word “YOU” from “What are YOU doing” was emphasized. To emphasize a word, we applied a predefined pitch profile (high pitch to low pitch) along with a 20% increase in the volume to the predicted speech features. c. An example trial of closed-loop pitch modulation for singing a melody with three pitch levels. The pitch classifier output was used to continuously modulate the predicted pitch feature output from the brain-to-voice decoder.