Figure 4.

Fracturing temporal structure in offline training data helps RNN decoders generalize to the online setting. (a) Diagram of the decoding pipeline. First, neural activity (multiunit threshold crossings) was binned on each electrode (20-ms bins). Then, a trainable day-specific linear input layer transformed the binned activity from a specific day into a common space to account for day-to-day variability in the neural recordings. Next, an RNN converted the day-transformed time series activity into continuous left and right cursor velocities ($v_R,v_L)$, and discrete movement context signals ($e_R,e_L,e_B)$. The movement context signals were then used to gate the cursor velocity outputs. (b) Example open-loop, minimum-jerk cursor velocity (black) and modified saturated velocities (gray/red). Saturated velocity with a prescribed reaction time of 200 ms (red) was used for RNN training since it better approximates the user’s intended behavior. (c) Diagram of data alteration technique that introduces variability in the temporal and behavioral structure of the training data. Data are subdivided into small snippets of variable length; each snippet is then dilated or compressed in time, and the order of the modified snippets is shuffled. (d) Offline decoding performance of RNNs trained with and without data alteration. Sample snippets of x-direction decoded velocities are shown for both cursors during unimanual movement with RNNs trained with and without alteration. Corresponding decoding performance (Pearson correlation coefficient) is summarized via bar plots. Offline performance is better without data alteration. (e) Decoders trained with unaltered data generated pulse-like movements online, as shown in the sample decoded cursor speeds for the right hand (top panel), whereas the RNN trained with altered data (bottom panel) allowed for quicker online corrections. Vertical black bars indicate 95% CIs (bootstrap, n = 10 K). Decoders trained with altered data acquired targets more quickly online.