Extended Data Fig. 4. Assessing model predictions of mean changes in behavior.
Given the mean changes in behavior due to silencing (where the mean is taken over the entire session, Extended Data Fig. 1), we wondered to what extent the knockout (KO) network predicted these overall changes versus training a dropout (DO) network, for which a randomly-chosen model LC unit was inactivated during training, and a noKO network, for which no inactivation of any model LC unit was performed during training. a. For each LC neuron type, we computed the average behavior across all held-out courtship frames in the test set (‘real data’). We then computed the mean behavior as predicted by the KO network across the same frames (‘KO network’). Each dot denotes one LC type (color dots) or control session (black dot); colors are indicated at left. Dashed lines are the best linear fit; the correlation ρ is taken across all LC types excluding the control sessions. The KO network has large ρ’s across behavior outputs, indicating good prediction of overall changes. b. Same as in a except for the DO network; for evaluation, no model LC units were inactivated (i.e., dropout was used for regularization36). Correlations were smaller for the DO network than for the KO network (compare ρ’s between a and b). However, for the movement variables, correlations for the DO network were only slightly smaller than those of the KO network. Because the DO network had no access to which LC type was silenced, this suggests that the statistics of visual inputs differed across LC types. For example, imagine if the DO network accurately predicted the behavior of control male flies, including that the male does not sing when the female is far away. Then, if silencing LC10a resulted in the male not being interested in courting the female, the female would be far away in most frames, and the DO network would correctly predict a decrease in song production, even though the DO network has no knowledge LC10a was silenced. Thus, DO training is an appropriate control to ask whether the sensorimotor transformation has changed or if the male has altered his desire to pursue courtship. This also motivates future experiments with virtual reality where the male’s visual statistics can be matched between LC-silenced and control males. c. Same as in a and b except for the noKO network. Correlations were substantially smaller than for the KO and DO networks (compare ρ’s between a and c), indicating that the noKO network could not recover behavior from LC-silenced flies. d. We trained 10 networks each for KO, DO, and noKO training. Each of the 10 networks had different random initializations and different random orderings of training samples. For a fair comparison, the same initialized network and ordering was shared across KO, DO, and noKO training for each of the 10 runs. We then computed the ρ’s of overall mean behaviors for each network and real data. For each of the six behavioral outputs, we found that the KO network predicted the changes in behavior across LC types better than the predicted changes for the DO and noKO networks (red dots above blue and black dots). Each dot denotes one network, and each asterisk denotes that the mean of the KO network is significantly greater than the mean of either the DO or noKO network (p < 0.05, paired, one-tailed permutation test, n = 10). Network run 1 was chosen as the exemplar network in a-c (as well as in Figs. 1–4).