Figure 4. Anomalous posture detection using Selfee (Self-supervised Features Extraction)-produced features.

(A) The calculation process of anomaly scores. Each query frame is compared with every reference frame, and the nearest distance was named IES (the thickness of lines indicates distances). Each query frame is also compared with every query frame, and the nearest distance is called IAS. The final anomaly score of each frame equals IES minus IAS. (B) Anomaly detection results of 15 fly lines with mutations in neurotransmitter genes or with specific neurons silenced ( n = 10,9,10,7,12,15,29,7,16,8,8,6,7,7,9,7, respectively). RA is short for CCHa2-R-RA, and RB is short for CCHa2-R-RB. CCHa2-R-RB^Gal4>Kir2.1, q<0.0001; Trh^Gal4, q=0.0432; one-way ANOVA with Benjamini and Hochberg correction. (C) Examples of mixed tussles and copulation attempts identified in CCHa2-R-RB^Gal4>Kir2.1 flies. (D) The temporal dynamic of anomaly scores during the mixed behavior, centralized at 1.67 s. SEM is indicated with the light color region. (E) Examples of close body contact behaviors identified in Trh^Gal4 flies. (F) The cosine similarity between the center frame of the close body contact behaviors (1.67 s) and their local frames. SEM is indicated with the light color region. (G) The kicking index of Trh^Gal4 flies (n=30) was significantly lower than w¹¹¹⁸ flies (n=27), p=0.0034, Mann-Whitney test. (H) Examples of social aggregation behaviors of Trh^Gal4 flies and w¹¹¹⁸ flies. Forty male flies were transferred into a vertically placed triangle chamber (blue dashed lines), and the photo was taken after 20 min. A fly was indicated by a blue arrow. The lateral sides of the chamber were 16.72 cm. (I) Social distances of Trh^Gal4 flies (n=6) and w¹¹¹⁸ flies (n=6). Trh^Gal4 flies had much closer social distances with each other compared with w¹¹¹⁸ flies; nearest, p=0.0043; median, p=0.002; average, p=0.0087; all Mann-Whitney test. (J) Distributions of the median social distance of Trh^Gal4 flies and w¹¹¹⁸ flies. Distributions were calculated within each replication. Average distributions were indicated with solid lines, and SEMs were indicated with light color regions.

Figure 4—figure supplement 1. Using intra-group score (IAS) to eliminate false-positive results in anomaly detections.

(A) Anomaly scores without IAS of wild-type male-male interactions with the same genotype as references. The blue region indicates the max anomaly score when using IAS; blue dots indicate anomaly scores without IAS that fall into the blue region; red dots indicate false-positive anomaly scores. (B) The cosine similarity between the center frame of wild-type courtship behaviors (1.67 s) and their local frames. SEM is indicated with the light color region. Seven videos containing 70,000 frames were split into non-overlapping 100-frame fragments for calculations. Beyond ±50 frames, the cosine similarity dropped to a much lower level, not affecting anomaly detection.

Figure 4—video 1. The anomaly detection on male-male interactions of RB-Gal4 >Kir2.1 flies.

Download video file ^{(665.3KB, mp4)}

The anomalous behavior was brief tussle behavior mixed with copulation attempts. This behavior was ultra-fast and lasted for less than a quarter second.

Figure 4—video 2. The anomaly detection on male-male interactions of Trh^Gal4 flies.

Download video file ^{(372.5KB, mp4)}

The anomalous behavior was short-range body interactions. These social interactions could last for around half to 1 s on average.