Automated Behavior Analysis in the Novel Object Recognition Test

Abstract

Accurate identification of animal behaviors is fundamental to behavioral neuroscience, yet manual annotation remains a major bottleneck due to its subjectivity, limited scalability, and high labor cost. This work presents a unified deep learning and heuristic-based framework for automated behavior classification in the Novel Object Recognition Test (NORT), a widely used paradigm for assessing memory and cognitive function in rodents. We fine-tuned a YOLOv11 Pose model to detect nose and tail-base keypoints specific to Long-Evans rats, and used these keypoints to derive spatial heuristics for identifying NORT behaviors. In parallel, we trained YOLOv11 classification models to predict Standing, Object Interaction, and Other behaviors from individual frames, using datasets containing either 2-object or 5-object videos, as well as a combined dataset containing both 2-object and 5-object videos. Our model trained on the combined dataset, enhanced with heuristic-based post-processing to identify the specific objects that the rat is interacting with, achieved the best overall accuracy and generalization across arenas, outperforming heuristic-only and subset-specific 2-object or 5-object models. Detailed confusion matrices and error analyses reveal that most errors occur near behavioral transitions, reflecting the inherent ambiguity of manual labels at frame-level. The proposed framework enables scalable, reproducible, and objective annotation of NORT videos and provides a foundation for future extensions toward temporally-aware behavioral analysis. All the code and model weights, together with sample NORT videos, will be made publicly available to support future rodent behavior recognition research.

1. Introduction

In neuroscience and behavioral research, the task of accurately identifying and quantifying specific behaviors is essential for understanding underlying biological mechanisms relevant to cognitive health [1]. Rat models have been widely used for this task in both basic and translational research, as rats exhibit a rich and diverse repertoire of human-like behaviors that are crucial for studying a range of phenomena, from cognitive processes to disease mechanisms [2]. Traditional approaches to animal behavior identification have heavily relied on manual coding. However, manual coding requires significant human effort, despite the availability of tools that aid the coding process, and is often prone to subjectivity and potential biases in data interpretation, given that animal behaviors can occur in complex and sometimes subtle patterns. Furthermore, given the increasing volume of behavioral data generated by high-throughput experimental paradigms, e.g., automated video tracking systems, manual coding has limited scalability and becomes unsustainable, especially when large datasets are required for robust statistical analysis [3].

The need for efficient, objective, and scalable methods of animal behavior annotation has led to growing interest in automated solutions. Advances in machine learning, computer vision, and deep learning offer the potential to significantly enhance the accuracy, efficiency, and consistency of behavior annotation [4, 5, 6]. By leveraging these technologies, it is possible to accelerate data processing, reduce human error, and provide reproducible results, which are crucial for the credibility and reproducibility of scientific findings.

Various machine learning-based approaches have been developed in recent years to enable automated animal behavior. Some approaches focus on video frame classification using deep neural networks, such as convolutional neural networks (CNNs) [7] to identify more static behaviors (e.g., object interaction or standing behaviors). Other approaches focus on frame sequence classification using recurrent neural networks (RNNs) [8] or temporal convolutional networks (TCNs) [9] that aggregate a sequence of video frames to classify dynamic behaviors (e.g., resting or grooming). Keypoint detection is another powerful technique, in which deep learning models, such as DeepLabCut [10] or DeepPoseKit [11], are trained to identify and track specific body parts (e.g., head or tail) in animals. The detected keypoints can be used as inputs to machine learning models, which infer specific behaviors, e.g., SimBA [12]. Alternatively, heuristics can be applied to refine the identification of behaviors by leveraging keypoint-based rules or patterns derived from biological knowledge, e.g., BehaviorDEPOT [13] or RAINSTORM [14].

Despite the widespread development of deep learning approaches and models for automated rat behavior identification, the variability in animal genetics and environmental factors make it hard to directly reuse deep learning models across studies. For instance, differences in rat genetics, such as color or size, or environmental factors such as cage design and housing conditions, complicate the use of models for keypoint detection or even rat detection and call for customized models fine-tuned using data from the study at hand [6]. Similarly, rat behaviors of interest in diverse behavioral neuroscience studies call for models fine-tuned to those specific studies and behaviors [15].

One particular behavior of interest in this work is the novel object recognition test (NORT). NORT is used to assess learning and memory functions in rodents and focuses on the rodents’ innate tendency to explore novel cues [16]. This strong novelty preference extends not only to objects that were not previously encountered by the rodents, but also to objects that were previously encountered in a different location. NORT generally consists of two phases, the sample and test phases, separated by a delay. In the traditional NORT test, the rodents are allowed to explore two identical objects in the sample phase, one of which is replaced with a novel object in the test phase. If the rodents have intact memory for the previously presented object, they are expected to spend more time exploring the novel object [16]. The time spent with each object is used by neuroscientists in their analyses.

The behavioral neuroscientists generating the videos in this study work with a rat model of fragile X syndrome (FXS), which is the most common single-gene cause of intellectual disability. Specifically, the rat is a Long-Evans rat, a breed characterized by white coat and black hood. The NORT test is used to explore the cognitive deficits of knockout rats with the genetic abnormality causing FXS by assessing their memory in comparison to wildtype rats. Data obtained from these videos will allow the behavioral neuroscientists to compare the amount of time the knockout and wildtype rats spent with the novel objects over the familiar objects.

As for other rodent behaviors, manually tracking the time spent with each object is labor-intensive, with manually annotating interactions with two objects in a 10-minute video taking up to 40 minutes. This task becomes more challenging when the number of objects is increased, as with variations of the traditional NORT. Thus, manual annotation of NORT in a large number of videos is a daunting task and automated approaches for scalable behavioral analysis are greatly needed.

To address this need, rule-based approaches, such as BehaviorDEPOT [13], have been designed and used in conjunction with pose/keypoint estimation models. Furthermore, Ibañez et al. [17] developed a deep learning-based tool, called EXPLORE, which allows users to label data and train a convolution neural network (CNN) based on the labeled data through the means of a graphical user interface. Users can also correct data mislabeled by the CNN model and retrain the model until satisfactory performance is obtained. Experimental results on several diverse videos showed that EXPLORE achieved performance comparable to the performance of human annotators. Furthermore, Ibañez et al. [17] showed that EXPLORE has better performance than expensive commercial tools such as EthoVision XT (Noldus Information Technology, Wageningen, The Netherlands) [18] and ANY-maze (Stoelting Co, IL, USA), which rely on keypoint-based heuristics.

Leveraging prior work on BehaviorDEPOT [13] and EXPLORE [17], we designed and evaluated a generic workflow for identifying NORT-relevant behaviors, including Standing, Object Interaction, Freezing, and Resting behaviors in 2-object or 5-object NORT videos. Figure 1 shows two frames that illustrate the two types of NORT arenas. The Object Interaction behavior is shown in the 2-object frame on the left and the Standing behavior is shown in the 5-object video on the right. In the 2-object videos, the objects are referred to by their position in the frame as Upper or Lower objects as shown in Figure 1 (left), while in the 5-object videos, the objects are referred to as Object 1 to Object 5, as shown in Figure 1 (right).

Figure 1:

NORT arenas. Frames depicting two types of NORT arenas, specifically 2-object (left) and 5-object (right) arenas. The rat is a Long-Evans rat, a breed characterized by white coat and black hood. The sample frame on the left illustrates the Object Interaction behavior, while the one on the right illustrates the Standing behavior. Object naming conventions are also illustrated. Objects are identified by their position in the frame, regardless of what the actual object at that position may be. In the 2-object videos, objects in the upper-right/lower-left corners of the frame are referred to as Upper/Lower, respectively. In the 5-object videos, the object in the upper-left corner is Object 1, the object in the upper-right corner is Object 2, the object in the lower-left corner is Object 3, the object in the lower-right corner is Object 4, and the object in the middle is Object 5.

A high level overview of the proposed workflow for identifying NORT behaviors is shown in Figure 2. As part of this workflow, we used LabelMe [19] to label a set of frames with nose and tail-base keypoints. We also used CVAT [20] to label a set of frames in terms of Object Interaction and Standing behaviors. We first fine-tuned a YOLOv11 Pose model [21] to detect nose and tail-base keypoints specifically for the rats and arenas in our NORT tasks/videos. Similar to BehaviorDEPOT, using nose and tail-base keypoints obtained with our fine-tuned YOLOv11 Pose model, we designed a heuristic-based approach to identify NORT-related behaviors.

Figure 2:

Workflow Diagram. Both 2-object and 5-object NORT video frames are labeled in terms of keypoints (using LabelMe) and NORT behaviors (using CVAT). YOLOv11 Pose and Classification models are trained for keypoint annotation and behavior classification, respectively. Subsequently, the keypoints are used with a heuristic-based approach to identify NORT behaviors, and also to refine the predictions of the YOLOv11 behavior classification model. The results of the YOLOv11 Pose and Classification models are evaluated using standard keypoint annotation and classification metrics, respectively.

Furthermore, inspired by EXPLORE [17], we trained frame-based YOLOv11 classification models [21], independently for the 2-object and 5-object videos, and also for the combined set of 2-object and 5-object videos, to identify the Standing, Object Interaction and Other behaviors. The models trained on 2-object or 5-object videos independently can identify specific object interactions (e.g., Upper or Lower in 2-object videos, and Object1 to Object5 in 5-object videos), while the model trained on the combined set can only identify generic object interactions. To transform generic object interactions into specific object interactions, we leveraged the heuristic-based approach with keypoints from the fine-tuned YOLOv11 Pose model to identify the specific location (marked by a bounding box/circle) of the object that the rat is interacting with. The use of 2-object and 5-object videos together in the training data has the potential to enhance model robustness and generalization to other NORT arenas with a variable number of objects.

We compared three approaches/models - the keypoint-based heuristic approach, the models trained independently on the 2-object and 5-object videos, and the heuristic-enhanced model trained on the combined set of 2-object and 5-object videos. Experimental results showed that the model trained on the combined set, which leveraged keypoint annotations to identify specific object interactions, had the best overall performance. Thus, we used this model to perform a large-scale analysis of approximately 500 videos in terms of group differences in object exploration, novel object and place recognition, rearing behavior, and resting/freezing behavior across multiple days.

To summarize, the main contributions of this work are as follows:

• We introduce a unified deep learning and heuristic-based framework for automated identification of key NORT behaviors (Standing, Object Interaction, and Other) that generalizes across arenas with different object configurations (specifically, 2-object and 5-object NORT settings in our experiments).

• Second, we fine-tuned a YOLOv11 Pose model for precise Long-Evans rat keypoint detection (nose and tail-base) and used these keypoints to derive spatially informed heuristics for NORT behavior detection.

• Third, we developed and evaluated YOLOv11 Classification models trained separately on 2-object and 5-object videos, as well as on the combined set of videos. Experimental results showed that the combined model enhanced with heuristic post-processing achieves the best overall generalization across experimental configurations.

• We performed a large-scale analysis of approximately 500 videos to understand differences in NORT behaviors among different groups.

To accelerate further research on NORT tasks with various numbers of objects, we provide an open-source implementation of the proposed framework, including model weights and annotation tools. Furthermore, to promote reproducibility, we also make available a subset of the NORT videos used in this study, thus addressing a great need for publicly available datasets in automated rat behavior annotation.

2. NORT Dataset

2.1. Video Data Collection

Over 500 trial videos were recorded for both 2-object and 5-object NORT tests, using a top view recording format as shown in Figure 1. Each video in the dataset is approximately 10 minutes long with a resolution of 1920x1080 pixels and a frame rate of 60 frames per second (fps). In the 2-object videos, there is a Long-Evans rat in a rectangular enclosure, while in the 5-object videos, the Long-Evans rat is in a square enclosure. The objects used had different visual features and were made of either metal or glass, to ensure that they were not chewable and did not retain any scent cues. The objects were fixed on metal platforms, which were then fixed to the floor of the cage using Velcro. In the rare case that a rat was able to dislodge an object, the trial was terminated and the video was eliminated from the dataset. The arena and objects were cleaned between each animal using Peroxigard to sanitize and remove scent cues.

The 2-object NORT test was performed over a period of three days, and it was done for two timepoints, when the rats were young (postnatal day 50-65) and also when the rats were old (12 months). The procedure was similar at both timepoints. The sequence of days is illustrated in Figure 3 (Top Row). On the first day, the rats were allowed to explore an empty arena for 10 minutes. On days two and three, two objects were placed in the arena and the animals were allowed to explore the arena and objects for 10 minutes. Specifically, on the second day, two identical objects were introduced in the arena. For the young age group, these objects could either be metal sockets or glass candle holders. For the old age group, these objects could be ramekins or glass dice. One the third (final) day, the arena had an object that the rats saw the previous day and a novel object.

Figure 3:

NORT setup for 2-object and 5-object videos. (Top Row) Frames illustrating the setup across the three days of the 2-object NORT test. On the first day, the rat explores the empty enclosure. Two objects are introduced on the second day. On the third day, a new object replaces one of the original objects. (Bottom Row) Frames illustrating the setup/changes across the four days of the 5-object NORT test. On the first day, the rat explores the empty enclosure. Five objects are introduced on the second day. The two objects on the right swap positions on the third day. Finally, a new object replaces the original object in the upper left corner on the fourth day.

The 5-object NORT test was carried out over a period of four consecutive days as illustrated in Figure 3 (Bottom Row) when the rats were young (postnatal day 60-75). On the first day, the rats were allowed to habituate to an empty arena for 10 minutes. On days two through four, five objects were placed in the arena and the animals were allowed to explore the arena and objects for 10 minutes. On the second day of the test, one object was placed at each of the four corners of the arena, and the fifth object was placed at the center of the arena. On the third day, two objects were swapped places while the other three objects were kept the same. On the final day of the test, the arrangement of objects was the same as that on the third day, but one of the objects was replaced by a novel object. The objects and their positions were not counterbalanced in the 5-object videos, due to the inherent complexity introduced by a larger number of objects.

2.2. NORT Behaviors of Interest

The time each rat spent with each of the objects needs to be tracked to determine whether the rats were able to detect the changes in the objects and their locations. If the rats had intact memory for the objects and their locations on the previous days, they were expected to spend more time with the objects that had swapped places (on the third day in the 5-object videos) and the new object on the final day of either the 2-object or the 5-object videos. To be able to track the time each rat spent with each of the objects, each frame needs to be annotated according to specific object interactions, i.e., Upper or Lower for 2-object videos, and Object 1 to Object 5 for 5-object videos, globally referred to as Object Interaction behaviors. Furthermore, the neuroscientists were also interested in the Standing, Resting, and Freezing behaviors, while any other behaviors are referred to as Other.

A heuristic approach based on nose/tail-base keypoints was originally used to identify the behaviors. For Resting and Freezing behaviors, established heuristics based on a sequence of frames were used and visual examination of the results by domain experts showed satisfactory results. For the Object Interaction and Standing behaviors, frame-level keypoint-based heuristics were designed. However, preliminary examination by domain experts suggested that these behaviors needed further improvement. Thus, the Object Interaction and Standing behaviors versus all the Other behaviors are the focus of the models developed in this work. In what follows, we describe the manually annotated dataset for training and evaluating a YOLOv11 Pose model for keypoint-annotation (nose/tail-base) tailored to the rats in our dataset. We also describe the the manually annotated dataset for training and evaluating the YOLOv11 frame-level behavior classification models.

2.3. Manual Frame-based Keypoint Annotation

Manual keypoint annotation for training and evaluating rat pose models was performed in LabelMe [19] as shown in Supplementary Information (SI) Figure S1. Each annotated frame included nose and tail-base keypoints and a bounding box around the rat, with clear fallback rules for handling partial occlusions (e.g., huddled postures); full examples and edge cases are shown in SI Figure S2. We constructed the keypoint dataset via an iterative bootstrapping loop: starting from a small seed of manually labeled frames, we repeatedly trained a YOLOv11 Pose model, ran inference on unlabeled frames, and added failure cases back into the training set. This procedure yielded 2,250 annotated frames drawn from multiple 2-object and 5-object NORT videos plus one non-NORT video, with a standard train/validation/test split; detailed video composition and per-split statistics are provided in SI Table T1. More details on keypoint annotation are provided in SI Appendix A.

2.4. Manual Frame-level Behavior Annotation

Frame-level behavior annotation for training and evaluating behavior recognition models was performed in CVAT [20] by a domain expert following standardized guidelines (screenshot shown in SI Figure S3 ). To exploit the temporal persistence of behaviors, the annotator marked the onset and offset of the Object Interaction and Standing behaviors, and intermediate frames were automatically filled, while all remaining frames were labeled as Other. Object Interaction was defined by close proximity and orientation of the rat toward an object, and Standing was defined by rearing on hind legs, with mutually exclusive labels enforced on each frame; representative edge cases are illustrated in the SI Figure S4. Overall, approximately 360,000 frames from a diverse set of 2-object and 5-object NORT videos (young and old rats) were annotated, with an approximate class ratio of 1:1:2 for Object Interaction, Standing, and Other, respectively; the exact video selection, per-video counts, and train/validation/test splits are reported in Table 1.

Table 1:

Behavior Annotated Dataset. Videos selected for each of the training (Train), validation (Val), and Test subsets, together with their characteristics and statistics about the number of frames annotated in each video and each category (Object Interaction - Obj. Int., Standing and Other), as well as for each subset and globally for the dataset.

Subset	#Objects	Video	Type/Day	Obj. Int.	Standing	Other	Total
Train	2-object	F1a.02_G3	young, day 3	13,459	10,165	12,364	35,988
	2-object	F8.03_G2.3	old, day 3	6,617	7,950	21,442	36,009
	5-object	F5a.03_B4	young, day 4	9,004	11,634	15,381	36,019
		Total		29,080	29,749	49,187	108,016
Val	2-object	B8.13_G3	young, day 3	6,497	7,405	22,098	36,000
	2-object	B8.02_G2	young, day 2	8,515	15,006	12,479	36,000
	5-object	B6.04_B3	young, day 3	16,428	5,092	14,480	36,000
		Total		31,440	27,503	49,057	108,000
Test	2-object	B06_G3	young, day 3	7,924	14,220	13,881	36,025
	2-object	F8.16_G2.3	old, day 3	2,801	8,413	24,811	36,025
	5-object	B10_B2	young, day 2	10,535	6,870	18,595	36,000
	5-object	F1a.09_B3	young, day 3	8,398	11,914	15,669	35,981
		Total		29,658	41,417	72,956	144,031
All				90,178	98,669	171,200	360,047

SI Table T2 and SI Table T3 show statistics for the specific object interactions for the 2-object and 5-object videos, respectively. More details on behavior annotation are provided in SI Appendix A.2.

To assess the difficulty of the annotation task and establish human error estimates for the NORT behavior classification task, two of the validation videos B8.13_G3 and B6.04_B3 were also annotated by a student, in addition to the domain expert. The student was given the guidelines designed by the domain expert and was allowed to annotate several minutes of a practice video to familiarize themselves with the guidelines and the annotation tool (CVAT). After the practice annotation was complete, the student’s work was checked to ensure that the guidelines were properly followed and that there were no overlapping behaviors. Feedback was provided, as needed. Subsequently, the student completed the annotations of two validation videos, a 2-object video and 5-object video. The agreement between the domain expert and student annotations had a Cohen Kappa coefficient of 0.872 for the 2-object video, and a slightly lower coefficient of 0.863 for the 5-object video.

3. Methods

3.1. Overview of YOLOv11

YOLOv11 advances the YOLO series by providing unified variants for object detection, pose estimation, and image classification within a single framework [21, 22]. Architecturally, it retains a three-stage design–Backbone, Neck, and Task Head. The Backbone utilizes the C3k2 module and Cross-Stage Partial with Spatial Attention (C2PSA) for efficient feature extraction and enhanced spatial focus [21]. The Neck fuses multi-scale features, which are then passed to specialized Task Heads. In this study, we utilize the YOLOv11 Pose Head to jointly regress bounding boxes and 2D keypoints (nose and tail-base), and the YOLOv11 Classification Head, which aggregates global features to output calibrated frame-level behavioral probabilities. Additional architectural details of YOLOv11 are provided in the SI Appendix B.

3.2. YOLOv11 Pose Model for Rat Keypoint Annotation

To detect rat keypoints, we fine-tuned the YOLOv11 Pose model with the medium configuration (yolo11m). Training followed the default hyperparameter settings provided in the Ultralytics framework, which are commonly used as starting points for fine-tuning YOLO pose models. The model was trained for up to 2000 epochs with early stopping (patience of 100 epochs) to prevent overfitting. A batch size of 16 was used and input images were resized to 640 × 640 pixels. Mosaic augmentation was applied during training.

The training objective of the YOLOv11 pose model is defined as a weighted combination of several loss components:

$$L_{\text{total}}={\lambda }_{\text{box}}{L}_{\text{box}}+{\lambda }_{\text{pose}}{L}_{\text{pose}}+{\lambda }_{\text{obj}}{L}_{\text{obj}}+{\lambda }_{\text{cls}}{L}_{\text{cls}}+{\lambda }_{\text{dfl}}{L}_{\text{DFL}}$$ (1)

where L_box (λ_box = 7.5) denotes the bounding box regression loss based on the Intersection-over-Union metric, L_pose (λ_pose = 12.0) corresponds to the keypoint regression loss for predicting nose and tail-base coordinates, L_obj (λ_obj = 1.0) represents the objectness loss, L_cls (λ_cls = 0.5) is the classification loss computed using binary cross-entropy, and L_DFL (λ_dfl = 1.5) denotes the Distribution Focal Loss used to improve bounding box localization accuracy. The coefficients λ_box, λ_pose, λ_obj, λ_cls, and λ_dfl control the relative importance of each component during training.

After training, a small number of frames contained missing keypoint predictions, typically due to occlusions or extreme postures, as those shown in SI Figure S2. To address this issue, missing nose or tail-base coordinates were interpolated using the nearest temporally adjacent frames with valid predictions. This approach preserves temporal consistency while minimizing the impact of short detection gaps.

3.3. Keypoint-based Heuristics for Behavior Identification

Keypoint-based heuristics were designed with input from domain experts to identify behaviors of interest in NORT videos. The behavioral categories considered in this work include Standing, Object Interaction, and Other behaviors. Standing behavior was identified using two complementary rules. First, wall-supported standing was detected when the nose position extended outside the annotated arena boundary. Second, unsupported standing in the center of the arena was detected using body compression, measured as the Euclidean distance between the nose and tail-base keypoints. Frames in which this distance fell below a threshold relative to the average body length were classified as Standing. Object Interaction was determined using the nose keypoint position relative to manually annotated object regions. If the nose position fell within the bounding box of an object, the frame was classified as Object Interaction and the corresponding object identity was recorded.

3.4. YOLOv11 Models for Frame-level Behavior Classification

In addition to the heuristic-based approach, we trained YOLOv11 classification models to predict frame-level behaviors. Two training strategies were considered. First, models were trained separately on 2-object and 5-object videos to predict specific object interactions (e.g., Upper or Lower in 2-object videos, and Object1 to Object5 in 5-object videos), along with Standing and Other behaviors. One main advantage of the specific-object models is that they require a simpler inference pipeline (without the need for keypoints and postprocessing to find the object’s identity). In addition, direct supervision on object identity can potentially yield higher precision. However, one disadvantage of this approach is that it can lead to data fragmentation and class imbalance, reducing effective sample size per class. At the same time, the specific-object approach has poor scalability, as any variations of the arena with the number or arrangement of the objects requires re-labeling and retraining the model, and thus has reduced robustness and generalization to unseen arenas or objects due to overfitting to setup-specific cues (materials, shapes, lighting).

Second, a generic model was trained on the combined dataset of 2-object and 5-object videos to predict three classes: Object Interaction, Standing, and Other. In this case, the identity of the interacted object was determined through a post-processing step using keypoint geometry. For this post-processing step, the YOLOv11 Pose model was used to obtain nose coordinates for each frame. Object midpoints were computed from the annotated object bounding boxes, and the interacting object was assigned as the object whose midpoint was closest to the nose position.

In this approach, the model is trained on a pooled, larger, and more diverse dataset, improving representation learning, reducing variance and mitigating the risk of overfitting [23]. The visual characteristics of a rat interacting with an object—such as the proximity of its nose to the object’s surface, its posture, and head orientation—are fundamentally similar regardless of whether there are two or five objects in the arena. Thus, this approach is naturally scalable to any number of objects and arrangements as the object identity is derived geometrically (e.g., nearest object to the nose keypoints). In effect, this ensures better transfer to new arenas and object layouts because the model learns object-agnostic interaction patterns, and also simpler model maintenance as one model serves multiple experimental configurations. However, these advantages come at the cost of a post-hoc mapping from keypoints to object boxes, introducing an additional source of error (i.e., error propagation from pose to final label), and also a slightly more complex inference pipeline and evaluation protocol.

All classification models leveraged pre-trained YOLOv11 backbones and were fine-tuned on the NORT dataset using the AdamW optimizer with a batch size of 16. Models were trained for up to 50 epochs with early stopping (patience of 7 epochs). Input images were resized to 224 × 224 pixels and mixed-precision training was used to improve computational efficiency.

We evaluated multiple YOLOv11 model sizes (nano, small, medium, large, and extra-large) to analyze the trade-off between model capacity and generalization performance. All models were implemented using the Ultralytics YOLOv11 framework. To facilitate reproducibility, the source code, trained models, and a subset of annotated NORT videos are publicly released with this publication.

3.5. Evaluation Metrics

3.5.1. Metrics For Keypoint Annotation

We evaluate the YOLOv11 Pose model using standard performance metrics that capture mean Average Precision (mAP) at different IoU (Intersection-over-Union) thresholds, specifically mAP@0.50 and mAP@[0.50:0.95] (i.e., average over all thresholds from 0.50 to 0.95 with a step of 0.05).

3.5.2. Metrics For Behavior Classification

We evaluate the YOLOv11 classification models using standard performance metrics that quantify classification accuracy and robustness across behavioral categories. Specifically, we report precision, recall, and F1-score (which balances precision and recall) for each behavior class, along with their macro-averaged values to provide an overall measure of performance independent of class imbalance. In addition, confusion matrices are used to visualize class-wise prediction patterns and identify systematic confusions between similar behaviors. Together, these metrics provide a comprehensive assessment of model performance, highlighting both per-class discrimination ability and overall generalization across behavioral contexts.

4. Experimental Results and Discussion

4.1. YOLOv11 Keypoint Annotation Model Performance

The results of the model keypoint annotation model are summarized in what follows. More information is provided in SI Appendix C. The model achieved an mAP@0.50 value of 87.4% and an mAP@[0.50:0.95] value of 87.3%. While the model demonstrated strong overall performance in terms of mAP, some instances of missed keypoints were observed. These omissions occurred either in isolation, where a single keypoint was not detected, or concurrently, with both nose and tail-base keypoints missing within the same frame. Upon reviewing frames with missing annotations, it was noted that these errors generally corresponded to instances in which the keypoints were visually obscured or when the rat appeared slightly out of focus, thereby reducing visibility. Across all four videos, consisting of over 144, 000 frames, only 280 frames had any missing annotations, which represents a very small percentage of frames out of the total number of frames in the test videos used (see SI Table T4 ). Furthermore, across the four behavior test videos, the longest continuous sequence with missing nose and/or tail-base annotations was 43 frames (≈0.7 seconds at 60 fps), and only a handful of gaps exceeded 10 frames; the full gap-length distribution is provided in SI Figure S6.

To mitigate the effect of missing data on behavioral metrics, an interpolation strategy was employed for both nose and tail-base (details are presented in SI Appendix C.3). After interpolation, nose keypoints were highly accurate for the behavior test videos: in a representative 2-object video, 90% of interpolated nose positions lay within 20 mm of the ground-truth location (maximum error 133 mm), and in a 5-object video 90% were within 14 mm (maximum error 80 mm); full error distributions are shown in Supplementary Figure S7.

4.2. Behavior Classification

In this section, we discuss the results of the heuristic-based approach by comparison with the YOLOv11 classification models trained from the combined 2-object and 5-object dataset and also individually from the 2-object and 5-object video datasets, respectively. We evaluated all approaches in terms of their ability to predict generic Object Interaction, Standing, and Other behaviors, as well as their ability to accurately predict specific object interactions. Finally, we performed error analysis to understand the limitations of the models.

4.2.1. Hyperparameter Optimization

We selected all heuristic and model hyperparameters based on validation performance. For the keypoint-based heuristics, we tuned the arena boundary and body-compression threshold for Standing, as well as the tight versus loose object regions for Object Interaction. The final configuration used a loose arena boundary with a 75% body-compression threshold for Standing and loose object regions for Object Interaction, as together they achieved the best balance between precision and recall across validation videos. For the YOLOv11 behavior classifiers, we compared model sizes (nano, small, medium, large, xlarge) trained on combined and separate 2-object and 5-object datasets. The YOLOv11-large model performed best on the combined dataset, whereas YOLOv11-medium models were optimal for the separate 2-object and 5-object datasets, and these configurations were therefore used in all subsequent experiments. More details on hyper-parameter optimization are provided in SI Appendix D.

The large-scale NORT analysis on ≈ 500 videos was done using the combined YOLOv11-large model, which had overall best performance in our study on the validation data.

4.2.2. Human Agreement on Behavior Classification

To estimate the performance of our best model by comparison with human agreement (specifically, student versus domain expert), in SI Appendix E, SI Table T11 compares student and model results on a 2-object validation video, while SI Table T12 compares the student and model results on a 5-object validation video. Furthermore, SI Figure S11 shows confusion student/model matrices for these two validation videos. Together, these suggest that the our best model has performance comparable to the human/student performance.

4.2.3. Performance Results for Three-class Predictions

Table 2 compares three approaches for predicting Standing, Object Interaction, and Other behaviors in NORT videos of rats. The combined model, trained jointly on both 2-object and 5-object datasets, achieved the highest overall performance across nearly all metrics, attaining the best Macro-avg precision, recall, and F1-score for both 2-object and 5-object video groups. This result suggests that exposure to a more diverse set of object configurations enabled the model to learn a generalized representation of object-interaction behavior, improving balance between precision and recall across behavioral classes. In contrast, the 2-object and 5-object models, trained specifically on their respective subsets, often achieved slightly higher recall for Object Interactions (e.g., 0.988 and 0.955 for 2-object and 5-object groups, respectively), but at the cost of lower precision and overall F1. The heuristic baseline maintained competitive recall values in some videos but consistently underperformed in F1, reflecting a tendency to over-predict interactions. Overall, the results indicate that the combined model provides the most robust and transferable behavior predictions across varying experimental setups, whereas the specialized models may be advantageous when maximizing recall for interaction detection is a specific experimental priority.

Table 2:

Performance results on test videos. Three approaches are compared: the heuristic-based approach, a YOLOv11-large model trained on the combined 2-object and 5-object dataset, and YOLOv11-medium-IR+TS models trained independently on the 2-object and 5-object videos, respectively. Models are evaluated in terms of their ability to predict Standing, generic Object Interactions and Other behaviors. For each video group and also for the 2-object and 5-object average groups, best performance for each metric in that group is highlighted in bold font. For both groups, the best results overall, highlighted in blue are obtained with the combined model.

Video (#obj.)	Model	Standing			Object Int.			Other			Macro-avg
		Pr	Re	F1	Pr	Re	F1	Pr	Re	F1	Pr	Re	F1
B06_G3 (2)	heuristic	0.872	0.838	0.854	0.839	0.649	0.732	0.727	0.850	0.784	0.813	0.779	0.790
	2-object	0.853	0.979	0.911	0.904	0.857	0.880	0.892	0.783	0.834	0.883	0.873	0.875
	combined	0.934	0.957	0.946	0.937	0.958	0.947	0.931	0.896	0.913	0.934	0.937	0.935
F8.16_G2.3 (2)	heuristic	0.793	0.922	0.853	0.682	0.994	0.809	0.977	0.873	0.922	0.817	0.930	0.861
	2-object	0.714	0.990	0.830	0.561	0.864	0.680	0.996	0.804	0.890	0.757	0.886	0.800
	combined	0.890	0.970	0.928	0.793	0.875	0.832	0.983	0.942	0.962	0.889	0.929	0.907
Avg. (2)	heuristic	0.833	0.880	0.854	0.761	0.822	0.771	0.852	0.862	0.853	0.815	0.855	0.826
	2-object	0.784	0.985	0.871	0.733	0.861	0.780	0.944	0.794	0.862	0.820	0.880	0.838
	combined	0.912	0.964	0.937	0.865	0.917	0.890	0.957	0.919	0.938	0.911	0.933	0.922
B10_B2 (5)	heuristic	0.784	0.788	0.786	0.888	0.944	0.915	0.923	0.888	0.905	0.865	0.873	0.869
	5-object	0.738	0.899	0.811	0.855	0.921	0.887	0.955	0.836	0.892	0.849	0.885	0.863
	combined	0.914	0.818	0.864	0.933	0.835	0.881	0.863	0.948	0.903	0.903	0.867	0.883
F1a.09_B3 (5)	heuristic	0.913	0.452	0.604	0.569	0.944	0.710	0.826	0.852	0.839	0.769	0.749	0.718
	5-object	0.886	0.808	0.845	0.592	0.994	0.742	0.945	0.665	0.781	0.808	0.822	0.789
	combined	0.985	0.849	0.912	0.781	0.870	0.835	0.861	0.883	0.872	0.876	0.867	0.873
Avg. (5)	heuristic	0.849	0.620	0.695	0.729	0.944	0.813	0.875	0.870	0.872	0.818	0.811	0.793
	5-object	0.812	0.854	0.828	0.724	0.958	0.815	0.950	0.751	0.837	0.829	0.854	0.826
	combined	0.950	0.834	0.888	0.857	0.853	0.858	0.862	0.916	0.916	0.890	0.868	0.887

By observing both 2-object and 5-object configurations during training, the model captures common visual patterns associated with interaction and posture that are independent of the specific arena layout. This improved generalization is particularly important for behavioral neuroscience experiments, where variations in object configurations and arena setups are common.

4.2.4. Performance Results for Specific Object Interactions

Table 3 compares the heuristic approach, the YOLOv11 model trained only on 2-object videos, and the YOLOv11-large model trained on the combined 2-object and 5-object dataset in terms of specific object interactions in 2-object test videos, i.e., Upper Object and Lower Object. Averaged over the two 2-object videos, the combined model yielded the strongest F1 for both Lower Object (0.885) and Upper Object (0.888), with balanced precision/recall values (Pr≈0.86-0.87, Re≈0.90-0.92). The 2-object-only model attained good recall (e.g., Upper Object Re=0.892) but substantially lower precision (e.g., Lower Object Pr=0.684). The heuristic was competitive on recall (especially for Upper Object in F8.16_G2.3, Re=1.00) but typically traded off precision, leading to lower average F1 as compared to both the combined model and the 2-object model.

Table 3:

Performance results for specific object interactions in the 2-object test videos. Three approaches are compared: the heuristic-based approach, a YOLOv11-large model trained on the combined 2-object and 5-object dataset, and a model trained independently on the 2-object videos. Models are evaluated in terms of their ability to predict specific object interactions, i.e., Lower Object and Upper Object behaviors. For each video group and for the 2-object average group, best performance for each metric is highlighted in bold; ties are bolded in all tied entries.

Video (#obj.)	Model	Lower Object			Upper Object
		Pr	Re	F1	Pr	Re	F1
B06_G3 (2)	heuristic	0.770	0.903	0.831	0.899	0.537	0.673
	2-object	0.814	0.999	0.897	0.963	0.794	0.871
	combined	0.927	0.944	0.935	0.942	0.964	0.946
F8.16_G2.3 (2)	heuristic	0.716	0.989	0.831	0.644	1.000	0.784
	2-object	0.553	0.764	0.642	0.569	0.990	0.723
	combined	0.803	0.868	0.834	0.781	0.884	0.829
Average (2)	heuristic	0.743	0.946	0.831	0.772	0.769	0.729
	2-object	0.684	0.882	0.770	0.766	0.892	0.797
	combined	0.865	0.906	0.885	0.862	0.924	0.888

Table 4 shows performance for specific objects in the 5-object test videos. On average across the two 5-object videos, the combined model again provided the most consistent F1 - better than both the heuristic and the 5-object-only model for Objects 2-4 (e.g., Obj3 F1=0.919, Obj4 F1=0.903) and comparable with the results of the 5-object-only model on Object 1 (with F1=0.765 for the 5-object-only model and F1=0.760 for the combined model). Notably, the heuristic was strongest on Object 5 (Avg F1=0.952 vs. 0.923 for the 5-object model and 0.879 for the combined model), and also better for several objects in B10_B2 (e.g., Objects 2, 4 and 5). These exceptions are consistent with scene-specific geometry: when the rat sits directly on or partially occludes an object, the rule-based can be more robust than appearance cues, whereas the YOLOv11 models occasionally revert to Other. The 5-object-only model showed a similar pattern to the 2-object-only model: recall remained high (often >0.95) but precision lagged, producing lower F1.

Table 4:

Performance results for specific object interactions in the 5-object test videos. Three approaches are compared: the heuristic-based approach, a YOLOv11-large model trained on the combined 2-object and 5-object dataset, and a YOLOv11-medium-IR+TS model trained on the 5-object videos. Best (maximum) values for each metric within a video group (and within the Average group) are bolded.

Video (#obj.)	Model	Object 1			Object 2			Object 3			Object 4			Object 5
		Pr	Re	F1	Pr	Re	F1	Pr	Re	F1	Pr	Re	F1	Pr	Re	F1
B10_B2 (5)	heuristic	0.899	0.779	0.835	0.913	0.905	0.909	0.832	0.998	0.907	0.864	0.953	0.906	0.951	0.967	0.959
	5-object	0.770	0.989	0.866	0.854	0.831	0.843	0.817	0.980	0.891	0.817	0.982	0.892	0.983	0.870	0.923
	combined	0.876	0.872	0.874	0.927	0.770	0.841	0.930	0.969	0.949	0.923	0.877	0.899	0.978	0.708	0.821
F1a.09_B3 (5)	heuristic	0.421	0.860	0.565	0.399	0.914	0.555	0.693	0.991	0.816	0.642	0.959	0.769	0.900	0.991	0.944
	5-object	0.496	0.999	0.663	0.414	0.995	0.585	0.684	0.997	0.812	0.706	0.979	0.820	0.856	0.999	0.922
	combined	0.515	0.867	0.646	0.815	0.853	0.834	0.832	0.955	0.889	0.890	0.922	0.906	0.983	0.894	0.937
Average (5)	heuristic	0.660	0.820	0.700	0.656	0.910	0.732	0.763	0.995	0.862	0.753	0.956	0.838	0.926	0.979	0.952
	5-object	0.633	0.994	0.765	0.634	0.913	0.714	0.751	0.989	0.852	0.762	0.981	0.856	0.920	0.935	0.923
	combined	0.696	0.870	0.760	0.871	0.812	0.838	0.881	0.962	0.919	0.907	0.900	0.903	0.981	0.801	0.879

Across both settings, the combined YOLOv11 model offered the best overall trade-off between precision and recall and the most stable F1 across objects and videos, while single-subset (2-object and 5-object videos, respectively) models biased toward recall (over-prediction near interaction onsets/offsets). The heuristic excels in specific layouts (notably Object 5), where a fixed geometric tolerance may trump appearance features (e.g., cases where the object is occluded by the rat).

4.2.5. Error Analysis for Behavior Classification

To gain deeper insight into YOLOv11-large model performance and error patterns, we analyzed normalized confusion matrices for each test video, shown in Figure 4 (similar confusion matrices for the single-subset YOLOv11-medium-post models and for the heuristic approach are shown in SI Figures S12 and S13, respectively). The matrices reveal strong diagonal dominance across all behaviors, confirming that the YOLOv11-large model accurately capture the main behavioral categories. However, consistent off-diagonal structure indicates systematic sources of confusion, with both Standing and Object Interaction behaviors being misclassified as Other most frequently, similar to what is observed when comparing human/student annotations on two validation videos in SI Figure S11. This suggests errors potentially due to sensitivity to the distance threshold used to identify object interactions, as human annotations may have some tolerance to slightly smaller or larger thresholds, depending on the actual pose, and similarly sensitivity to the criterion for standing as it can imply more or less upright position.

Figure 4:

Normalized confusion matrices illustrating the classification performance of the YOLOv11-large model on the NORT behavior prediction task. Each subpanel corresponds to one test video and shows per-class agreement between model predictions and expert annotations. High diagonal values indicate strong within-class accuracy, while off-diagonal values reveal systematic confusions. Most misclassifications occur near behavioral transitions (e.g., Object Interaction ↔ None or Standing ↔ Other), where frame-level ambiguity and annotation inconsistencies are greatest.

It is also worth noting that there are no misclassifications where one specific object interaction is confused with the other. That is because the specific object the rat is interacting with is identified as the closest object/midpoint to the rat’s nose. While the nose annotations may not always be very precise, they are unambiguously identifying the object given the arena’s geometry.

To better understand the sources of disagreement between the YOLOv11-large model and the human annotations, we conducted a detailed error analysis focusing on disagreements between Standing↔Other and Object Interaction↔ Other behaviors. We observed that many discrepancies occur near behavioral transition points (i.e., when the rat transitions from one behavior to another), where subtle contextual cues are required to identify the precise frame in which the behavior changes. These transitions are often ambiguous at the frame level and depend on viewing short sequences of frames rather than isolated images. As a result, even expert annotations occasionally exhibit perceived inconsistencies in their labeling, which causes the model to make inconsistent predictions. Some examples for the Other ↔ Standing disagreements are shown in Figure 5 (Top Rows), while examples for the Object Interaction ↔ Other are shown in Figure 5 (Bottom Rows).

Figure 5:

Error Analysis. (Top Rows) Qualitative examples for Standing vs. Other disagreements. (Bottom Rows) Qualitative examples for Object Interaction vs. Other disagreements. Each subpanel shows a single frame with the ground-truth (GT) and model prediction (Pred), along with the frame index. All frames in a row are from the same video. Some frames illustrates typical errors that may arise from frame-level ambiguity (behavior onset/offset judged over short sequences) or subtle posture cues. The consecutive frames directly illustrate the annotation challenge at the boundary between two behaviors.

4.2.6. YOLOv8 Baseline

To provide a baseline comparison with another popular YOLO model, we evaluated YOLOv8 classification models in SI Appendix G. We first showed the number of parameters for different variants of the YOLOv8 and YOLOv11 models in SI Table T13. The results of the YOLOv8 models on the test videos are reported in SI Tables T6, T17 and T18. Overall, YOLOv8-medium trained on the combined set of videos has competitive average performance. However, our best YOLOv11-large model gives consistently better results in terms of macro-averaged F1-scores on the test videos. As can be seen from SI Table T13, the YOLOv11-large model is slightly smaller than the YOLOv8-medium model (25.3 M versus 25.9 M parameters), suggesting that the improved YOLOv11 performance stems from the architectural refinements rather than increased model capacity.

4.3. Global NORT Video Analysis

We used the model to annotate all 500+ videos in our collection. The output from the model was used to explore group differences in behavior during the novel object recognition task in rats. Data from the 5-object NORT videos are reported here. A Bayesian model was used to analyze the pattern of object exploration, rearing, and resting/freezing behaviors across the three days of testing, where animals were exposed to objects. Data from the habituation day was not included in the analysis. The model included the genotype-sex group of the animals, day of testing, and the interaction of these two variables as predictors, and the intercept and slope of day as random effects. The model converged well without any divergent transitions, sufficiently large effective sample sizes, and Rhat values of 1.01 or less. Furthermore, visual check of the Markov Chain Monte Carlo (MCMC) sampling also showed good convergence, with overlapping chains in the caterpillar plots. This model revealed that heterozygous knockout animals spent considerably more time rearing across all three days and the overall object exploration time was not considerably different across groups (Figure 6).

Figure 6:

Behavior of rats on days 3 and 4. As indicated by the outlined slices in the figure, rearing behavior was greater among heterozygous females throughout the test (estimate = 0.27, estimated error = 0.10, 95% credible interval = 0.08 – 0.45). Total exploration time was not considerably different among all genotype-sex groups. KO female = homozygous knockout female, KO male = hemizygous knockout male, WT = wildtype.

On the third day of testing, two objects from day 2 swapped places. These objects were in positions 2 and 4. A difference score was used to ascertain whether the rats spent more time with these positions of interest on the third day than the second day, indicating that they noticed the difference in the positions of the objects. Results of linear regression models showed that while none of the groups spent more time in position 2 on day 3 compared to day 2, homozygous females (b=18.787, SE=7.882, t=2.38, p=.021), wildtype females (b=19.847, SE=7.882, t=2.52, p=.015), and wildtype males (b=18.377, SE=7.676, t=2.39, p=.021) spent more time in position 4 on day 3, compared to day 2 (Figure 7). This data suggests that wild type rats and homozygous females noticed a change in the environment had occurred between day 2 and day 3.

Figure 7:

Difference in time spent in position 4 on day 3 and day 2. Homozygous females (b=18.787, SE=7.882, t=2.38, p=.021), wildtype females (b=19.847, SE=7.882, t=2.52, p=.015), and wildtype males (b=18.377, SE=7.676, t=2.39, p=.021) spent more time in position 4 on day 3 than on day 2, indicating that they noticed the change in environment. KO female = homozygous knockout female, KO male = hemizygous knockout male, WT = wildtype.

On the final day of testing, the object in position 1 was replaced by a novel object. A linear regression model was run using a different score to determine whether the rats spent more time in position 1 on the final day compared to the second day of testing. Tukey-adjusted pairwise comparisons showed that wildtype male rats spent significantly more time in position 1 on the final day compared to the heterozygous knockouts (p=.001), hemizygous males (p=.033), and wildtype females (p=.002), but not homozygous females. Homozygous females also spent more time in position 1 on the final day. However, it did not survive Tukey-adjusted pairwise comparisons (Figure 8).

Figure 8:

Difference in time spent in position 1 on day 4 and day 2. Wildtype males (b=23.182, SE=5.567, t=4.16, p<.001) and homozygous females (b=13.260, SE=5.716, t=2.32, p=.025) spent more time in position 1 on day 4 than on day 2, indicating that they noticed the novel object. Pairwise comparisons show that the wildtype males spent significantly more time in position 1 on the final day compared to heterozygous knockouts (p=.001), hemizygous males (p=.033), and wildtype females (p=.002). KO female = homozygous knockout female, KO male = hemizygous knockout male, WT = wildtype, * = significant Tukey-adjusted pairwise comparison, # = significant main effect.

In addition to tracking time spent with objects, the algorithms allowed tracking of freezing/ resting, and rearing behavior. Heterozygous females spent significantly more time rearing than other rats. While wildtype males spent less time rearing than the other groups across all three days, the difference did not reach statistical significance. Homozygous females and hemizygous males had similar rearing time across all three days of testing. Heterozygous females spent less time freezing and resting than the other four groups. However, this difference was not statistically significant.

Overall, the results from this experiment show that heterozygous females spent more time rearing. Wildtype male rats, homozygous and wildtype female rats were able to recognize the changes made to the position of objects on day 3. However, when the novel object was introduced on day 4 only wildtype males and homozygous females recognized this change. Prior studies which included only male wildtype and FMR1 knockout rodents found impairment in object recognition and object location memory in the knockouts [24, 25]. The current results replicate those findings, where hemizygous males failed to recognize that two objects switched places on day 3, and that a novel object was introduced on day 4. Sex differences in object recognition and object location memory among wildtype rats is inconsistent in literature, with most studies reporting no sex differences, while some studies report better object recognition memory in females and better object location memory in males [26]. Here, wildtype females were not completely impaired as they noticed the switched objects and homozygous females were not impaired at all. It is possible that heterozygous females were impaired due to their fixation on rearing behavior. A future study should be carried out over more days with more changes in the arena to better capture differences in object recognition memory and object location memory.

5. Conclusions, Limitations and Future Work

In this study, we developed and evaluated a YOLOv11-based framework for automated classification of NORT behaviors. Our best model, which was trained on a combined set of 2-object and 5-object videos, can effectively distinguish between Standing, Object Interaction, and Other behaviors, achieving high precision and recall across both 2-object and 5-object configurations. By identifying generic object interactions, and integrating a heuristic-based approach to convert them into specific object interactions, our framework achieved both high accuracy (and also limited interpretability in terms of specific object interactions), supporting scalable behavioral annotation in neuroscience experiments. This work establishes a foundation for automated, fine-grained behavior annotation and opens the path toward temporally aware and multimodal extensions that can further improve reliability and reduce manual labeling effort.

While the proposed YOLOv11-based framework achieved strong performance in classifying rat NORT behaviors, several limitations remain. First, the model operates primarily on frame-level predictions, which makes it sensitive to brief fluctuations at behavioral transition boundaries. As shown in our error analysis, many discrepancies between model predictions and expert annotations occur near transitions between Object Interaction, Standing, and Other behaviors, where temporal context is essential for accurate interpretation. Training sequence-based models (e.g., temporal convolutional networks, or transformer-based video architectures) could mitigate these issues.

Furthermore, while we included both 2-object and 5-object videos in our dataset, the training videos are still limited in diversity and scale. Overall, the dataset includes a relatively small number of videos and object configurations, which may constrain generalization to other types of arenas, lighting conditions, animal types or animal postures. Additional data collection across laboratories, strains, and object types would enable stronger cross-domain robustness and reduce overfitting to experiment-specific conditions.

A significant challenge in the current landscape of automated behavioral neuroscience is the scarcity of standardized, publicly available NORT datasets. This is particularly true for complex experimental paradigms, such as those involving 5-object spatial arrangements, which currently hinders direct baseline comparisons and the broad validation of existing computer vision methods across different laboratories. To address this critical gap and promote reproducibility, a primary contribution of our work is the public release of a subset of our annotated NORT video dataset, alongside our model weights and annotation tools. By open-sourcing these resources, we aim to establish a robust benchmark that facilitates future comparative studies and accelerates advancements in automated rodent behavior recognition.

Finally, the current system focuses on a limited set of discrete behaviors. Future work will extend this framework toward continuous behavior tracking and higher-level behavioral state inference, combining detection with trajectory, velocity, and pose dynamics.

Biographies

Emily is a graduate student in the Department of Computer Science pursuing her Ph.D. Her work is primarily focused on machine learning related to image and video annotation. Currently, her main datasets are rodent trial videos with the goal of annotating rodent behaviors.

Bhavana Sivayokan is pursuing a Ph.D in behavioral neuroscience at the Department of Psychological Sciences, Kansas State University. Broadly, her interests lie in neurodevelopmental disorders. Her current work focuses on understanding the neurobiology of restricted repetitive behavior.

Aliva Bakshi is a Ph.D. candidate in the Department of Computer Science at Kansas State University. Her research focuses on machine learning and computer vision, particularly object detection, counting, and deep learning for large-scale image datasets. Her work explores scalable AI solutions for applications in agriculture, veterinary medicine, and other real-world data analysis.

Sanaz Gheibuni is a Ph.D. student in the Computer Science Department at Kansas State University. Her research focuses on computer vision, with a particular emphasis on action recognition and action classification. She is especially interested in developing machine learning and deep learning methods that can automatically analyze and understand complex behaviors from video data.

Doina Caragea received her Ph.D. in Computer Science from Iowa State University. She is currently a Professor in the Department of Computer Science at Kansas State University (K-State). Her research interests are in artificial intelligence and machine learning, with applications in natural language processing, computer vision, and other areas.

Bethany Plakke received her Ph.D. in Behavioral Neuroscience Psychology from the University of Iowa and was a postdoctoral scholar at the University of Rochester Medical School. She is currently an Associate Professor in Psychological Sciences at Kansas State University. Her lab uses Fmr1 knockout animals to examine hyperexcitability of the brain and behavioral phenotypes to better examine mechanisms that underlie cognitive deficits and repetitive behavior in Fragile X syndrome.

Dave Turner earned his PhD in experimental Physics from Iowa State University in May of 1991. Since then he has been doing supercomputing research, user support, and applying his knowledge of supercomputers to help on scientific projects. As an employee of Ames Laboratory for 14 years, he parallelized many codes for the Condensed Matter Physics group as part of the Materials Science Grand Challenge project. Now at Kansas State University, he does supercomputing support, tools development, and assists in research projects.

Daniel Andresen holds a PhD from UCSB, and is a professor and head of Computer Science at Kansas State University, where he leads campus research cyberinfrastructure, including the Beocat HPC cluster. His research focuses on distributed systems, sensor networks, and high-performance computing. He has served as a program director at the NSF in the Computer and Network Systems division. He is a member of ASEE and ACM.

Data availability

All of our code and models together with a subset of the video dataset used in this study are made available through GitHub at: https://github.com/emilyalfs/rat-trial-annotations.