Research on machine vision online monitoring system for egg production and quality in cage environment

Abstract

In the domain of egg production, the application of automation technologies is essential for boosting productivity and quality. This study introduces an online monitoring system designed for egg quality assessment within caged environments, incorporating a robotic patrol system for egg localization and a fixed video stream for quality analysis. The project involved upgrading traditional henhouses with enhanced wireless connectivity and developing data transmission techniques for video streams and image data. The core of the system, an enhanced You Only Look Once Version 8-small (YOLOv8s) model, was augmented by substituting the Residual Network-18 backbone and integrating the Shuffle Attention mechanism, significantly improving egg detection precision. This refined model was implemented on Jetson AGX Orin industrial computer to facilitate real-world applications. To diverse operational needs, two distinct post-processing algorithms were developed: one for counting eggs and detecting abnormalities during robotic patrols, and another for assessing egg quality through fixed video streams, which measured crucial parameters such as egg dimensions and shape indexes. Experimental results revealed that the henhouse average network latencies of 35 ms, with signal strengths between -30 and -71 dBm, ensuring data transmission to the poultry management system. The enhanced YOLOv8s model, deployed on the Jetson AGX Orin, demonstrated well improvements: a Precision of 94.0 % (+2.4 %), Recall rate of 92.8 % (+4.6 %), Average Precision_50:95 of 91.5 % (+3 %) and F1 score of 93.4 % (+3.9 %), with a minor decrease in detection speed to 91.7 Frame Per Second (-18.2). Field experiment in 60 chicken cages during robotic patrols achieved an egg recognition rate of 98.9 %, validating the system's effectiveness. In fixed settings, an 83-minute experiment managed to analyze egg numbers and abnormalities, attaining a 100 % recognition rate with all scoring data promptly relayed back to the management system. Overall, this research introduces a comprehensive system for monitoring egg production and quality in cage environments, addressing manual recording and quality assessment challenges in caged poultry farming. This study is crucial for optimizing modern livestock management, enhancing production efficiency, and ensuring animal welfare.

Graphical abstract

Introduction

As the global population grows and living standards improve, the demand for poultry products continues to rise (Neeteson et al., 2023). Eggs, as a staple in diets worldwide, have a direct impact on the quality of life and health of consumers (Castro et al., 2023). With rising labor costs and increasingly stringent consumer demands for food safety and quality, traditional poultry farming methods are struggling to meet market needs (Franzo et al., 2023). Currently, Europe has implemented policies to phase out caged farming in favor of animal welfare, and China is actively exploring similar measures (Sinclair et al., 2020). However, given China's large population, a rapid transition to free-range farming may not sufficiently meet supply and demand, while caged farming offers scalability and consistency. Therefore, in China, the focus remains on utilizing modern technologies to enhance egg production efficiency within caged environments. This approach not only meets market demands but also actively seeks to improve animal welfare within scalable farming systems (Cui et al., 2023; Hemsworth, 2021).

In practical production management, flocks with poor egg production performance must be timely identified and culled, a process that involves recording the number of eggs produced and noting any abnormalities such as blood spots, cracks, soft shells, and irregular deformities during production (Barzegar et al., 2020; Guyonnet et al., 2023). Currently, in traditional caged hen facilities, the recording of egg quantity and quality is primarily conducted by inspectors who patrol the henhouse two to three times daily, observing and manually recording each cage's egg output and quality. This labor-intensive process not only poses a risk of errors in egg recording but also results in visual fatigue that can cause irreversible physical harm to the inspectors. Furthermore, the collection and sorting of eggs represent critical steps in poultry farming. Traditional farms rely on conveyor belts at the bottom of cages to automate egg collection to a manual sorting platform (Ahmed et al., 2023). Although this method efficiently collects eggs, the rapid and accurate sorting of egg quality during conveyor operation demands more from existing manual sorting techniques. Traditional manual sorting not only involves high labor intensity but also introduces human error, often failing to meet the accuracy and efficiency standards required by modern poultry management (Demyanenko et al., 2021).

As the limitations of traditional poultry farming management become increasingly apparent, employing advanced digital technologies and automation to monitor, manage, and optimize farming processes has emerged as a crucial strategy for improving competitiveness (Berckmans et al., 2017; Yang et al., 2023; Fang et al., 2024a). The application of information and precision management technologies not only significantly boosts production efficiency but also effectively reduces operational costs, while ensuring product quality meets increasingly stringent food safety standards (Ellen et al., 2019; Wu et al., 2022; Candelotto et al., 2022; Yang et al., 2024). The adoption of precision farming technologies poses new challenges and demands for enhancing flock productivity, ensuring the sustainability of farming processes, and promoting environmental friendliness (Norton et al., 2019; Fang et al., 2021; Wu et al., 2023).

Given the escalating demand for intelligent egg detection systems, several researchers have devoted efforts to developing devices and algorithms for this purpose. Ab Nasir et al. (2018) developed a machine vision-based device that extracts eight shape features from eggs and classifies them into six quality levels, achieving a classification accuracy of 94.16 %. Alikhanov et al. (2019) extended this technology to conveyor belt systems, indirectly estimating egg weight across four categories (S, M, L, XL) and attaining a sorting accuracy of 94.6 % at a rate of two eggs per second. In 2023, Yang et al. (2023) introduced an automated egg grading and defect detection system based on machine vision, which reached an optimal classification accuracy of 94.8 %. Mizuno et al. (2020) developed a visual egg inspection system tailored for small poultry farms using Convolutional Neural Networks, which accurately detects egg defects and swiftly displays results. Liu et al. (2023) engineered an egg sorting system that integrates a mechanical arm and a smart tray, utilizing CenterNet machine vision to achieve 99 % accuracy in egg handling and positioning. Nasiri et al. (2020) implemented an automatic sorting system for unwashed eggs using a modified VGG-16 Network, achieving a classification accuracy of 94.84 % through 5 fold cross-validation. Okinda et al. (2020) introduced a system for estimating chicken egg volume using depth imaging and regression models, attaining high accuracy across different egg configurations. Han et al. (2022) devised egg sorting models using visible-near infrared spectroscopy to classify eggs by color, integrity, and hen feeding mode, achieving high accuracy with techniques such as the backpropagation neural network and principal component analysis. Dibakoane et al. (2023) utilized multi-elemental fingerprints with chemometrics to accurately distinguish between cage and free-range eggs using atomic absorption spectrometry and colorimetry, achieving high classification accuracy. Turkoglu (2021) created a machine vision system employing deep learning and BiLSTM for defective egg detection, attaining a 99.17 % accuracy rate in identifying egg surface anomalies like cracks, blood, and dirt. Wang et al. (2021) unveiled a high-efficiency dirty-egg detection system that utilizes YOLOv4 on a Jetson Nano, identifying dirty spots on eggs with 75.88 % accuracy and processing images at 2.3 Frame Per Second (FPS), offering a promising solution for improving assessments of egg cleanliness.

Moreover, enhancing object detection algorithms with attention mechanisms and backbone network substitutions has been demonstrated as a viable strategy in the livestock industry (Jiang et al., 2024; Wang et al., 2024; Yang et al., 2023). For instance, Jiang et al. (2022) integrated three Convolutional Block Attention Module units into the backbone network of YOLOv7 to address the counting issues in muscovy duck farming, enhancing the network's feature extraction capabilities and achieving a 95.8 % accuracy rate. Yu et al. (2022) addressed the challenge of timely and accurate detection of estrus behavior in large-scale sheep farms by replacing the Darknet-53 backbone in the original YOLOv3 with EfficientNet-B0, making the model more lightweight and easier to deploy while reducing detection time. Although the integration of attention mechanisms and backbone replacements has theoretically shown potential to enhance model performance, the practical effects of these improvements on egg detection still require rigorous experimental validation. This paper will test and compare these advanced mechanisms in practice to verify their real-world applicability and effectiveness in enhancing egg detection performance.

In current practices of caged egg production, precisely monitoring each hen's egg output and the quality of the eggs is crucial for enhancing farming efficiency. While previous research has provided valuable techniques and methods, these predominantly focus on specific areas, such as algorithmic development or equipment design, and are often tested in experimental settings rather than real poultry environments. Consequently, few systems comprehensively meet the demands of actual poultry farms (Harnsoongnoen et al., 2021; Fang et al., 2024b; Zhang et al., 2023; Gayathri et al., 2023). In response, this study aims to develop an integrated detection system capable of real-time recording of egg production and accurate assessment of egg quality, providing a viable tool for optimizing management in modern animal husbandry.

This study enhances traditional detection methods by integrating machine vision, deep learning, and database technologies in the precision farming sector, making the proposed egg quality detection and record system more intelligent. The system employs machine vision and deep learning algorithms to monitor and record egg production and quality in each cage at caged poultry farms in real-time, allowing farm managers to efficiently identify and cull underperforming flocks, thereby reducing operational costs and enhancing productivity. Additionally, the system meticulously assesses egg quality and detects anomalies on the henhouse conveyor belts, laying the groundwork for eliminating low-quality eggs from the outset. Moreover, by uploading detection data to a cloud database in real-time, this system not only provides immediate data support for farm managers but also enriches the resources available for data analysis and decision-making, aiding in the development of more effective management strategies and the optimization of farming processes.

The main contributions of this paper are summarized as follows:

• (1)Network Infrastructure and Data Transmission Design: The study designed a network architecture to ensure stable and real-time data transmission within the henhouse. By integrating routers and wireless access points, comprehensive network coverage inside the henhouse was achieved, supporting rapid data transfer. Utilizing Alibaba Cloud database and Real-Time Streaming Protocol (RTSP) for video streaming, the system facilitates real-time data synchronization from the henhouse to the control room.
• (2)Egg Detection Algorithm Enhancement: The study improved the egg detection algorithm based on the YOLOv8s model by incorporating attention mechanisms and backbone network replacement techniques, enhancing the accuracy and reliability of egg identification.
• (3)Patrol Egg Detection System Application: During stationary periods of the egg conveyor, the system, equipped with a gimbal-stabilized camera mounted on a robot, automatically captures and classifies images of eggs and monitors each chicken cage's egg production in real-time. A threshold processing algorithm was developed to select key frames from the video stream that contain the highest number of eggs or exhibit anomalies. The results for each cage, including counts and conditions of eggs, were recorded and uploaded to a cloud-based management system for further analysis and decision-making by farm manager
• (4)Fixed-Position Egg Detection System Research: While the egg conveyor is in operation, high-definition cameras capture egg images. Post-processing algorithms measure and analyze key quality parameters such as size, shape index, and volume, automatically scoring egg quality and organizing data for upload to the cloud database, supporting subsequent automated egg sorting.

Methodology

Ethical statement

All experiments were performed according to regulations and guidelines established by the experimental animal administration and ethics committee of South China Agricultural University (SCAU#SYXK-2019-0136).

Technical roadmap of the study

The technical roadmap of this study is depicted in Fig. 1.

Fig. 1.

Overall technical roadmap.

This roadmap outlines an approach to the development of an intelligent egg quality monitoring system within a Precision Livestock Farming (PLF) context. The process begins with the setup and testing of network transmission capabilities to ensure data connectivity. Following this, the egg dataset is collected under moving and fixed states. After filtering and labeling, a comprehensive egg dataset is obtained, which is then used to build and refine the original egg detection model. Enhancements to the model include the replacement of the network backbone and the introduction of an attention mechanism, aimed at optimizing detection accuracy and speed. The roadmap culminates in the integration of these advancements into two main systems: the egg localization and analysis system, and the egg surface quality inspection system, both designed to support real-time online monitoring of egg production and quality.

Image transmission and network configuration in the henhouse

Network Coverage Design within the Henhouse

The system utilizes 3 wireless Access Point (AP) and 1 router, interconnected via Power over Ethernet (PoE) enabled cables, to establish a stable network transmission system. Under normal conditions, this network can connect to the external internet and synchronize data via Alibaba Cloud servers. The Alibaba Cloud servers store a list that contains the algorithm mode and location, the number and status of eggs, detection time, and images in base64 format. The industrial computer uses Python code to upload the processed images to this list on the server, and the management system displays the content by accessing the server's list information. Should the external network connection be interrupted, the system can still maintain data exchange between the henhouse robots and the monitoring center via an internal wireless Local Area Network, ensuring the continuous operation of the monitoring system. The network connection configuration is illustrated in Fig. 2.

Fig. 2.

Network connection layout of devices.

To adapt to the complex environmental conditions within the henhouse, the EAP662(G) device (Ruijie, China) was selected as the signal source. This device not only supports PoE but also boasts an IP68 rating for water and dust resistance, ensuring stable operation in the harsh environment inside the henhouse. The main control device, the router EG210G-P (Ruijie, China), is responsible for data aggregation, processing, and forwarding. The router also supports PoE, providing a stable power supply to the APs via Ethernet cables.

Given the dimensions of the henhouse at 90 × 12.5 m² and considering that each AP has a recommended signal coverage radius of 50 m, three APs were deployed to ensure stable transmission of image and video stream data from any corner of the henhouse. The approximate locations of the devices are shown in Fig. 3.

Fig. 3.

Main networking equipment location diagram.

To ensure the efficiency and stability of network coverage within the henhouse, tests were conducted following the network setup. The tests covered ten different points inside the henhouse, assessing the environment and signal strength at each location. Fig. 4 displays the variation in signal strength from positions 0 to 9, presented on an expanded horizontal layout of the henhouse for clarity.

Fig. 4.

Network signal test points.

Internet image transmission based on Alibaba Cloud

To enhance the efficiency of poultry farming management and monitoring, this study implemented an internet image transmission technology based on Alibaba Cloud. Utilizing this technology, the industrial computer can upload image data obtained from egg recognition patrols in real-time to the cloud server. This process encompasses recording egg counts and anomalies as well as categorizing egg quality, providing managers with a comprehensive and real-time data viewing and decision-making platform. The image transmission process is illustrated in Fig. 5.

Fig. 5.

Internet image transmission flowchart.

The procedure starts with the initialization of the cloud database connection using preset user credentials, ensuring a secure and reliable starting point for data transfer. The system regularly checks the connection status to the cloud database and immediately attempts reconnection while notifying the user if any interruption is detected, thus maintaining continuous and stable data transfer. Image files are converted into base64 format to optimize the transmission process and ensure that the image data is stored in the database as a byte array. The converted images and their associated data, including algorithm type, cage location, quantity, quality, timestamp, and additional notes, are then uploaded to the cloud server. This distinction between algorithm types facilitates differentiation between patrol and fixed-point detection data. Upon completion, the system provides feedback to confirm the successful cloud storage of the data.

Through these steps, managers can access and review the data in the cloud database in real-time using the poultry management system developed by our team. This process not only improves the efficiency of data handling and viewing but also supports poultry farming management technically. Utilizing these real-time data, managers can make timely decisions, such as investigating cages with abnormal production or optimizing the breeding environment based on egg quality data, thereby enhancing overall farming efficiency and product quality (Zheng et al., 2021).

Local area network video streaming based on RTSP

To facilitate real-time video surveillance under conditions of network instability or when a secure, enclosed data transmission environment is required, this study employs a local area network video streaming solution based on the RTSP. This setup allows managers in the control room adjacent to the henhouse to access video streams processed by industrial computers via WIFI, effectively enabling real-time monitoring and management of the interior of the henhouse. The RTSP setup process is depicted in Fig. 6.

Fig. 6.

Local network RTSP video stream diagram.

The procedure involves initializing the video streaming RTSP address by configuring the FFmpeg streaming format and setting up the RTSP address to ensure correct video stream delivery, with a resolution of 1280 × 720, a frame rate of 30 FPS, and video encoding in H.264 format. Following this, the Mediamtx streaming software is launched to push the video processed by the industrial computer into the local area network. Simultaneously, the object detection algorithm running on the industrial computer processes the real-time captured video and integrates the detection results (video frames with detection boxes) into the stream. On the monitoring side, the RTSP stream is accessed to receive and view the real-time video stream with object detection information, facilitating real-time surveillance of the henhouse interior.

Through the RTSP video streaming solution designed in this study, managers can maintain real-time awareness of the dynamics within the henhouse via a local network environment without the need for an external internet connection, providing a new and efficient monitoring tool for poultry farming management.

Datasets

In pursuit of achieving efficient poultry farming, developing a model that can accurately identify eggs is crucial. This capability not only enhances the operational efficiency of the farm but also allows for real-time monitoring of egg production, thereby optimizing management strategies. The realization of this goal relies on a high-quality dataset that reflects various egg production scenarios. Consequently, this section establishes a dataset collected in a real henhouse environment, encompassing patrol and fixed conveyor belt egg data. It details the process of constructing training datasets designed for patrol and fixed detection of eggs, including the environment for data collection, the equipment used, the methods of collection, and the specific steps involved in dataset construction and pre-processing.

Acquisition of datasets

The data collection for this study was conducted at the Lixing Agricultural Development Co., Ltd., located in Raoping County, Chaozhou City, Guangdong Province. The henhouse is divided into left and right sides, each containing four levels of chicken cages. The subjects of the study were Guangdong yellow-feathered laying hens, with each cage housing between six to nine chickens aged from 40 to 70 weeks. The henhouse's large daily egg output provided a rich and effective field data resource for the study. The actual environment of the henhouse is shown in Fig. 7.

Fig. 7.

Actual henhouse environment.

For patrol data acquisition, the A8 Mini gimbal camera (Siyi, China) equipped with mechanical stabilization was utilized, specifically designed for video stream capture in patrol environments. The camera's mechanical gimbal stabilization technology effectively eliminates image blurring caused by robot movement, ensuring the clarity of the captured video stream. Additionally, its ethernet control functionality allows flexible adjustment of the gimbal's directional angle, optimizing data collection angles. The camera, mounted on the side of the patrol robot, is positioned approximately 50 cm away from the target. The camera ensures continuous monitoring of egg production in each row of cages.

Data collection was scheduled from 4:00 PM to 7:00 PM on January 18, 2024, during which the egg conveyor belt was stationary. The camera was set to a resolution of 1280 × 720 pixels, with adjustable capture angles from 0° to -30°, and a frame rate of 30 FPS. The robot patrolled different aisles, recording video data collected during this period, from which image samples were subsequently extracted through frame pulling. Considering image quality, the patrol robot's movement speed was set to 0.2 m/s to obtain clear and stable image data. Fig. 8 shows the collection site environment and some image samples.

Fig. 8.

Patrol dataset collection site and selected image samples. Note: A. Patrol dataset collection site; B. selected image samples from patrol.

For fixed-point video stream data collection, the C922Pro camera (Logitech, Switzerland) was selected due to its high image acquisition capability and suitability for capturing detailed features of eggs, providing rich learning material for the deep learning models. This camera supports 1080p HD video recording and has an automatic light correction feature, ensuring clear images under varying lighting conditions. This data collection trial was conducted from 6:00 AM to 11:00 AM on January 16, 2024, a timing chosen based on the egg-laying cycle to ensure the collection of representative egg samples. The collection system, composed of a mobile power source, industrial computer, and camera, was installed at the end of the conveyor belt to ensure stable operation and continuous data collection. The camera was positioned at a 90° downward angle with a fixed shooting distance of 26.1 cm, ideally capturing eggs on the conveyor belt. The frame rate was set to 10 FPS, and the conveyor belt speed was 0.1 m/s, adequately capturing the state of each egg.

To address the issue of a low number of anomalous egg samples during the collection process, this trial included the collection of anomalous eggs from within the henhouse, individually placing them on the conveyor belt to effectively expand the sample of anomalies. This step is crucial for training the model to recognize various anomalies. Fig. 9 displays the fixed-point dataset collection site environment and some image samples.

Fig. 9.

Fixed-position dataset collection site and selected image samples. Note: A. fixed-position dataset collection site; B. selected image samples from fixed-position detection.

Data pre-processing

The data pre-processing stage is crucial for building a high-accuracy egg detection model, encompassing steps such as data filtering, annotation, and format conversion. This stage aims to enhance the quality of the dataset to ensure its suitability for model training.

During the filtering process, images rendered unusable due to operator errors, exposure issues, or blurriness were excluded. By extracting one frame per second from the video streams, 1,004 images were selected from the patrol video stream and 2,090 from the fixed-point video stream, totaling 3,094 images. This selection ensures the dataset is not overly large yet covers a diverse range of scenarios.

Annotation was performed using the EasyDL platform on Baidu, and the datasets in VOC format were subsequently converted to the format required for training YOLO models (Ma et al., 2019). In the patrol video stream dataset, due to the dynamic nature of video shooting causing some images to blur, it became challenging to precisely differentiate the detailed states of the eggs. Therefore, annotation categories were simplified to "egg", "egg_abnormal", and "fence". Fence annotations help determine the pixel range of chicken cages. For the fixed-point video stream dataset, where images are stable and clear, allowing for finer distinction of egg conditions, annotations included "egg", "egg_broken", "egg_dirty", "egg_blood", and "egg_softshell". The annotation interface and examples of abnormal samples are shown in Fig. 10.

Fig. 10.

Annotation interface and examples of anomalous samples. Note: A. Annotation interface; B. examples of anomalous samples.

Table 1 presents an overview of the annotated sample quantities after completion. This table reflects the annotation strategy employed in dataset construction, aimed at balancing the ratio of normal to abnormal egg samples to ensure the model can learn from various scenarios. The total of 3,094 samples includes normal and various abnormal eggs, demonstrating the dataset's diversity and complexity. The distribution of annotation boxes across categories shows the dataset's coverage breadth, aimed at enhancing the model's ability to recognize different egg conditions. The number of collected samples considers the needs of model training, ensuring there is sufficient data to support model learning and generalization capabilities.

Table 1.

Number of labeled samples.

Total number	Classification	Number of labels
3,094	egg	3,944
	egg_broken	285
	egg_dirty	128
	egg_blood	158
	egg_softshell	184
	fence	808
	egg_abnormal	194

Egg detection model based on YOLOv8

Structure and Features of the YOLOv8 Algorithm

In egg recognition algorithms, the object detection component is crucial as it plays a key role in identifying and locating chicken cage fences and eggs. YOLO is an end-to-end object detector that is easy to train and performs well in terms of speed and accuracy. YOLO has undergone several iterations, one of the advanced versions being YOLOv8. Developed by Ultralytics, YOLOv8 represents a continuation of the YOLOv5 original team's efforts (Jocher et al., 2020), and its excellent scalability facilitates further improvements. Considering the need to deploy an egg detection algorithm on industrial computers within robots, this study selected YOLOv8s as the initial object detection model. The size of the model affects both computational power and accuracy: larger models are more precise but process data more slowly, while smaller models are faster but less accurate. While our current industrial computers can handle data from a single camera, future applications may require processing data from multiple cameras simultaneously. Therefore, the model must be sufficiently lightweight to run multiple instances without exceeding computational resources. Balancing accuracy and computational efficiency, we chose YOLOv8s because it meets our computational requirements while offering relatively high accuracy. The specific network structure of the object detection model is illustrated in Fig. 11.

Fig. 11.

Improved YOLOv8s network structure for detecting eggs and fences. Note: CBL = Convolution with batch normalization and SiLU; SiLU = Sigmoid linear unit; C2f = CSPBottleneck with 2 convolutions; SA = Shuffle attention module.

The YOLOv8s model comprises four main parts: Input, Backbone, Neck, and Head. The Input stage handles the pre-processing of images, such as resizing and normalization. The Backbone network primarily serves for feature extraction, which includes multiple convolution layers, batch normalization, and activation functions. The Neck module enhances the feature maps through upsampling and other techniques, preparing for detection at various scales. The Head module is responsible for generating the final detection results, including the bounding boxes of objects, class identification, and confidence scores.

To enhance the efficiency and accuracy of feature extraction, this paper modifies the original YOLOv8s backbone network. By replacing it with the theoretically more efficient Residual Network Layer 18 (ResNet-18) network and testing it in practical experiments, this improvement has enhanced the model's ability to recognize egg features against complex backgrounds (He et al., 2016). This backbone network replacement not only optimizes the feature processing workflow but also improves the overall performance and accuracy of the model in detecting egg quality.

Additionally, this study introduces the Shuffle Attention (SA) mechanism at the end of the backbone and the beginning of the head of the model (Vaswani et al., 2017; Zhang and Yang, 2021). These mechanisms specifically target and weight key features of eggs, significantly increasing the model's focus on and accuracy in detecting targets. With the incorporation of the attention mechanism, the model's performance in complex environments has markedly improved, effectively enhancing the detection system's adaptability and accuracy in practical applications.

Evaluation metrics

To assess the performance of the object detection algorithm introduced in this paper, several evaluation metrics were used, including Precision (P), Recall (R), F1 Score, Intersection over Union (IoU) with an Average Precision (AP_50:95) across IoU thresholds from 0.50 to 0.95, FPS, the number of parameters, and model size. P is used to evaluate the accuracy of the model's predictions, with higher values indicating greater accuracy. R assesses the model's ability to minimize missed detections, where higher values indicate fewer misses. The F1 Score evaluates the balance between P and R, with values closer to 1 indicating better model performance. AP_50:95 represents the model's average precision across various IoU thresholds, from 0.50 to 0.95, with higher values indicating superior performance. FPS measures the model's processing speed, where higher values indicate faster processing. The number of parameters and model size are important indicators of the model's scale, where the number of parameters represents the total internal parameters of the model, and model size reflects the storage space occupied on the industrial computer, with smaller values indicating a more compact and lightweight model.

P is calculated as the ratio of True Positives (TP) to the sum of true positives and False Positives (FP), where higher values denote higher precision. The formula for precision is shown in Eq. (1):

$$P=\frac{TP}{TP+FP}$$ (1)

R is the ratio of true positives to the sum of true positives and False Negatives (FN), where higher values indicate lower rates of missed detections. The formula for recall is shown in Eq. (2):

$$R=\frac{TP}{TP+FN}$$ (2)

The F1 Score is the harmonic mean of P and R, treating both metrics as equally important for evaluating model performance. Values closer to 1 indicate high levels of both P and R. The formula for the F1 Score is shown in Eq. (3):

$$F1=2·\frac{P·R}{P+R}$$ (3)

AP measures the proportion of correct predictions among all positive predictions, with higher proportions indicating fewer detection errors. AP₅₀ represents the average precision when IoU is 0.5. AP_50:95 is the mean of AP values at 10 different IoU thresholds (from 0.5 to 0.95, in increments of 0.05).

FPS measures the number of images frames the model can process per second, with higher values indicating faster processing speeds.

Additionally, to assess the performance of threshold handling and algorithm deployment, an egg detection rate metric was established. This metric evaluates the performance of the deployed algorithm in detecting eggs, calculated as the number of detected eggs divided by the actual number of eggs. The formula for this metric is shown in Eq. (4):

$$Eggrecognitionrate=\frac{Detectedeggcount}{Actualegg\text{count}}$$ (4)

Experimental setup and parameter configuration

In this study, the model training was conducted using an Intel(R) Core (TM) i7-3960X CPU @ 3.30 GHz processor, 32 GB of RAM, and a GeForce GTX 1080Ti 11 GB graphics card. The operating system utilized was Windows 11 Professional Edition. The software environment included Python 3.9, Pytorch 1.12.0, CUDA 11.3, CUDNN 8.4.1, and Visual Studio 2019. The hyperparameters used for training the model are presented in Table 2.

Table 2.

Hyper-parameter configuration for object detection model.

Hyper-parameter	Object Detection Model
Batch_Size	16
Epoch	300
Iterations	58,013
Learning Rate	1 × 10−4
Number of Iterations	193
Num_Workers	4
Optimizer	SGD

Patrol environment egg localization and analysis system

This system is designed to explore an egg localization and analysis system suitable for patrol environments. It is capable of real-time identification and recording of the number and status (normal or abnormal) of eggs within chicken cages during robotic patrols. This provides precise data support for poultry farms, enhancing the management and monitoring of egg production.

Patrol environment detection algorithm workflow

To avoid erroneously counting eggs from adjacent chicken cages, this study introduced a post-processing algorithm following the egg detection model to refine the results of the object detection, ensuring the effectiveness of the object detection model in real environments for patrol detection within the henhouse. After object detection, a threshold processing step was added, which not only filters the target eggs but also updates the count and identifies abnormalities of the eggs within each targeted cage area in real-time. The overall schematic and flowchart of the patrol detection algorithm are illustrated in Fig. 12.

Fig. 12.

Overall diagram and flowchart of the patrol detection algorithm. Note: A. Overall diagram of the patrol detection algorithm; B. Flowchart of the patrol detection algorithm.

In the egg detection and analysis within a patrol environment, the developed algorithm initially identifies the bounding boxes of chicken cage fences and eggs, extracting their horizontal coordinates to establish the basis for determining cage positions and filtering egg counts and conditions. The algorithm then precisely defines the lateral spatial range of the cage by calculating the minimum and maximum horizontal coordinates of the fence bounding boxes. In dynamic environments, especially when the number of fences is stable, the algorithm can update the number and condition of eggs in real-time. By comparing the detected egg positions with the spatial thresholds of the cages, the accuracy of the data is ensured.

Considering the dynamic changes in the number of fences, when the number stabilizes at two, the algorithm activates the process for real-time updating of egg counts. The algorithm compares the horizontal coordinates of the detected egg bounding boxes with the lateral spatial thresholds of the cages; if an egg is within the cage range, it is counted as valid data. The subsequent counting logic accounts for dynamic changes in the number of fences; when the number stabilizes at two, the algorithm activates the real-time update process for counting and identifying the condition of eggs. If the number of fences temporarily changes to one, the algorithm will perform continuous frame checks. Setting the continuous frame number to ten is based on field test results, which effectively balance computational efficiency and accuracy, avoiding misjudgments due to too few frames or resource wastage from too many frames. If fewer than ten continuous frames detect a single fence, it is determined as a false detection, and the original count and quality assessment of eggs are maintained. If the continuous frames exceed ten, the algorithm determines that it has passed that cage and then outputs the count, condition, and detection time of the eggs in that cage. This temporal data can be used in conjunction with the robot's coordinates to locate the position of the cage in the cloud database. Furthermore, the algorithm integrates QR code recognition technology to further enhance the accuracy of cage localization. Ultimately, the algorithm can output the number and health status of eggs in each cage along with key images, saving and uploading this information to the cloud database, providing managers with a comprehensive, real-time monitoring and analysis tool.

Patrol detection experiment design

To thoroughly evaluate the performance of the egg detection and analysis system in a dynamic environment, this study designed a series of experiments aimed at assessing the accuracy and reliability of the developed algorithms in real-world settings. The experimental setup included a Jetson AGX Orin industrial computer as the core processing unit, paired with A8 mini gimbal camera for video capture.

The experiments covered the bottom two layers inside the henhouse, selecting 30 cages per layer to comprehensively assess the algorithm's performance under different environmental conditions. To ensure precise identification and localization of each cage, QR code labels fixed with strong magnets were placed above each cage, providing crucial positional information for subsequent data matching and analysis.

Before starting the experiments, the robot was positioned at the entrance of the henhouse aisle. All devices were then activated, and the experiment program was initiated at a speed of 0.1 m/s. During the first layer's experiment, the camera settings were adjusted to a vertical height of 30 cm from the ground and a horizontal distance of 40 cm from the cages, with the pitch angle set to 0° to ensure comprehensive coverage inside the cages. After collecting data from the first layer of cages, the robot returned to its starting position, and the column height was adjusted to 50 cm to accommodate the detection requirements of the second layer of cages. Throughout the monitoring process, video data were reviewed in real-time on a laptop in the control room, and the egg detection results were queried using the management system to monitor the progress of the experiments. Fig. 13 shows the operational setup of the equipment and the QR code placement.

Fig. 13.

Operational diagram of equipment and QR code fixation effect. Note: A. Operational diagram of equipment; B. QR code fixation effect.

Fixed environment egg quality detection system

This system is designed to automatically identify and measure key physical parameters of eggs, such as the major and minor axes, egg shape index, and volume, based on which a preliminary quality assessment is conducted.

Egg classification algorithm

The overall schematic of the egg classification algorithm is shown in Fig. 14.

Fig. 14.

Overall diagram of the egg classification algorithm.

The key object detection algorithm initially identifies the bounding boxes of the eggs and analyzes them to determine their longest and shortest axes, thereby precisely measuring their dimensions. Based on a conversion ratio of 1 cm to 35.76 pixels, the system converts pixel dimensions into actual physical sizes. Subsequently, using the egg shape index calculation Eq. (5), the system assesses whether the shape of the eggs meets the standard criteria, with abnormally shaped eggs being classified as anomalies. The volume of the eggs Eq. (6) is calculated by considering the eggs as ellipsoids, providing another significant quality assessment parameter for this study.

Combining these geometric and shape parameters, the study further defines a quality scoring standard Eq. (7). This classification process is based not only on the physical dimensions of the eggs but also on their conformity to shape standards, thereby facilitating a comprehensive quality assessment.

Eq. (5) calculates the egg shape index, defined as the ratio of the egg's minor axis to its major axis, where a standard egg's shape index typically ranges between 0.7 to 0.75.

$$Eggshapeindex=\frac{Shortdiameter}{Longdiameter}$$ (5)

Eq. (6) outlines the method for calculating the volume of an egg. The study treats the egg as an approximate ellipsoid, calculating its volume using the formula for the area of an ellipse by obtaining its length and width.

$$\begin{array}{c}\hfill \mathit{Eggv}\mathit{olume}\approx {\frac{4}{3}\pi (\frac{\mathit{Long}\mathit{diam}\mathit{eter}}{2}\times \frac{\mathit{Shor}\mathit{tdia}\mathit{meter}}{2})}^2\end{array}$$ (6)

After determining the volume of the eggs, both the volume and the egg shape index serve as references for assessing the quality of the egg. When the volume remains constant, the quality scoring is highest between an egg shape index of 0.72 to 0.76 (with 0.74 being the optimal value). The quality score decreases as the egg shape index deviates further from the range of 0.72 to 0.76, and eggs with an index lower than 0.48 are directly classified as anomalies. The egg quality calculation is provided in Eq. (7).

$$\{\begin{array}{cc}Eggvolume& 0.72\le Eggshapeindex\le 0.76\\ Eggvolume·[1-2·(Eggshapeindex-0.74\left)\right]& 0.76<Eggshapeindex\le 1\\ Eggvolume·[1-2·(0.74-Eggshapeindex\left)\right]& 0.48\le Eggshapeindex<0.72\\ Abnormal& else\end{array}$$ (7)

Fixed-Position Detection Algorithm Workflow

To avoid confusion caused by multiple eggs on the conveyor belt, a post-processing algorithm was also implemented in the fixed-position detection component. The workflow of the fixed-position detection algorithm is depicted in Fig. 15.

Fig. 15.

Fixed-position detection algorithm workflow diagram.

Initially, the algorithm restricts the detection threshold space to the primary detection area on the conveyor belt. Only when a complete egg appears in the field of view does the algorithm calculate the corresponding parameters. Detection data from partially visible eggs on the edges are discarded. When a normal egg is detected, the algorithm outputs the quality score and key images of the eggs on the conveyor belt, saving and uploading this information to the cloud database. This enables managers to later retrieve and categorize the data, providing them with a comprehensive, real-time monitoring and analysis tool.

Fixed-position detection experiment design

A series of experiments were designed to validate the application effectiveness of the fixed-position egg quality detection system in actual poultry farm environments. Using a Jetson AGX Orin industrial computer as the primary computing platform and running the proprietary detection algorithm in conjunction with C922Pro camera, the system captures high-definition images. The system is powered by a mobile power source, ensuring stable operation on-site. The experimental setup was placed on a conveyor belt-controlled environment located at the end of the second layer within the actual henhouse, ensuring consistency with the data collection stage environment. Details of the equipment operation configuration and layout are provided in Fig. 16:

Fig. 16.

Operational diagram of fixed-position detection algorithm equipment.

During the experiment, all algorithms were activated, and the equipment continuously operated for 82 min as the conveyor belt moved. This setup aimed to simulate the actual egg sorting process by setting a detection interval of every 11 s to ensure that each egg was detected only once as it appeared in the camera's field of view, thus avoiding duplicate detections and data overload. Concurrent manual observations and recordings were made to compare and analyze against the results generated by the algorithm.

Results

Network strength and stability experiment

The results of the network stability experiment are shown in Table 3.

Table 3.

Network signal strength and stability experiment results.

Point Movement Route	RSSI Range (-dBm)	Ping Fluctuation Range (ms)
0-1	32∼65	2∼12
2-3	39∼62	2∼14
4-5	41∼71	2∼13
6-7	50∼70	3∼14
8-9	43∼68	0∼16

The experiment results indicate that as the test personnel moved from 1 AP to another, the RSSI values exhibited fluctuations with an attenuation followed by a rise, yet consistently maintained within the normal operating range of -30 to -70 dBm. Throughout the sequential movement across 10 test points, the network latency remained within 16 ms, demonstrating excellent network stability. This confirms the stable transmission of network signals with the configuration of 3 APs.

Analysis of the improved YOLOv8s model

The ablation experiment of the YOLOv8s model, after improvements with the ResNet-18 backbone replacement and the introduction of the SA attention mechanism, compared with YOLOv7-Tiny and YOLOX-s, is detailed in Table 4.

Table 4.

Comparative evaluation of enhanced YOLOv8s models in ablation trials and comparison with other algorithms.

Algorithms	Experimental Results
	P	R	F1 Score	AP50:95	FPS	Parameters	Model Size (MB)
YOLOv8s	91.6 %	88.2 %	89.87 %	88.5 %	109.9	11,128,293	21.97
YOLOv7-tiny	92.7 %	82.8 %	87.47 %	83.2 %	172.4↑	6,023,832↓	12.01↓
YOLOX-s	85.2 %	85.4 %	85.30 %	81.8 %	32.3	8,940,000	70.17
YOLOv8s+ResNet-18	93.4 %	91.1 %	92.24 %	91.5 %	102.0	17,886,245	35.16
YOLOv8s+SA	93.7 %	91.7 %	92.69 %	89.6 %	106.4	11,128,821	21.99
YOLOv8s+ResNet-18+SA	94.0 %↑	92.8 %↑	93.40 %↑	91.5 %↑	91.7	18,017,845	35.43

P = Precision; R = Recall; AP_50:95 = Average Precision for IoU values ranging from 0.5 to 0.95; FPS = Frame Per Second; MB = Mbyte.

The best values within the detection model are underlined with arrows.

Performance comparison analysis shows that while YOLOv7-Tiny exhibited excellent P, demonstrated high processing speed, and smaller model size, it lagged behind both YOLOv8s and its improved versions in terms of R and overall detection effectiveness (F1 Score, AP_50:95). This suggests that the advantages in speed and model size did not translate into higher detection accuracy. YOLOX-s performed inferiorly across all evaluation metrics compared to YOLOv8s and its improved models, particularly in precision and recall, which may be due to its architecture's inadequate adaptability for specific tasks. YOLOv8s with ResNet-18 and YOLOv8s with SA each showed the positive impact of integrating the ResNet-18 backbone and SA attention mechanism on model performance, especially in improving the model's recall and precision.

Ultimately, the combination of YOLOv8s + ResNet-18 + SA excelled across almost all evaluation metrics, particularly notable in the improvements in F1 Score and AP_50:95. This verifies the complementary enhancements of ResNet-18 and SA technologies in boosting the model's detection capabilities.

Considering performance, speed, and model complexity, the application of YOLOv8s + ResNet-18 + SA achieved significant accuracy improvements, making it the preferred model for egg detection tasks. This model not only surpassed the original YOLOv8s in detection P and R but also maintained a reasonable balance in processing speed and model size, proving its applicability and feasibility in real poultry farming environments. As shown in Fig. 17, the improved model significantly outperforms the original YOLOv8s in terms of detection accuracy and reducing misses.

Fig. 17.

Visual comparison of detection results between the original YOLOv8s and YOLOv8s+ResNet-18+SA.

Fig. 18 displays the trajectory of performance metrics for the YOLOv8s+ResNet-18 + SA model throughout the training process for both the training and validation datasets. All loss metrics, including bounding box (train/box_loss, val/box_loss), classification (train/cls_loss, val/cls_loss), and distribution focal loss (train/dfl_loss, val/dfl_loss), show a declining trend, indicating continual improvement in object localization, classification accuracy, and bounding box corner prediction. Meanwhile, the P and R metrics, although fluctuating initially, exhibit an upward trend and eventually stabilize. This stabilization suggests that the model effectively balances FP and FN, a hallmark of a maturing object detection model.

Fig. 18.

Performance metric variation curve of the YOLOv8s+ResNet-18+SA model during training.

The average precision scores at the IoU threshold of 0.5 (mAP50, i.e., AP₅₀) and from 0.5 to 0.95 (mAP50-95, i.e., AP_50:95) demonstrate a steady increase, reaching stability by the end of the training process. This feature indicates that the model has achieved an optimal fit, avoiding both overfitting and underfitting with the training data. The comprehensive improvement in these metrics underscores the increasing robustness and P of the model in object detection tasks, highlighting the effectiveness of the object detection model and training strategies.

Patrol detection experiment results and analysis

The experiments conducted in this study were designed to validate the performance of the egg localization and analysis system in a mobile environment, particularly its capability for egg recognition and quality analysis in dynamic henhouse conditions. Utilizing a gimbal-stabilized camera mounted on a patrol robot, the system demonstrated effective recognition and analysis of eggs in caged henhouse environments. Table 5 displays the egg detection efficiency and recognition rates across different levels of the henhouse.

Table 5.

Egg recognition rate for patrol detection algorithm trials.

Experimental Results	Position
	First Layer	Second Layer	Total
Actual Number of Eggs	40	50	90
Abnormal Number of Eggs	6	10	16
Number of Eggs Detected - YOLOv8s	35	48	83
Number of Eggs Detected - YOLOv8s+ResNet-18+SA	39↑	50↑	89↑
Number of Abnormal Eggs Detected - YOLOv8s	3	6	9
Number of Abnormal Eggs Detected - YOLOv8s+ResNet-18+SA	6↑	9↑	15↑
Egg Recognition Rate - YOLOv8s	87.5 %	96 %	92.2 %
Egg Recognition Rate - YOLOv8s+ResNet-18+SA	97.5 %↑	100 %↑	98.9 %↑

The best values within the detection model are underlined with arrows.

Data in Table 5 show that the introduction of the YOLOv8s + ResNet-18 + SA model not only improved egg recognition rates but also significantly enhanced the detection capabilities for abnormal eggs. Particularly in the well-lit second-level environment, the model exhibited high detection accuracy. However, despite the improved recognition rates with the model, the rates did not reach 100 %, especially in the dimly lit first level.

The experiments highlighted variations in model performance under different lighting conditions and its limitations in identifying eggs, providing clear directions for future optimization. Future research will focus on further optimizing the algorithm to enhance its accuracy and robustness under varying lighting conditions and comprehensively improve the precision of egg recognition.

By introducing a threshold post-processing algorithm, significant effectiveness in detecting valid data and filtering out invalid data was demonstrated, as illustrated in Fig. 19.

Fig. 19.

Data filtering effects of the threshold post-processing algorithm. Note: A. Effective data detection; B. Ineffective data filtering.

Fig. 19 (a) displays the detection results of valid data, when the robot can fully observe the cage interval and accurately detect the eggs therein. At this point, the system backend uses the pyzbar library to read the QR code of the chicken cage, recording the cage's number and position. With the help of the post-processing algorithm, the detection result for each cage is restricted to uploading a representative image and its related data, thereby establishing accurate, efficient records in the cloud database.

Fig. 19 (b) shows the effect of filtering out invalid data when the robot's view does not fully capture the cage, i.e., when the number of fence detection boxes is not two. In such cases, the collected data are automatically discarded, preventing the accumulation of excessive invalid data and unnecessary computational expenditure.

Fixed-position detection experiment results and analysis

The experiment focused on evaluating the performance of the fixed-position detection system in recognizing the quantity of eggs and identifying their abnormal states. Due to the subjectivity of manual measurement, the comparison primarily focused on the accuracy of identifying abnormal egg states. Table 6 shows the comparison results between the algorithm and manual detection:

Table 6.

Comparative results of fixed-position egg detection.

Egg Classification	Algorithm	Manual Method
Total Number	185	185
Number of Normal Eggs	179↑	177
Number of Abnormal Eggs	6	8↑
Egg Recognition Rate	100 %	100 %

The best values within the detection model are underlined with arrows.

The results indicate that the fixed-position detection system performed excellently on the conveyor belt in detecting egg quantities, correctly identifying all 185 eggs, which matched the count obtained through manual detection. However, its performance in recognizing abnormal eggs was slightly lacking, partly because the image information was captured from a top-down perspective, preventing a complete view of the eggs' back sides.

The visualization results of the fixed-position egg quality detection system are reflected in Fig. 20, illustrating the algorithm's application effects in an actual farming environment.

Fig. 20.

Recognition effects of the fixed-position detection algorithm. Note: A. Case of normal egg; B. case of low-quality score egg; C. Case of abnormal egg.

Fig. 20 (a) displays the detection results for normal eggs, where the algorithm can select the detection boxes in the central area of the eggs for analysis. Since the egg shape index for these eggs falls between 0.72 to 0.74, the algorithm first assigns a "good" rating to the egg shape index and begins calculating the quality score to output the recognition results.

Fig. 20 (b) shows an example of eggs with low quality scores. These eggs have an egg shape index close to 1 (0.97), combined with a shorter major axis, resulting in a quality score below 40.

Fig. 20 (c) represents cases of abnormal eggs. When eggs exhibit signs of "blood stains," "damage," "soft shells," or "dirt," no quality score is calculated; instead, an abnormal signal is directly outputted. All detected information is saved locally and uploaded to the management system.

Discussion

This study developed two systems for the poultry farming industry: a caged egg localization and analysis system, and an egg quality detection system, both utilizing machine vision and deep learning technologies. These systems automate the processes of egg localization, analysis, and quality detection, addressing the practical needs of the industry. However, the research also highlighted some shortcomings and outlined directions for future studies.

• (1)Regarding the patrol inspection algorithm: Currently, the patrol robot needs to avoid the light bulbs in the center of the henhouse aisles, which limits its ability to patrol centrally, thereby affecting the precision of quality scoring based on pixel-to-distance conversion. Since both the upper and lower levels of the henhouse require adequate lighting, supplementary lighting relies on light bulbs. To ensure coverage of the lower levels, some bulbs are installed at relatively low heights. If the robot were to move in a straight line without avoidance, it would risk colliding with these low-hanging bulbs. Therefore, the robot must perform path obstacle avoidance, leading to variable distances when capturing images. This variability affects the precision of quality scoring based on pixel-to-distance conversion. As a result, in patrol mode, it is currently difficult to perform detailed quality scoring. Future developments will include a dynamic pixel distance conversion algorithm to ensure accurate egg quality scoring during patrols.

  Moreover, there is room for improvement in egg detection efficiency and adaptability to changes in lighting conditions. Therefore, future research will focus on optimizing both hardware and software, including introducing a brightness enhancement algorithm before capturing video streams or equipping the patrol robot with supplementary lighting devices to enhance the algorithm's robustness to lighting variations.

• (2)Regarding the fixed-position detection algorithm: Due to the limitations of experimental conditions, this experiment primarily focused on egg count detection and quality classification, with limited actual comparative analysis of quality scoring. Additionally, the top-down detection approach struggles to assess quality defects on the backs of eggs. Therefore, future research will further validate and improve the quality scoring algorithm to address these shortcomings.

Regarding equipment, the study has not yet achieved automated egg sorting within the henhouse. Thus, future research will approach this from the perspectives of hardware and system integration, designing and implementing mechanical devices for effective egg diversion to reduce the labor intensity of manual sorting.

Conclusions

In this study, we developed a caged egg localization and analysis system and an egg quality detection system using machine vision and deep learning technologies, addressing practical needs in the modern poultry farming industry. By offering solutions to issues like low efficiency in egg classification and dependence on high-cost equipment, we provide a low-cost, high-efficiency egg detection solution that enhances automation and scientific management of egg quality through data analysis.

We established a stable image transmission network within the henhouse for efficient real-time video streaming. The system leveraged the YOLOv8s model, enhanced by a ResNet-18 backbone and SA attention mechanism, demonstrating robust performance across varied testing scenarios. Validated through experiments involving 60 chicken cages, the system utilized a robotic patrol system for anomaly detection and quantity analysis via a moving video stream, employing a threshold algorithm for precise condition assessments of eggs. For the fixed-position algorithm, it conducted detailed quality classification and scoring based on the eggs' physical dimensions over an 83-minute experiment. All data, including images, statuses, and quality scores, were systematically cataloged and integrated into the management system, illustrating potential improvements in operational efficiency and data management.

Future research will focus on enhancing detection robustness and efficiency by developing a dynamic pixel distance conversion algorithm and integrating lighting adjustment technologies. Efforts will also be directed towards refining the quality scoring algorithm and integrating automated egg sorting technologies to reduce manual labor reliance and promote system integration.

Disclosures

The authors declare no conflicts of interest.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.