Estimating body weight and body condition score of mature beef cows using depth images

Abstract

Obtaining accurate body weight (BW) is crucial for management decisions yet can be a challenge for cow–calf producers. Fast-evolving technologies such as depth sensing have been identified as low-cost sensors for agricultural applications but have not been widely validated for U.S. beef cattle. This study aimed to (1) estimate the body volume of mature beef cows from depth images, (2) quantify BW and metabolic weight (MBW) from image-projected body volume, and (3) classify body condition scores (BCS) from image-obtained measurements using a machine-learning-based approach. Fifty-eight crossbred cows with a mean BW of 410.0 ± 60.3 kg and were between 4 and 6 yr of age were used for data collection between May and December 2021. A low-cost, commercially available depth sensor was used to collect top-view depth images. Images were processed to obtain cattle biometric measurements, including MBW, body length, average height, maximum body width, dorsal area, and projected body volume. The dataset was partitioned into training and testing datasets using an 80%:20% ratio. Using the training dataset, linear regression models were developed between image-projected body volume and BW measurements. Results were used to test BW predictions for the testing dataset. A machine-learning-based multivariate analysis was performed with 29 algorithms from eight classifiers to classify BCS using multiple inputs conveniently obtained from the cows and the depth images. A feature selection algorithm was performed to rank the relevance of each input to the BCS. Results demonstrated a strong positive correlation between the image-projected cow body volume and the measured BW (r = 0.9166). The regression between the cow body volume and the measured BW had a co-efficient of determination (R²) of 0.83 and a 19.2 ± 13.50 kg mean absolute error (MAE) of prediction. When applying the regression to the testing dataset, an increase in the MAE of the predicted BW (22.7 ± 13.44 kg) but a slightly improved R² (0.8661) was noted. Among all algorithms, the Bagged Tree model in the Ensemble class had the best performance and was used to classify BCS. Classification results demonstrate the model failed to predict any BCS lower than 4.5, while it accurately classified the BCS with a true prediction rate of 60%, 63.6%, and 50% for BCS between 4.75 and 5, 5.25 and 5.5, and 5.75 and 6, respectively. This study validated using depth imaging to accurately predict BW and classify BCS of U.S. beef cow herds.

Introduction

Beef production is one of the most important agricultural industries in the United States, accounting for $66.2 billion in cash receipts in 2019 (USDA, 2021). In 2021, the cow–calf production totaled 40.9 million head of cows and heifers that have calved, comprising approximately 40.5% of the total U.S. beef cattle inventory (USDA-NASS 2015, 2021). Of 882,692 cattle and calf operations, 802,317 are cow–calf and stocker/backgrounder cattle farms, of which 96% are family-owned and operated (NCBA, 2021). Like many other livestock production systems, beef cattle production, particularly the cow–calf production on western rangelands, faces a growing quest for improved production efficiency for producers to survive and thrive amidst increasing cost of production and declining profitability.

The average beef cow weighs 550 kg or more and can range from 408 to 800 kg, depending on cow age and breed (NRC, 2016; Bir et al., 2018). Obtaining an accurate body weight (BW) of the cows is an essential measure to guide cow–calf producers regarding production management. Accurate measurement of cow BW can provide valuable and timely information to the producers and result in greater production efficiency (Watson et al., 2013). However, measuring cow BW is challenging, and variations can be present due to animals themselves (breeds, age, size), different weighing techniques from the care handlers, day of weighing, and ambient conditions (Watson et al., 2013). Stock et al. (1983) concluded that weighing cattle over multiple days yielded greater precision, yet this method is economically and practically prohibitive for industry settings. Obtaining accurate cow BW remains a challenge for producers, particularly those that manage in large extensive cow–calf environments.

Precision livestock management (PLM), also known as precision livestock farming, has been fast emerging to provide producers with more accurate or individualized information on the health, welfare, and production of the cows in the herd. This information helps producers make better and faster decisions about the animals’ individual needs and identify management problems (Salau et al., 2016; Norton and Berckmans, 2018). Among different PLM tools, imaging technologies have advantages over other technologies, providing more efficient data collection and the ability to monitor individual animals for multiple species (Halachmi et al., 2013; Van Hertem et al., 2014; Condotta et al., 2020a; Li et al., 2020). Among a wide variety of imaging sensors, depth cameras have been identified as low-cost monitoring tools for agricultural applications (Ruchay et al. 2019, 2020; Condotta et al. 2020a, 2020b; Jang et al., 2020; Kamchen et al., 2021). Condotta et al. (2018) conducted a study using depth images to obtain dimensions of pigs for weight estimation. In their research, different geometric characteristics of a standing pig (length, width, shoulder width, and height) were measured using depth cameras (Kinect I, Microsoft, Seattle, WA, USA), and algorithms were developed to accurately estimate animals’ weight with a standard error of 3.01 kg and a linear regression co-efficient (R²) of 0.99 (Condotta et al., 2018).

The dairy industry, particularly the European dairy sector, has embraced computer vision technology earlier than the United States, as evidenced by more research and advanced adoption among commercial operations (Kang et al., 2021). For example, image processing and computer vision have been studied in dairy production for multiple purposes, including feed intake of dairy cows (Bezen et al., 2020), milk production and udder traits (Shorten, 2021), individual cow identification (Kumar et al., 2018; Tassinari et al., 2021), physiological responses (Jorquera-Chavez et al., 2019), body condition scoring (Bewley et al., 2008; Halachmi et al., 2013; Rodríguez Alvarez et al., 2019; Liu et al., 2020; Zhao et al., 2020), backfat thickness (Weber et al., 2014), behavior classification (Porto et al., 2013; Chen et al., 2021; Li et al., 2021), lameness detection of dairy cows (Van Hertem et al., 2014; Zhao et al., 2018; Kang et al., 2021), and BW estimation (Song et al., 2018; Rudenko et al., 2020; Dohmen et al., 2022). Using depth imaging techniques, Song et al. (2018) achieved a 41.2 kg root mean square error (RMSE) when predicting BW of mature Holstein cows using daily intake, parity, and hip width as input variables. Weber et al. (2014) reported a correlation co-efficient of 0.96 between observed values and estimated backfat thickness. For studies that predict body condition scores (BCS) of dairy cows, using advanced machine learning algorithms such as transfer learning and ensemble models, researchers were able to achieve an accuracy above 76% with a BCS deviation smaller than 0.25 (Rodríguez Alvarez et al., 2019; Liu et al., 2020).

Although such low-cost technologies are proven effective in the dairy industry and have extended merit for monitoring beef cattle in efficient and noninvasive ways, they have not been widely studied nor validated for the U.S. beef cattle sectors. Current studies utilize depth image sensing concentrate in other locations of the world (e.g., European Union, Australia, etc.), including weight prediction for Brazilian Nellore heifers (Cominotte et al., 2020; Kamchen et al., 2021), recognition of Pantaneira cattle (de Lima Weber et al., 2020), and individual feedlot cattle identification via muzzle images (Li et al., 2022). Owing to the significant difference between the body composition and morphological traits of dairy and beef cattle breeds, and different rubrics used to predict BCS of dairy cows (Ferguson et al., 1994) and beef cows (Rasby et al., 2014), this gap in literature calls for a critical need to explore of the feasibility of using such imaging technology in predicting BW and BCS for cow–calf operations in the United States. Prediction of BW and BCS would provide livestock producers the opportunity to pro-actively monitor and manage BW and BW changes in grazing production systems that have limited or no access to weighing facilities (Ndlovu et al., 2007; Wangchuk et al., 2018; Creamer and Horback, 2021).

The objectives of this study were to (1) estimate the body volume of mature beef cows from time-of-flight depth images, (2) quantify the BW and metabolic weight from the estimated volume derived from the depth images and (3) classify cow BCS from image-obtained morphological traits using a machine-learning-based classification approach.

Materials and Methods

Animal Information

All cattle were managed and cared for following the protocol approved by the Institutional Animal Care and Use Committee at the University of Nebraska-Lincoln (IACUC Protocol No. 1787). This study was conducted at the Gudmundsen Sandhills Laboratory near Whitman, NE. A total of 58 Red Angus × Simmental mature cows with red hair coats weighing between 310.7 and 575.6 kg with a mean shrunk BW of 410.0 ± 60.3 kg and between 4 and 6 yr of age, were used for data collection. Cows were imaged during two different data collection dates between May to December 2021. The hair coat condition in both measurement dates had minimal mud and were dry. In May, cows had slicked off hair coats, whereas cows in December had a winter coat. To minimize variation in gut fill, cows were held off water and feed for approximately 16 h for a shrunk BW prior to depth imagery data collection.

Imagery data for this study were collected in a sheltered facility and without direct contact with the animals. Cows were sorted into a holding pen and then proceeded to a hydraulic chute for routine weighing. Color (RGB) and depth images, scale-measured BW, and BCS (1 = emaciated, 9 = obese; Wagner et al., 1988) were collected for individual cow.

Data Acquisition

A low-cost, commercially available time-of-flight (ToF) depth sensor (Azure Kinect DK, Microsoft) was selected and used to collect depth images for its reliability and accuracy under indoor and filtered light conditions. The depth camera was mounted on the ceiling at approximately 3 m from the floor and 1.5 m from the top of the chute area to collect top-view images of individual cattle (Figure 1). A C++ based program was written in the Visual Studio development environment (version 2019, Microsoft) to collect depth images in a resolution of 640 × 576 pixels, a narrow field-of-view (FOV) mode with 75 × 65° field of interest, and a shutter speed of 15 frames/s. RGB images in 4096 × 3072 pixels resolution were concurrently taken. The entire back of the individual cow needed to be captured inside the camera’s depth-view for acceptable image quality desired for data processing (Figure 2). To avoid causing unnecessary stress, cows were allowed to move naturally in the chute until the desired pictures were taken, which lasted between 1 and 5 min. Upon completion of individual cow image collection, cow BW were recorded. Two experienced BCS technicians assessed the cow BCS by visual evaluation and palpation, and recorded the BCS (1 = emaciated, 9 = obese; Wagner et al., 1988) according to standard guidelines to a degree of precision of 0.25. For each data collection day, an additional depth image of the empty alley was taken to calculate the average distance from the camera to the floor.

Figure 1.

The setup of the data acquisition system. Positions of the depth camera and the hydraulic chute are illustrated. The depth camera was fastened at approximately 1.5 m above the walking alley. The average distance between the depth camera and the alley floor was ~2.8 m.

Figure 2.

Top view imagery data (a) color image and (b) depth image for a beef cow in the walking chute before entering the hydraulic weighing scale. The cow in this figure has a measured shrunk body weight of 380.6 kg and a body condition score of 5.75. The contrast of the depth image was adjusted for better visualization.

Data Processing and Analysis

The imagery data was processed and analyzed by a MATLAB-based (version R2022b, The MathWorks, Inc., Natick, MA, USA) program to obtain the desired cow biometrics in pixels (Figure 3). When measuring the cow’s cavity and associated depth profile, the depth camera emits a brief burst of infrared light towards the object (i.e., cows) and precisely measures the time it takes for the light to return to the lens. By analyzing these time measurements, the camera calculates the distance between the camera and various points of the cow’s body (Horaud et al., 2016). Consequently, the camera generates a depth map by capturing the ToF data for multiple points within its FOV, enabling the creation of a three-dimensional representation of the cow, as depicted in Figure 4. The following measurements were acquired from top-view depth images in pixels: (1) estimated morphologic dimensions, including overall body length (from neck to rump), average height , maximum body width, and the dorsal area of the body (without head region) and (2) projected body volume. Dimension’s unit transformation (i.e., pixels to mm) was performed using the relation of field-of-view (FOV) and image resolution vs. distance, obtained using generic equation (1).

Figure 3.

Schematics show the distance measurements using the time-of-flight depth camera and the extracted cow body measurements from depth images. ZA, distance from camera to animal; ZG, distance from camera to the ground; AH, average height; MW, maximum width; L, length from neck to rump; DA, dorsal area.

Figure 4.

An example of projected cow dorsal volume using information extracted from the depth images. The head region of the cow was removed during image analysis for uniform quality control.

$${\displaystyle \begin{array}{l}{\displaystyle l_{\mathrm{m}}=2\times Z\times \tan \left(\frac{\mathrm{F}\mathrm{O}\mathrm{V}}{2}\right)\times l_{\mathrm{p}\mathrm{x}}\times \mathrm{R}\mathrm{e}{\mathrm{s}}^{-1},}\end{array}}$$ (1)

where L_m is the dimension, in mm; Z is the distance from camera to object being measured, in mm; FOV is the camera field of view, in degrees; L_px is the measured dimension, in pixels; and RES is the image resolution, in pixels.

Body weight prediction.

Cow projected body volumes were calculated through the following image processing procedures. The depth image was imported, and values of distance from the sensor to the cow were converted into the cow’s average height (AH) by subtracting the distance between the sensor and the cow (ZA) from the distance between the sensor and the ground (ZG, obtained from the empty alley’s image, Figure 3). Then, pixels within a height range of approximately ± 20% of the approximate height of the cows were selected. Later, extraneous information (e.g., parts of the chute, farm crew) were eliminated, making the values of rows and columns around the cow equal to zero through morphological operations (a combination of erosion and dilation operations). Next, the cow’s body was rotated to a horizontal position in the image. The head and tail regions were then manually eliminated by making their values equal to zero to obtain a better correlation with the BW (Schofield, 1990). The projected volume of the cow excluding the head area (Figure 4) was then determined by summing all the pixels of the resulting body image. For simplification, this projected volume will be referred to as “volume” throughout the manuscript. This “volume” unit was transformed to m³ using a variation of equation (1). The Pearson’s correlation co-efficient between the image-projected cow body volume and their scale-measured BW was performed using the CORR procedure in SAS (v 9.4, SAS Institute Inc., Cary, NC, USA).

The metabolic BW (MBW) is an important factor that provides insight into the heat production and net energy maintenance, thus, affecting the decision-making regarding energy maintenance requirements. Metabolic BW for each cow was calculated using the power adjustment expression (equation 2; NRC, 2016).

$${\displaystyle \begin{array}{l}{\displaystyle \mathrm{M}\mathrm{B}\mathrm{W}=\mathrm{B}{\mathrm{W}}^{0.75},}\end{array}}$$ (2)

where MBW is the power adjusted metabolic bodyweight of the cow, in kg; BW is the scale-measured cow BW, in kg.

Upon completion of all BW and image-extracted biometric information, data from five cows were not retrievable due to the low image quality, and a final dataset containing 53 cows’ data was formed. The dataset was then randomly partitioned into two subsets using 5-fold cross-validation and was stratified by BW classes: 80% for training (N = 43) and 20% for testing (N = 10). A 5-fold cross-validation strategy was deployed to iterate through the training data to reduce overfitting.

Using the training dataset, linear regression models were developed between body volume obtained through image processing and (1) scale-measured weight and (2) calculated metabolic weight. For all cows, data were plotted with each cow’s scale-measured BW as the independent variable on the x-axis, and the image predicted BW as the dependent variable on the y-axis. Correlations between predicted vs. actual measurements were plotted and analyzed for BW and the MBW. The linear regression equation developed using the training dataset was then applied to the testing dataset to obtain predicted BW for 10 cows. The predicted BW using the linear regression equation was plotted against the scale-weighed BW for comparison. For the prediction models using the linear regression curve, performance was evaluated by the mean absolute error (MAE), mean absolute percentage error (MAPE), co-efficient of determination (R²) of the regression, and standard error of the mean (SEM). The above-mentioned descriptive statistics were calculated and tabulated (Table 2).

Table 2.

A summary of the training, testing, and total datasets. Cow information and descriptive statistics for the measured metrics including the body condition score (BCS), scale-measured weight, and metabolic body weight (MBW = BW^0.75) and the predicted metrics, including the predicted weight, co-efficient of determination for predicted weight vs. the measured weight, mean absolute error (MAE), and mean absolute percentage error (MAPE) are calculated

Dataset	Cow information		Measured metrics			Predicted metrics
	No. of cows	Cow age, yr	BCS	Scale-measured shrunk weight, kg	MBW kg	Predicted weight kg	R 2	MAE, kg	MAPE, %
Training	43	4.7 ± 0.7	5.2 ± 0.44	404.2 ± 56.56	90.0 ± 9.38	403.3 ± 52.21	0.8255	19.2 ± 13.50	4.8 ± 3.50
	SEM		0.07	8.64	1.43	7.96	–	2.06	0.53
Testing	10	4.7 ± 0.8	5.2 ± 0.46	434.9 ± 71.92	95.0 ± 11.76	439.2 ± 60.74	0.8661	22.6 ± 13.44	5.5 ± 3.89
	SEM		0.15	22.74	3.72	19.21	–	4.25	1.23
Total	53	4.7 ± 0.8	5.2 ± 0.44	410.0 ± 60.30	90.9 ± 9.95
	SEM		0.06	8.28	1.36

The average value (± standard deviation) and the standard error of the mean (SEM) of each analyzed parameter are also provided.

Body condition score classification.

The BCS is recognized as a discrete categorical variable to be predicted. Due to the poor correlation between BCS and a single input variable (e.g., BW, body volume) reported (Wildman et al., 1982), a machine-learning-based multivariate classification analysis was performed with the dataset to assess the accuracy of predicting BCS using multiple input variables that can be conveniently obtained from the cows and the depth images.

All cows involved in this study were reproductively healthy and had a BCS ranging between 4.25 and 6. A histogram was created for the original BCS measurements using a 0.25 increment, from which the lack of repeatability of each BCS increment was evident (Figure 5a). Particularly, the BCS dataset is extremely skewed for 4.25, 4.5, and 5.75. Such a lack of repeatability within a small dataset creates challenges in using machine-learning algorithms to classify the BCS correctly and accurately. To increase the repeatability and the predictability for each BCS category, the originally assigned BCS were categorized into four categories: 4.25 and 4.5 (A), 4.75 and 5 (B), 5.25 and 5.5 (C), and 5.75 and 6 (D), with each category representing two BCS, and a histogram of the frequency of BCS categories assigned is shown (Figure 5b). Besides category A, all BCS categories have more balanced repeatability for proper classification analysis.

Figure 5.

Histogram plots for all cattle, illustrating the frequency of occurrence of (a) original assigned body condition score (BCS) using a 0.25 increment and (b) newly assigned BCS bin with a 0.5 increment.

These four BCS categories were the predicted variable whereas multiple input variables, including cow age, depth-image predicted cow body volume and BW, average height, average distance to camera, length from neck to rump, maximum width, and dorsal body area of the cow were fed into 29 machine-learning algorithms included in the Categorical Classification Learner tool package in MATLAB (version R2022b, The MathWorks, Inc.). The Categorical Classification Learner categorizes these models into Discriminant Analysis, Classification Ensembles, Kernel, k-nearest neighbor classification (KNN), Naïve Bayes, Neural Network, Support Vector Machine Classification, and Tree models (Table 1). The machine-learning algorithms were first trained using the training dataset with a 5-fold cross-validation and were then tested with the test dataset. The model performance metrics, including the accuracy of the validation and testing (%) and the total cost of validation and test, are provided. In general, a higher accuracy and a lower total cost indicate a more robust model during training and testing.

Table 1.

General information of the 14-machine learning classifiers from the Classification Learner in MATLAB (version 2022b)

Classifier category	Highlights	Subcategory models
Discriminant (Bekios-Calfa et al., 2010)	Generates data based on different “Gaussian” distribution	Linear discriminant
Ensemble (Polikar, 2006;Ganaie et al., 2022 )	Meld results from many weak learners into one high-quality ensemble model	Boosted trees
		Bagged trees
		Subspace discriminant
		Subspace KNN
		RUSBoosted trees
Kernel (Hofmann et al., 2008)	Performs nonlinear classification of data with many observations. Trains and predicts faster than SVM classifiers with Gaussian kernels	SVM Kernel
		Logistic regression kernel
k-nearest neighbor (KNN) (Liao and Vemuri, 2002)	Categorizes query points based on their distance to neighbors in a training dataset to effectively classify new points	Fine, medium, corse KNN
		Cosine KNN
		Cubic KNN
		Weighted KNN
Naïve Bayes (Tuytelaars et al., 2011)	Leverages Bayes theorem and assumes that predictors are conditionally independent, given the class	Kernel Naïve Bayes
Neural network (Sharkawy, 2020)	Optimizes squared errors with backpropagation based on Broyden–Fletcher–Goldfarb–Shanno algorithm or Stochastic Gradient Descent	Narrow, medium, wide neural network
		Bilayered neural network
		Trilayered neural network
Support vector machine (SVM) (Bhavsar and Panchal, 2012; Somvanshi et al., 2016)	Classifies data by finding the best hyperplane that separates data points of one class from those of the other class	Linear, quadratic, cubic SVM
		Fine, medium, corse “Gaussian” SVM
Tree (Loh, 2011)	Develops tree nodes for predicting a target location in specific ranges of target features	Fine, medium, coarse tree

The one-way analysis of variance (ANOVA) feature importance algorithm was selected to perform a feature importance ranking analysis that ranks the importance and contribution of each input variable to the predicted variable. The ANOVA feature ranking algorithm performs one-way ANOVA for the BCS categories, then ranks features (input variables) using the P-values. The feature importance scores correspond to –log(p). A feature with a higher score plays a more important role in predicting BCS, and a score near zero indicates little importance of that feature on the prediction, indicating that its removal may not influence prediction results and will reduce computational complexity (Kalousis et al., 2007). The Shapiro–Wilks test with Bonferroni correction was performed to test the normal distribution hypothesis for all input variables and the predicted variable in this analysis, at an α level = 0.05. The normality statistical test was performed in RStudio (Build 421).

A confusion matrix was plotted from the best-performed classifier to visualize the professionally assigned BCS, and the machine-learning model predicted BCS. The confusion matrix shows two important metrics, the true positive value (TPV) and the false discovery rate (FDR). The TPV indicates the classification precision and is represented as the proportion of correctly classified observations per BCS category predicted; the FDR is an indication of the Type I error rate of the classification and is described as the proportion of incorrectly classified observations per predicted BCS category, calculated using the following equations.

$${\displaystyle \begin{array}{l}{\displaystyle \mathrm{T}\mathrm{P}\mathrm{V}=\frac{\mathrm{N}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{r}\mathrm{u}\mathrm{e}\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}}{\mathrm{N}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r}\mathrm{o}\mathrm{f}\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{s}},}\end{array}}$$ (3)

$${\displaystyle \begin{array}{l}{\displaystyle \mathrm{F}\mathrm{N}\mathrm{R}=\frac{\mathrm{N}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r}\mathrm{o}\mathrm{f}\mathrm{f}\mathrm{a}\mathrm{l}\mathrm{s}\mathrm{e}\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}}{\mathrm{N}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r}\mathrm{o}\mathrm{f}\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{s}},}\end{array}}$$ (4)

where a “true positive” is the event that the model makes a positive prediction, and the subject has a positive result under the correct BCS category, and a “false positive” is the event that the test makes a positive prediction, but the subject was assigned to an incorrect BCS category.

Results

Table 2 provides a summary of the training, testing, and total datasets. For each dataset, the cow information, descriptive statistics for the measured metrics, and the predictive metrics are summarized. The measured metrics include the body condition score (BCS), scale-measured BW, and metabolic body weight (MBW). The predicted metrics include the predicted BW, co-efficient of determination (R²) for predicted weight vs. the measured weight, mean absolute error (MAE) and mean absolute percentage error (MAPE). The average value (± standard deviation) and the SEM of each analyzed parameter were also provided. For the total dataset used in data analysis, the average cow BW was 410.0 ± 60.30 kg, and the assessed BCS ranged from 4.25 to 6, with an observed median score of 5.

Figure 6 shows the relationship between the depth image-predicted body volume and the scale-measured BW (Figure 6a) and the MBW (Figure 6b), acquired using the training dataset. Each point represents an animal. The linear regression trendline between the two variables, the slope, intercept, and the R² of the regression are also presented. The plot demonstrates a strong positive correlation between the image-projected body volume and scale-measured shrunk BW (r = 0.9166). The standard error of regression was 19.20 ± 13.50 and 3.20 ± 2.22 kg, or 4.80% ± 3.50% and 3.50% ± 2.56% for the scale-measured BW and the MBW, respectively. The R² of the two regressed variables was comparable, being 0.8255 and 0.8282 for the scale-measured BW and the MBW, respectively, which is expected as the MBW was calculated using an adjusted power of the BW.

Figure 6.

The relationship between beef cows’ body projected volume obtained from depth images analyzed using the training dataset data (N = 43) and (a) scale-measure shrunk body weight and (b) the metabolic body weight.

The depth image-obtained linear regression equation (Figure 6a, equation 5) was then applied to the testing dataset to calculate the predicted BW, and the predicted values were regressed and plotted against the scale-measured BW (Figure 7).

Figure 7.

Correlation between measured and predicted shrunk body weight using the testing dataset that contains 10 beef cows. The circles represent individual measurements/predictions, the dotted blue line represents the linear regression of the testing dataset, whereas the black dotted line represents a linear regression with R² = 1.0 (y = x). The standard error of the slope and the intercept was 0.11 and 48.11 (kg), respectively.

$${\displaystyle \begin{array}{l}{\displaystyle y=594.88x-49.45,}\end{array}}$$ (5)

where x is the animal dorsal volume projected from the depth images, in m³; y is the predicted cow BW, in kg.

Using the training dataset, the regression between the scale-measured BW and the equation-obtained predicted BW demonstrated that the equation developed using the training dataset was well-validated. There was an increase in the MAE of the predicted BW, 22.7 ± 13.44 kg (or 5.5% ± 3.88%), but a slightly improved R² of 0.8661, when compared to the metrics obtained from the training dataset.

A summary of the classification model performance in predicting cow BCS categories is provided in Table 3. The classifier class, model name, and performance parameters, including the accuracy of the validation and testing (%) and the total cost of validation and test, are provided. The general information on these classifiers is listed in Table 1. The best model is highlighted in italics.

Table 3.

A summary of the classification model performance in predicting cow BCS categories

Classifier class	Model name	Validation accuracy, %	Validation total cost	Test accuracy, %	Test total cost
Discriminant	Linear discriminant	41.86	25	20	8
Ensemble	Boosted trees	44.19	24	40	6
	Bagged trees	58.14	18	70	3
	Subspace discriminant	41.86	25	40	6
	Subspace KNN	41.86	25	50	5
	RUSBoosted trees	37.21	27	40	6
Kernel	SVM kernel	46.51	23	50	5
	Logistic regression kernel	44.19	24	50	5
k-nearest neighbor (KNN)	Fine KNN	44.19	24	20	8
	Medium KNN	32.56	29	40	6
	Coarse KNN	44.19	24	40	6
	Cosine KNN	34.88	28	20	8
	Cubic KNN	37.21	27	30	7
	Weighted KNN	44.19	24	30	7
Naive Bayes	Kernel Naïve Bayes	39.53	26	40	6
Neural network	Narrow neural network	41.86	25	10	9
	Medium neural network	48.84	22	40	6
	Wide neural network	34.88	28	20	8
	Bilayered neural network	37.21	27	50	5
	Trilayered neural network	39.53	26	50	5
Supported vector machine (SVM)	Cubic SVM	41.86	25	40	6
	Fine Gaussian SVM	44.19	24	40	6
	Medium Gaussian SVM	46.51	23	30	7
	Coarse Gaussian SVM	44.19	24	40	6
	Linear SVM	51.16	21	20	8
	Quadratic SVM	44.19	24	30	7
Tree	Fine tree	44.19	24	80	2
	Medium tree	44.19	24	80	2
	Coarse tree	44.19	24	70	3

The classifier class, model name, and performance parameters, including the accuracy of the validation and test (%) and the total cost in validation and test are provided. The general information of these classifiers is listed in Table 1. The best model is highlighted in italic.

Table 3 demonstrates that among all the machine-learning-based classification algorithms evaluated, the SVM and Ensemble classifiers had a higher overall validation accuracy (41.86% to 51.16% and 37.21% to 58.14% for SVMs and Ensembles, respectively), while the Tree classifiers has the highest accuracy in model testing (70% to 80%). Among these three Classifier classes, the Bagged Tree model in the Ensemble class has the highest validation accuracy (58.14%), second-highest testing accuracy (70%), and the lowest total cost for both the validation (18) and the testing (3), and, therefore, was selected as the best classification model to classify BCS. The classification results using the bagged tree model in the Ensemble Classifier were visualized to demonstrate the most relevant features obtained from the cows and extracted from the depth images in predicting BCS (Figure 8) and the prediction performance (Figure 9).

Figure 8.

Feature importance score in predicting the cow body condition score (BCS) category, ranked using the ANOVA algorithm. The BCS categories are defined as following: A [4.25, 4.5], B [4.75, 5], C [5.25, 5.5], and D [5.75, 6].

Figure 9.

A confusion matrix demonstrating the classification results using eight input variables (cow age, predicted body weight, image-extracted cow dorsal length, width, and surface area, average body height, projected volume, and average distance to the camera) to predict the body condition score (BCS) category of the cows. The categories are defined as follows: A [4.25, 4.5], B [4.75, 5], C [5.25, 5.5], and D [5.75, 6]. The true predicted values (TPV) and false discovery rate (FDR) are provided at the bottom.

The results of feature importance scores ranked using the ANOVA algorithm are depicted in Figure 8. The Shapiro–Wilks normality test with Bonferroni correction results indicate that other than the variable “cow age”, all other input and response variables were considered normally distributed. The most relevant features in predicting BCS were the body dorsal area extracted from the depth images, followed by the cow’s age, the projected body volume, and the corresponding predicted weight calculated. Other parameters extracted from the depth images, including the maximum width, length from neck to rump, average height of the cow, and distance from the camera, were considered less important than the first three prominent features. Some less important features (e.g., the average distance from the camera) may be removed from the training dataset due to being less important than other selected features; however, including more features may enhance the model performance.

Figure 9 is a confusion matrix plot illustrating how the selected classifier performed in each actual or predictive class. In Figure 9, a blue cell of the diagonal displays high percentages, indicating good classification performance, and an orange cell indicates poor classification. The higher the percentage, the darker the hue of the cell color. The classification precision is represented by TPV and the FDR, where TPV is shown in blue for the correctly predicted points in each BCS category, and FDR is shown in orange for the incorrectly predicted points in each BCS category. The results indicate that the Bagged Tree model achieved a higher prediction precision for BCS categories B, C, and D compared to category A. Specifically, the model failed to predict any BCS correctly in category A [4.25 and 4.5]. At the same time, it accurately classified the BCS category with a TPV rate of 60%, 63.6%, and 50% for categories B [4.75 and 5], C [5.25 and 5.5], and D [5.75 and 6], respectively. The failure to predict for category A and a slightly lower prediction precision for category D may be explained by the limited incidence of the BCS observations for these two categories, as evidenced by the histograms demonstrated in Figure 7.

DISCUSSION

BW and changes in BW of cows or average daily gain of growing animals are key for livestock producers to monitor performance metrics, make adjustments to nutritional management, and help make overall management decision to achieve certain production goals. Currently, there are only a few techniques available to estimate or measure BW of livestock. The two of which are using scales or indirect approaches through estimates of body part measurements, while the most common method for measuring BW of livestock is using a calibrated scale. Both approaches have challenges for livestock producers. Walking livestock through an alley to weigh them on a scale can imposes some degree of stress on the animal (Lees et al., 2020). In addition, utilizing electronic scales increases labor needs and can be a costly initial investment (Heinrichs et al., 1992). Moreover, motions associated with weighing large animals over the walk-over scale impose fluctuation in measures, thus, eventually leading to inaccurate weights. Indirect estimation using other body measurements can be impractical and time consuming for livestock producers. In extensive beef production systems, even attaining a regular BW measurement as less frequent as a yearly record can be a challenge for many livestock producers (Ndlovu et al., 2007; Wangchuk et al., 2018; Creamer and Horback, 2021).

In addition to challenges to collect BW of grazing livestock, inaccurate weights in both research and producer settings can have a large financial impact due to over or underfeeding and marketing livestock. Obtaining accurate weights can be challenging due to variation in gut fill, technique error, and varied environmental conditions. Kelly et al. (2019) reported differences in BW taken in two consecutive days for growing cattle that range from −18 to 22 kg. Weight of digestive tract contents or differences in intake may be the largest source of error in weighing cattle, which is especially true for livestock grazing or consuming forage-based diets (Koch et al., 1958). Therefore, Watson et al. (2013) recommended limit-feeding a standardized diet that minimizes gut fill prior to weight collection, which would be crucial for accurate calculation of average daily gain of growing animals. However, passive BW estimation or data collection technologies that have the ability to conveniently collect multiple weights over a short period of time may minimize the error in BW measurements due to the number of measurements that can be recorded over a short time period.

Depth sensing or image analysis has been utilized to predict body volume and biometric measurements in pigs (Kashiha et al., 2014; Condotta et al., 2018), Limousin beef cows (Ozkaya et al., 2016), Angus and Nellore bulls’ carcass characteristics (Gomes et al., 2016), and Nellore heifers (Kamchen et al., 2021). Condotta et al. (2018) developed a multilinear regression model using convolutional neural network with different geometric characteristics of a standing pig (length, width, shoulder width, and height) and achieved accurate predictions of pigs’ BW with a standard error of 3.01 kg and a linear regression co-efficient (R²) of 0.99 (N = 772). Ozkaya et al. (2016) utilized an RGB color digital camera and manual acquisition of body measurements of 58 limousin beef cows to predict their BW. Their results demonstrated a linear relationship between body area and bodyweight with an R² of 61.5%, and a higher R² of 88.7% when all body traits were included as input variables. Kamchen et al. (2021) used a stereo depth camera to predict body measurements and BW of 260 Nellore heifers. Their results showed that it is possible to estimate body mass in Nellore heifers using depth-image-estimated body volumes with a correlation of 0.97, a mean absolute error of 8.85 kg, and a mean absolute percentage error of 3.13%. Gomes et al. (2016) studied the first-generation of Kinect I depth cameras on the relationships between BW and hot carcass weight (HCW) for 20 black Angus and 15 Nellore bulls and reported an R² between 0.69 and 0.84 for the black Angus and Nellore bulls, respectively.

Although these previous studies showed that digital/depth cameras demonstrated great potential to be used as a tool to estimate biometric information of livestock animals, they were limited on different animals, breeds, and environmental conditions. Moreover, none of the studies has taken the limited or restricted feeding into account for accurate shrunk BW of the cows. Results from this study support that the newer generation depth camera demonstrates greater potential to be used as a precision tool to accurately estimate shrunk BW of productively mature beef cows, with a strong positive correlation between the image-projected body volume and scale-measured shrunk BW (r = 0.9166). The prediction standard error of the regression between measured shrunk BW and image-model predicted BW was 19.2 ± 13.50 kg, with an R² of 0.83. The difference in standard error of the predicted BW and the R² of the regression between image-predicted and scale-measured BW may be explained by vastly different animal species, type of imaging tools used, total animal used, and data collection environments.

Body condition scoring (BCS) is an effective and critical management tool to estimate the energy reserves of a cow. If monitored multiple times across the production year, BCS is a good indicator of direction of BW change. Traditional recommendations suggest grazing beef cows need to be nutritionally managed at a BCS 5 or greater at breeding for optimal reproductive performance (DeRouen et al., 1994; Lents et al., 2008). Since visual BCS evaluations are subjective, livestock producers need to be trained to accurately estimate BCS, which becomes a challenging task of fine-tuning and training human eyes to see anatomical differences. With that in mind, visual evaluation of cowherd BCS can be subject to error and inconsistency (Kristensen et al., 2006). Even more challenging, visually estimating grazing cowherd BCS in the winter can be deceiving due to gut fill of low-quality forages, winter hair coat, and environmental conditions. Thin cows with increased gut fill of low-quality forages can be mistaken as being in better condition than they truly are. Long winter hair coats can mask prominent ribs or the vertebrae, which are landmarks used in scoring body condition. In addition, estimating BCS in large numbers of cattle can increase the error due to estimations being biased towards thinner or fatter cows in the herd rather than the herd average. Prior to this study, imagining technology and depth cameras in particular have been utilized to accurately estimate BCS in dairy cows (Liu et al., 2020). Kojima et al. (2022) reported a model to estimate BCS of Wagyu beef cattle using depth imagery that yielded an accurate estimation; however, 62% of their cows were classified as a BCS 5 with 13% less than a 5 and 25% classified as a 6 or 7. Similarly in this study, while a skewed BCS dataset was also encountered, a higher testing accuracy of BCS categories of 70% was achieved. However, skewed data population toward the small and large BCS is noted and should be taken into consideration. Therefore, a larger dataset with a diverse population of differing BCS cows is advised to increase the reliability of BCS predictions with depth images and machine-learning techniques.

Limitation and Future Considerations

The utilization of depth cameras in livestock operations presents several limitations, requiring careful consideration for practical application, especially in rangeland settings. Firstly, there is a need for a more user-friendly approach to data collection and analysis. For example, in our validation study, it was recognized that the time spent capturing a high-quality image, ranging from 1 to 5 min, was far from ideal. To address this, the image collection process was modified and improved by recording depth videos and developing an interactive frame selection tool. This advancement allowed for swift data collection while accommodating the natural movement pace of cattle. However, further refinements are still necessary to streamline the data collection process and simplify analysis procedures, ensuring efficiency and ease of use.

The placement of cameras and their resilience to environmental challenges also pose hurdles for practical applications. Livestock operations, particularly beef cattle operations, encompass diverse and often extensive environments. Hence, strategically positioning the cameras to capture accurate and reliable data becomes paramount. The cameras must withstand harsh conditions, including dust, moisture, and potential damage from animals. Thus, ensuring the durability and stability of the camera systems is crucial for successful implementation.

Furthermore, the validation of the technology’s practicality requires a more comprehensive and expansive dataset covering various cattle ages, production phases, and color patterns within different environmental settings. While our study acknowledges the relatively small size of the animal herd used for data collection, active gathering of imagery data from diverse herds, ages, and color patterns is essential. This endeavor will enhance the generalizability and reliability of the depth camera technology across different cattle populations.

Additionally, exploring the potential of data streaming and edge computing holds promise for depth sensing applications in livestock operations. By leveraging real-time or cloud processing and analysis at the edge, the technology architecture can provide immediate feedback and actionable insights for precision livestock management. However, it is essential to recognize the increased effort, labor, and complexity associated with model development, data streaming, synchronization, and imagery data analysis.

To summarize, while our study contributes to the pioneering use of depth images for beef cattle applications, particularly in the United States, it is crucial to acknowledge the significant limitations. Overcoming these limitations necessitates advancements in data collection methods, robust camera placement strategies, data diversity, and the utilization of emerging technologies like data streaming and edge computing. By addressing these challenges and incorporating future advancements, depth camera technology holds great promise for practical application and enhanced management and decision-making in livestock operations.

In conclusion, this study showcases the potential of using imaging techniques and machine-learning approaches to improve the accuracy and precision of estimating and predicting the BW and BCS of beef cattle in rangeland settings. However, to further advance precision beef management, the acquisition of a larger dataset encompassing variations in animal breeds, ages, weights, diverse BCS distributions, and different outdoor conditions and data collection environments is crucial and warrants further exploration.

Conclusions

Depth sensing technology has shown great potential for accurately measuring the BW, metabolic weight, and BCS of U.S. beef cattle, providing cow–calf producers with a new, low-cost approach to managing their herds. The use of a low-cost, commercially available depth sensor for collecting top-view depth images of cattle presents a new opportunity for cow–calf producers to improve their herd management and increase productivity. The results of this study demonstrate a strong positive correlation between image-projected body volume and measured BW, which can be useful for predicting BW and metabolic weight in beef cattle. The Bagged Tree model in the Ensemble class was found to have the best performance for classifying BCS in beef cattle, demonstrating the potential for machine learning algorithms to aid in herd management and decision-making. Validating this approach with a larger dataset that encompasses variations in animal breeds, ages, weights, diverse BCS distributions, and different outdoor conditions and data collection environments should be further explored.

Funding

The United States Department of Agriculture's multistate project W-3012 (Optimizing and Characterizing Sustainable Beef Cattle Production in Forage Based Systems on Western Rangelands) is gratefully acknowledged. This research was supported by 1) the University of Nebraska-Lincoln, Agricultural Research Division’s Hatch Multistate Enhanced Program (W-3012: Optimizing and Characterizing Sustainable Beef Cattle Production in Forage Based Systems on Western Rangelands) through the Nebraska Agricultural Experiment Station and 2) United States Department of Agriculture grant 58-3040-1-013: Precision Livestock Management Systems for Western Rangelands.

Conflict of interest statement

None declared.