Farm Exit among Smallhoder Farmers of Nepal: A Bayesian Logistic Regression Models Approach

Abstract

To leave or not to leave farming? This is a dilemma facing a large number of farm households in a rural agrarian setting of Nepal where nearly two-thirds of the population is small holder farmers. Using the uniquely detailed retrospective panel data collected in 2015 from farming households, we examine the influence of the access to cultivated land holding and land tenure on subsequent farm exit. We address the statistical modeling issue of complete separation by developing a robust Bayesian predictive model to predict the probability of farm exit. We use the Bayesian framework with weakly informative prior to carry out the logistic regression model and compare it with other available binary response models. Our results show that the size of cultivated land has a strong negative influence on farm exit, net of all other controls. Moreover, farmers who rented farmland from others or who rented farmland to others were significantly more likely to exit farming. We estimate that a farm household required at least 5 Katha of land (one sixth of a hectare) per year to stay in farming.

Introduction

The issue of complete separation is common in binary response regression models with correlated features. In general, results with unusually high coefficients or close to perfect accuracy in the prediction are some of the key indicators of complete separation [10]. We attempt to address this issue by screening different binary classification methods to model farm exit probability. The source of data and study area for this report is the South Asian country of Nepal. However, the proposed procedure can be applied on similar data that poses complete separation when modeled with a classical logistic regression method.

Understanding the dynamics of farm is important for making sound decision regarding policy and allocating resources. We investigate potential factors that contribute to farm exit in Nepal and develop statistical models to estimate exit probabilities. The Farm industry in Nepal is yet to be commercialized. In traditional Nepali farming culture, farmers mostly live in a joint family and plant rice, wheat, corn, vegetables, and most likely raise cattle (crop-livestock mixed farming). In Nepal, use of land is an important indicator of farming. Thus, we examine the likelihood of farm exit in Nepal and develop statistical models to address a common issue of complete separation for binary response variables. Next we discuss changes in farming practices, the literature on farm exit, followed by a summary of statistical models.

This paper examines the effects of a number of contributing factors to changes in farm practices, one of which is farm size. A study of U.S. farm exit found that exit rates decline as farm size increases [19]. Similarly, some studies in Canada and Israel also found that farm size, land productivity, and family characteristics influence farm exit probability[16,7,22]. In our study, farm size is defined as a combination of all or any one of the following three categories: 1) How much of their land they cultivated; 2) How much land they rented to others; and 3) How much land they rented from others. Other significant variables examined in the study are described in the data section.

A natural choice of statistical models with binary response variables is the logistic regression model. We began by exploring possible contributing variables to farm exit using a logistic regression model. For the data set in this study, the estimated coefficients of logistic regression are inundated with large coefficients. Also, the prediction was perfectly accurate which indicates the presence of complete separation [1]. One viable alternative to address the issue of complete seperation is to use Bayesian inference such as the Firth’s penalized logistic regression model. Firth’s method uses Jeffreys prior distribution[10], but these have not been set up for reliable computation and are not always clearly interpretable as prior information in a regression context [14]. Moreover, when Firths method was applied on current data it failed to converge. Another alternative for Bayesian inference is the generalized linear model developed with weakly informative priors [13,14]. The latter Bayesian method resulted in stable, interpretable and converging estimates of parameters. The Hamiltonian Markov Chain (HMC) simulation is used with 10000 iterations to develop the final model. More detail about model development procedure and prediction accuracy are described in the Methods and Results sections.

In recent decades, the instability in politics and policy in Nepal has left small farmers with little or no compensation. In addition, out-migration of the working population has created a scarcity of labor in the farming workforce. Consequently, farmers have started renting out their land, and in general, the farming system of owner occupation is shifting towards a rental system. The literature suggest that Norway iis experiencing similar situation[11]. Moreover, Martin et al. studied out-migration in Nang Rong, Thiland and Chitwan Valley, Nepal and assert that agricultural factors are significant determinants of rural out-migration [29]. Rozelle et al. studied northeast China and recommend using rural credit system or encouraging informal credit institutions to increase agricultural production efficiency to keep the labor force primarily on farms and ensuring on-farm income shocks [33].

An overwhelming majority of the Nepali population relies on labor-intensive, subsistence-based farming. However, the labor farming model in this area is changing rapidly. Firstl, the proportion of the population employed in agriculture has declined from 76 percent in 1998 to 67 percent in 2008 [8]. Second, many farm households are transitioning away from labor-intensive farming to a more commercialized farming system with increasing use of farm technology [28,24,2]. For example, tractors are gradually replacing human labor traditionally involved in land preparation. Moreover, farmers are increasingly using chemical fertilizers, pesticides, and other farm implements [6,31]. The fact that the use of technologies —particularly those designed to perform labor-intensive jobs —has the potential to replace labor[31,6] may have critical demographic implications, particularly on migration.

The remainder of this article is unfolded as follows: In section 2 we discuss the data, definition of variables, data visualization and possible similarity of the strata within the data. In section 3, we discuss modeling methods. In this section, we supply a summary of statistical methods used in this paper. In section 4, we discuss the modeling procedure and selected model with prediction accuracy. In section 5, we discuss the conclusion with a possible extension of current research.

Data

We examine household-level data from multiple surveys collected by the Chitwan Valley Family Study (CVFS). The data was collected in 2015 using an innovative household agriculture and remittance history calendar (HARHC) method. This survey collected information on farming and farming practices, farm technology use, and remittances received on an annual basis from 2006 to 2015. This data was a continuation of the 2005 household consumption and agriculture survey. In order to enhance respondents recall the HARHC was pre-edited with important community and household events that were collected in other CVFS surveys. Similarly, to improve the accuracy of the amount of remittances received by the household, detailed information on the migration history (including dates and places of migrations) of each household member was pre-edited in the calendar. This data consists of 3331 households, I including households that moved out of the study area (which we tracked to their destination and interviewed there), new households formed from splits of the original households, and households that moved into the sample neighborhood after 2006. The HARHC data was collected using a face-to-face interview technique with a 99 percent response rate.

Descriptive statistics and other visualizations revealed that there are possibly three major groups of farmers, who quit farming, regarding land use. We observed that a significant majority of farmers who left farming were either renting farmland from othersor farming less than 5 Katha of land. Those who quit farming during the study period either rented farmland from others or planted a total of at most 45 Katha of their land in 10 years (2006 to 2015). Figure 1 represents a schematic diagram of land use. We fit classification models for each of these groups. Three different models reveal that the rate of farm exit is linked to the amount of land used.

Fig. 1:

Schematic diagram of land use distribution across the three major groups of households during the ten-year study period.

Methods

For CVFS data, we explored many binary response models: logistic regression, linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), regression tree (RT), random forest [21] (RF), and Firth penalized regression [10]. However, LDA, QDA, RT, and RF did not depict a significant improvement over logistic regression. With logistic regression, we encountered a typical issue of complete separation. Either the coefficient is high or the predicted probabilities are zero or one. The Firth penalized likelihood method [10] provided reasonable coefficients but failed to converge for group one and three. Firth methods use non-informative prior—providing no information to regularize the posterior distribution of the coefficients—resulting in either an uninterpretable coefficients or in a non-convergent algorithm, which was encountered with the current data. A better choice is to use a weakly informative prior distribution which will provide enough information to control the posterior distribution [14]. Before discussing the Bayesian method, we present the logistic regression model in the context of farm exit data below:

Let y_i denote the farm exit variable {y_i = 0 if HH still farming, and y_i = 1 if HH quit farming} such that y_i ∼ Bin(n_i,π_i), and X_i is the vector of contributing variables of the i^th case. We intend to estimate p(y_i = leaving farming|X_i) = π_i. The use of the logit link function $log\left(\frac{{\pi }_i}{1-{\pi }_i}\right)$ leads to the logistic regression. The logistic regression model for estimating π_i is

$$log\left(\frac{{\pi }_i}{1-{\pi }_i}\right)={\eta }_i$$ (1)

where i = 1···n. The quantity $log\left(\frac{{\pi }_i}{1-{\pi }_i}\right)$, represents the log odds ratio of leaving farming, and η_i = X_iβ is the linear predictor. In this case, the distribution of y given the coefficients β:

$$p\left(y|\beta \right)=\prod _{i=1}^n\left(\begin{array}{c}{n}_i\\ y_i\end{array}\right)\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\left(\frac{e^{{\eta }_i}}{1+e^{{\eta }_i}}\right)}^{y_i}\hspace{0.17em}\hspace{0.17em}{\left(\frac{1}{1+e^{{\eta }_i}}\right)}^{n_i-y_i}$$ (2)

For all three possible groups identified in the data section, we used 75% of the cases as a training set and the rest of the data as a testing set. In practice, the parameters are estimated by using maximum likelihood (MLE) method. However, there are two major issues with the logistic regression model: unusually large estimated coefficients and predicted probabilities are either close zero or one (Complete Separation).

We attempted to address the issue of large coefficients and complete separation by implementing the penalized likelihood method proposed by Firth [10](non-informative prior). Firth’s method is considered to work well with issues of complete separation[17]. However, the Firth’s method revealed only small improvement over logistic regression. In addition, it failed to converge for group 3. Due to complications—reported earlier on Firth’s method—we adopted the weakly informative prior method [14,15]. Our choice of a Bayesian model has three significant advantages over other available methods such as Firth [10], and Raftery [30]. First, we can interpret our prior distribution directly as a constraint on the logistic regression[15]. Second, coefficients are in a reasonable range. Third, the issue of complete separation is asymptotically close to non-existence with reasonably high prediction accuracy. Below we give a brief explanation on the use of Bayesian inference in logistic regression.

Bayes Generalized Linear Model

The Bayesian framework to estimate the posterior probability distribution of the coefficients β (of the regression model) conditional on data y and some predictors X is given by:

$$p\left(\beta |y,x\right)\approx p\left(y|\beta ,x\right)p\left(\beta |x\right)$$ (3)

where p(y|β, x) is the likelihood function which for a given observation y_i is given by:

$$L\left(y_i|{\eta }_i\right)=y_{i}log\left(\frac{e^{{\eta }_i}}{1+e^{{\eta }_i}}\right)+\left(n_i-y_i\right)log\left(\frac{1}{1+e^{[}{\eta }_i]}\right)$$ (4)

p(β|x) 3 is the prior distribution of model coefficients given the model predictors. The choice of prior distribution is student t-distribution with mean 0, degrees of freedom ν , and scale S. ν and S are chosen to provide minimal prior information to constrain the coefficients to lie in a reasonable range [15]. A full Bayesian computation for the model in 3 using the usual Markov Chain Monte Carlo (MCMC) methods. In this study we used the RStan package in R-software which is powered by the Hamiltonian Monte Carlo (HMC) algorithm[27]. The HMC algorithm used in Stan was amended by Hoffman and Gelman in 2014 [18] by the so-called No-U-turn Sampler(NUTS). NUTS avoids the tuning of step’s length needed for proper sampling in HMC.

Results

Variable Selection

We studied the correlation between available numerical variables, seven of which are: total number of years of Mustard planting (Sy_Mus), total number of years of rice planting (Sy_Rice), total number of years of Maize planting (Sy_Maize), total number of years of Wheat planting (Sy_Wheat), total number of years of Red Lentil planting (Sy_Red), total number of years of vegetables planting (Sy_Veg), and total money collected from selling vegetables (TM_Veg). They showed a strong positive correlation. The correlation possibly is a result from the fact that these are the most common crops planted by farmers in the study area. Also, the production of rice (P_Rice), production of Maize (P_Maize), and total minutes of tractor use—aggregated over the ten years —are correlated with overall land cultivated by owners O_L_U. Another two highly correlated variables are production of Mustard and the production of Red Lentils. Planting Mustard and Red Lentils together on the same land at the same time is a common practice among farmers in the study area. Because of the strong correlation we are going to use either the production of Mustard (P_Must) or production of Red Lentils (P_Red) in the model.

Farmers usually plant Maize in two different seasons (Summer and Spring). Rice is mainly planted in the summer season. However, a large number of farmers currently also plant Rice during the off-season (Spring) along with Maize. This explains the moderate correlation between the production of Maize and that of Rice in the data. Whereas the production of Wheat ((P_Wheat)) and that of maize (P_Maize) show a very low correlation. One of the reasons for this low correlation is that the land used for wheat production and Maize are different. Wheat is mainly planted in “Khet” (wet area of the available land) whereas Maize is planted in “Bari” ( a relatively drier part of the land).

Chitwan valley falls in the Terai (flat land) region of Nepal where there are sufficient irrigation facilities to plant Rice. This explains a robust positive correlation between the production of Rice and overall land used. In general, a drier land with less irrigation facility (“Bari”) is used for Maize and Mustard while land with irrigation facilities is used for Wheat and Rice production. Another interesting is the correlation between overall land use and total land rented from others. This indicates that a person currently involved in the farming is likely to rent more land from others. In recent years, traditional farmers in the plain (“tarai”) region are renting out their land.

In addition to examining correlation, we used Akaike information criterion(AIC) as a pre —screening tool for the variable selection procedure. First, the logistic regression model is used with backward elimination and forward selection criterion to find likely indicators for farm exit. The variables selected from these prescreening methods are taken into consideration while building the final model. The forward selection method with three contributing variables and one-way interaction resulted with better fit and prediction accuracy. In the appendix, tables 2 and 3 presents the estimated coefficients together with other relevant statistics. Also, figure 4 presents a set of selected variables from the backward elimination method together with their importance.

All the later pre —screening procedures resulted in three contributing variables, which are the total amount of land cultivated by owners (O_L_U), total amount of land cultivated by renting out to others (O_L_R), and total amount of land cultivated by renting from others (O_L_Rin).

Risk of farm exit

In this section, we discuss the procedure to predict farm exit probabilities using the weakly informative prior method. We used the R statistical software package (Rstan). Beside the family of t-distributions, the Rstan package offers two additional choices of prior namely: Normal and Cauchy distributions. Our analysis resulted in selecting the normal distribution with a mean of 0 and scale=2.0 for the intercepts’ prior distribution. Whereas a normal distribution with a mean of 0 and scale=1.0 for the coefficients’ prior distribution.

Table 1 shows a list of the estimated coefficients and corresponding 90% posterior prediction intervals. The numbers in parenthesis indicate exponentiated values of the estimated coefficients and represent the odds of leaving farming while keeping the rest of the variables constant. Several of the quantitative chosen explanatory variables possessed skewed distributions with high variability. As a result, we scaled all quantitative explanatory variables by subtracting their respective mean and dividing the difference by the corresponding standard deviations. The interpretation of the coefficients is performed in a way similar to the regular logistic regression. We focus on the interpretation by taking the coefficients resulting from the group 3 model since it contains all the available data in this study.

Table 1:

Estimated coefficients and respective 90% posterior prediction intervals for three groups of data. The numbers in the parenthesis are the exponentiated coefficients and represent odds of leaving farming.

		coefficients	10%PCI	90%PCI
Group1	Intercept	−3:129	−3:712	−2:581
	Overall Land Used	−6:623(0:001)	−7:518(0:0005)	−5:784(0:003)
	Overall Land Rented to Others	1:178(3:24)	0:715(2:04)	1:701(5:47)
	Overall Land Rented from Others	0:421(1:52)	0:032(1:03)	1:006(2:73)
Group2	Intercept	−3:358	−3:799	−2:947
	Overall Land Used	−4:611(0:009)	−5:183(0:005)	−4:088(0:016)
	Overall Land Rented to Others	1:169(3:218)	0:838(2:31)	1:522(4:58)
	Overall Land Rented from Other	0:545(1:72)	0:027(1:02)	1:077(2:93)
Group3	Intercept	−7:478	−8:046	−6:939
	Overall Land Used	−8:324(0:0002)	−9:122(0:0001)	−7:560(0:0005)
	Overall Land Rented to Others	0:779(2:1)	0:650(1:9)	0:914(2:49)
	Overall Land Rented from Other	1:041(2:83)	0:299(1:34)	1:732(5:65)

We interpret the results of table 1 by explaining the effect of each variable on the probability of farm exit while other variables are held constant. The odds of farm exit associated with land use (O_L_U) is 1/1000, which means for each one unit increase in land used (O_L_U) (approximately 11.5 Katha per year) the risk of farm exit decreases while keeping the rest of the variables constant. The lower and upper limits of the 80% prediction interval for the parameter of overall land used are below 0. The odds of farm exit associated with O_L_R is 2 to 1, which means for each one unit increase in land rented out (approximately 8.8 Katha per year) the probability of farm exit is almost doubled , when keeping the rest of the variables constant. Finally, concerning O_L_Rin , the odds of farm exit is expected to be 2.8 to 1 for HH that depend on farming by renting from others. These HH are at higher risk of farm exit than those who rent out their land. The interpretations of coefficients of group 1 and 2 are similar to those of group 3.

Figure 2 shows the estimated probability of farm exit when holding the rest of the variables at the median level. Based on the model, the farm exit rate is inversely proportional to overall land used whereas it is directly proportional to overall land rented from others and land rented to others. A household that farms more than 150 Katha in 10 years (15 Katha per year) of their land is likely to stay in farming. Renting land to other farmers will likely eventually lead the landowner to quit farming. The risk of leaving farming is likely to increase somewhat exponentially if a household rents (their land) more than 10 Katha per year. Renting farmland from others is also not likely to motivate farmers to continue farming. A household renting more than 10 Katha per year of farmland from others are more likely to quit farming compared to those who rent less than 10 Katha per year.

Fig. 2:

Prediction of farm exit probability for overall land used, overall land rented to others, and overall land rented from others (in 10 years) while keeping the other two variables at median levels.

One aspect of checking the fit of the Bayesian model is to visualize the posterior predictive distribution. If the posterior predictive distribution and observed data distributions are similar, then the model is a good fit. We performed a posterior predictive check [15] on the selected Bayesian models (see Figure 3, for model 3). The figure displays the posterior predictive distribution of 100 simulated data (grey lines) and the actual farm exit distribution(bold line). We observed that both distributions (posterior and observed) have similar distributions; thus the selected model adequately fits the data. In addition, the model predictive accuracy on the testing set is 92.9 percent.

Fig. 3:

Kernel density curves of 100 simulated data sets from the posterior predictive distribution (grey lines) compared to the actual distribution of farm exit(bold line)

Discussion

In the present study, we investigated examples of how to mitigate the issue of large coefficients and complete separation upon the use of logistic regression to model binary response variables. Besides the alternatives mentioned in this study, we explored others such as ridge and LASSO regression beyond the traditional logistic regression. The latter methods focus on penalizing the coefficients and lack reasonable interpretation. However, when applied on the CVFS data it resulted to poor models. Which agrees with the results of multiple simulation studies [14,17] that reveal the superiority of Bayesian methods over other penalized procedures (such as ridge and LASSO regression).

Our study showed that within the available Bayesian methods the use of a weakly informative prior is better than a vague prior (Firth’s method). However, the weakly informative prior lacked a proper method to choose the scale parameter for the prior distribution of the coefficients. The model presented in this study could possibly be improved by injecting the concept of hyperparameters. In this case, the knowledge on the coefficient (β) can be expressed through hyperparameters α, so that its prior distribution would become p(β|x, α) instead of the one currently being used p(β|x).

In addition, current study examines the factors that influence exit from farming —a fundamental societal transformation. We explore this relationship in rural Nepal, a society historically characterized by subsistence agriculture (until recently) that is now experiencing dramatic social and economic reorganization and massive international out- migration. These changes have had a substantial influence on the reorganization of local peoples social lives and economic activities.

Overall, the results of this study reflects the importance of land size cultivation to reduce the risk of farm exit. In the study area (Chitwan valley in Nepal), the model indicates more land used decreases the risk. However, the effect is reversed if farmers are only renting land from others to farm. Some studies discuss additional variables that influence the risk of exiting farming, such as land value and population size, but these variables were unavailable in our data, which have an effect on the risk of quit farming [22,11].

Goetz et al suggest that farmers quit farming if the value of their land is very high, they live closer to a metropolitan area or if they reside in counties with high population densities. Interestingly, the properties of the study site in Chitwan, Nepal match those mentioned by Goetz [16]. Chitwan is reasonably close to a big metropolitan city(Bharatpur), it has higher land values, and at the same time, it is a preferred destination of migrants from the mountains of Nepal and surrounding areas [5,4]. But due to the unavailability of this information in our data we couldn’t measure the effect numerically.

Kimhi and Bollman [22] compare determinants of farm exits in Canada and Israel using data on individual family farms over time and found that the odds of exiting decrease with off-farm work. Interestingly, this study found that farm size decreases exit probability in Canada (same as in Nepal) but increases it in Israel [22]. Gunnar et al. [7] investigated farm exit rates in 110 regions of Western Europe and found that exit rates are strongly influenced by farm structure, land productivity, farm size and family characteristics. Additionally, off-farm income was a significantly influential variables in the model.

In summary, our examination of CVFS data shows that farmsize significantly influences farm exit. Renting from others, in combination with the total area of land used (at least 5 Katha per year) for farming also signicantly impacts exit farming. Farmers in the study area (Chitwan) are more likely to remain in their profession if they farm their own land. The likelihood of leaving farming significantly increase when farmers rent out their farmland to other farmers. Besides, renting from others more land has a higher effect in quit farming than renting out land to others.

Appendix

Table 2:

Estimated coefficients and Akakai Information Criterion(AIC) values using logistic regression model.

	Intercepts	Overall Land Used	Overall Land Rented to Others	Overall Land Used : Overall Land Rented from Others	Overall Land Rented to Others	AIC
Model 1	−20.70*	−35.26*	10.10*	−1.19	0.48	124.14
Model 2	−4.48*	−6.06*	1.49*	0.89*	1.53	382.51
Model 3	−53.9*	−67.70*	3.38	−34.97	1.19	388.52

The asterisk(*) indicates statistical significance at 5%.

Table 3:

Prediction accuracy with sensitivity and specificity of logistic regression model

	Prediction Accuracy	95% CI	Sensitivity	Specificity
Model 1	0.977	(0.934, 0.995)	0.986	0.965
Model 2	0.91	(0.874, 0.951)	0.929	0.890
Model 3	0.96	(0.95, 0.97)	0.977	0.859

Fig. 4:

A variable importance plot for farm exit data. Variable importance is computed using a test statistic of corresponding coefficients, and expressed in decreasing order of their magnitude.

Table 4:

Descriptive statistics (in Katha) of major contributing variables to exit from agriculture

	Group 1			Group 2			Group 3
	Overall Land Used	Overall Land Rented to Others	Overall Rented from Others	Overall Land Used	Overall Land Rented to Others	Overall Rented from Others	Overall Land Used	Overall Land Rented to Others	Overall Rented from Others
Minimum	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
1st Quartile	0.00	0.00	0.00	0.50	0.00	0.00	24.00	0.00	0.00
Median	2.50	2.00	0.00	10.00	0.00	0.00	74.00	0.00	0.00
3rd Quartile	14.00	50.00	0.00	21.00	36.00	4.00	145.00	10.30	41.00
Maximum	45.00	1624.00	90.00	45.00	1624.00	90.00	1160.00	1624.00	1000.00
SD	11.07	140.64	5.58	11.52	114.75	8.75	115.03	87.66	75.11
Mean	8.18	57.94	1.03	12.44	42.95	4.15	104.51	27.98	35.82