On the application of mixed hidden Markov models to multiple behavioural time series

Abstract

Analysing behavioural sequences and quantifying the likelihood of occurrences of different behaviours is a difficult task as motivational states are not observable. Furthermore, it is ecologically highly relevant and yet more complicated to scale an appropriate model for one individual up to the population level. In this manuscript (mixed) hidden Markov models (HMMs) are used to model the feeding behaviour of 54 subadult grey mouse lemurs (Microcebus murinus), small nocturnal primates endemic to Madagascar that forage solitarily. Our primary aim is to introduce ecologists and other users to various HMM methods, many of which have been developed only recently, and which in this form have not previously been synthesized in the ecological literature. Our specific application of mixed HMMs aims at gaining a better understanding of mouse lemur behaviour, in particular concerning sex-specific differences. The model we consider incorporates random effects for accommodating heterogeneity across animals, i.e. accounts for different personalities of the animals. Additional subject- and time-specific covariates in the model describe the influence of sex, body mass and time of night.

1. Introduction

When analysing how selection has shaped behaviours we observe today, it is usually assumed that an animal's decisions approximate an optimal solution based on the amount of information available to the individual [1]. Individuals are believed to balance trade-offs based on profitability and availability, and behavioural decisions of past generations are assumed to have been selected for a maximal contribution to the phenotypic fitness of the animals, so that current decisions can therefore be regarded as adaptations [1–4]. But each member of a species is distinct from its conspecifics. Some of these differences between individuals may be temporary and affect state variables, e.g. hunger, thirst, fear, whereas others concern fixed, long-term or slowly changing individual parameters, e.g. size, sex, age, degree of maturity, reproductive states and personality traits [5]. The attribution of factors into these classes might differ between studies since their persistence/continuation might also depend on the time scale of a study [6]. Nevertheless, both types influence the so-called ‘motivational state’. This term can be defined as the motivation of an individual generated by physiological and perceptual states [7]. The motivational state influences the likelihood of an occurrence of behaviour.

Determining the motivational state of free-ranging animals is a complex task. It is generally accepted that individuals have mechanisms to monitor their internal state [7]. The problem for behavioural ecologists is that the current motivational state of an individual includes many hidden aspects like physiological states (e.g. hormone and metabolite levels or protein and lipid stores), but also externally based motivational aspects (e.g. perceived predation risk) [8–10]. Estimates of the motivational state could be derived from physiological measures like hormone profiles [8], or from behaviours that are specific for a certain context, e.g. courtship behaviour [11]. But most often, motivational states or state changes remain a black box. The link between the typically unobserved motivational state and the actually observed behaviour is often not one-to-one; e.g. a hungry individual might have problems finding appropriate food, or it might be distracted by a perceived high predation risk, and thus not feed [7]. Some early attempts to model behavioural sequences—before the models we consider here have been developed—used Markov chains to explain the observed behaviour, thus not explicitly modelling the motivational component [12]. Transitions between internal states, such as moving/pausing or hungry/satiated, can typically only be inferred post-hoc, e.g. through gaps between feeding bouts. Thereby information about the actual behavioural process is lost because data are often simplified and/or converted to proportions, and because patterns are evaluated using statistical tests only. Moreover, the occurrence of motivational states for a single animal will not be independent over time [5], a fact that is often implicitly accepted.

Models integrating a link between motivational state and behaviour are relatively sparse. One of their requirements is that the model includes a probabilistic relationship between the action chosen and the animal's state [5]. Dependent mixture models such as hidden Markov models (HMMs) incorporate the presence of these underlying motivational states, as well as their autocorrelation, and facilitate their inference [13–17]. The different components of the mixture can conveniently be interpreted as being associated with the different motivational states of the animal. HMMs are relatively simple stochastic models that nevertheless exhibit immense flexibility; besides ecology they have proved useful in fields such as speech recognition [18] (for which purpose they were originally developed), finance [19,20], economics [21], biology in general [22,23], computer vision [24] and climatology [25]. Besides many other convenient features—such as the straightforward treatment of missing data—HMMs also facilitate the inference about underlying motivational states, enabling us to predict the most likely motivational state sequence [13], a feature that is not exploited in the current analysis, however. Classical likelihood-based inference for HMMs is convenient and efficient, it is thus not necessary—albeit possible [26]—to apply Bayesian methods, which despite growing popularity in the ecological literature [27,28] are presumably less accessible to practitioners in the case of HMMs. In a Bayesian approach, it is in particular difficult to estimate the number of states of an HMM, and the issue of the so-called label switching needs to be addressed [26].

HMMs have precisely the same dependence structure as state-space models (SSMs); the former assume a finite number of states, while the state space in the latter may be infinite. In recent years, SSMs have become increasingly popular tools for modelling animal behaviour, in particular animal movement [29–33]. However, the likelihood of SSMs with infinite state space involves a multiple integral that, in general, cannot be evaluated directly. In particular, nonlinear and non-Gaussian SSMs, to which the Kalman filter is not directly applicable, are rather difficult to fit. The literature offers a variety of possible methods for estimating the parameters of such models [34–37]. Given the difficulties involved in fitting SSMs, it sometimes may be more convenient to resort to the less flexible special case HMMs, if appropriate [17]. In some applications, the assumption of a finite number of (motivational) states can be perfectly reasonable. However, observation errors, e.g. caused by measurement inaccuracies in animal movement paths, can more easily be accommodated in the SSM framework [29].

Scaling individual models up to the population level is an issue of great ecological relevance. There are several different ways in which basic HMMs can be extended to deal with multiple time series (see the discussion in §2.3 below). In this paper, we follow suggestions of Zucchini et al. [13] and propose a model that incorporates both subject-specific covariates and random effects, combining—to some extent—the benefits of both approaches, which in this form to the best of our knowledge has not been done before in the ecological literature. (Zucchini et al. [13] incorporate one random effect, but no covariates in their model; ecological applications of HMMs that involve covariates, but no random effects, are given, for example, in Patterson et al. [17] and Morales et al. [38], although in the latter case, the model is not explicitly referred to as an HMM.) Our model belongs to the class of mixed HMMs [39].

The primary aim of this paper is to provide ecologists and other practitioners with a comprehensible introduction to mixed HMMs, and to discuss their potential in statistical ecology, particularly concerning analyses of multiple series. The explanations of the basic ideas and the associated methodology are given for one specific application, rather than in a more general manner. We chose this strategy for the presentation as the given application provides a very convenient means of introducing and illustrating the methods, and as for HMMs it is usually straightforward to transfer the basic ideas to other applications. On the other hand, the application given here is interesting in its own right, and we thus describe it in much detail. More specifically, we use a mixed HMM to model the feeding behaviour of a population of subadult (less than 1 year) grey mouse lemurs. The grey mouse lemur (Microcebus murinus) is a small (60 g), monomorphic, nocturnal, solitary primate and can be found from the dry deciduous forests in western and north-western Madagascar to the evergreen littoral rain forests and spiny forests in the south of the island [40–42]. They feed opportunistically on insects, small vertebrates, fruits, gum and insect secretions, and the composition of their diet varies with season [43–45]. In contrast to adult individuals, most of the subadult individuals do not engage in longer phases of inactivity, but remain active during the dry season characterizing western Madagascar [46]. Another distinctive feature of subadults from the male perspective is that subadult males separate from their families to disperse [47–49]. The solitary lifestyle of mouse lemurs (individual foraging decisions are not dependent on conspecifics as in group-living species), as well as general differences in the life-history strategies between males and females, makes them a good case for evaluating a mixed HMM for behavioural sequences.

2. Material and methods

2.1. Study site, animals and data description

Behavioural data were collected for subadult grey mouse lemurs of a study population situated within a 12 500 ha forestry concession of the Centre National de Formation, d'Etude et de Recherche en Environnement et Foresterie (CNFEREF) de Morondava in Kirindy Forest, 60 km northeast of Morondava in western Madagascar (44°39′ E, 20°03′ S [50]). The region is characterized by pronounced seasonality with a single rainy season lasting from December to March. The study took place in a 60 ha area, locally known as CS7 (for details see Eberle & Kappeler [47]). In this area, we captured subadult M. murinus and equipped them with radio collars for radio tracking (Holohil Systems Ltd. BD-2C; 1.8 g).

We collected behavioural data for 16 subadult females and 38 subadult males between 2008 and 2010. Sampling periods lasted from April to May in 2008, May to October in 2009 and May to July in 2010. Four individuals were observed per night over 40 min periods between 18.00 and 24.00 h in randomly changing combinations and order. Feeding data of focal animals were collected cumulatively for observation intervals of 1 min [51]. When the individual was not visible, these minutes were recorded as missing data (NA). About 500 h of focal observations were included in the present analyses. The numbers of available time series differ between focal animals because of predation events, non-functioning radio collars and differences in the length of total observation periods per year. Therefore, the dataset was heterogeneous. Numbers of available time series of feeding behaviour per individual ranged between 1 to 26 (mean = 9). Body mass of individuals ranged between 33 to 59 g (mean = 48 g) with a mean body mass of 49 g for females and 47 g for males.

2.2. Stochastic model for a single animal

For illustration purposes, we start by considering a relatively simple HMM for a single animal before we move on to the more challenging population models in §2.3. Figure 1 represents the (dependence) structure of a basic HMM. The state process {S_t} can not be observed (it is hidden). In our application, it can be interpreted as generating the motivational states of the observed animal; S_t then is associated with the motivational state of the animal at time t. We model {S_t} by a Markov chain, in particular assuming that the distribution of S_t is completely determined by the motivational state the animal is in at time t − 1:

2.1

Figure 1.

Dependence structure of an HMM.

The Markov chain thus is of first order. We fit models that involve two different motivational states. In the context of feeding behaviour, it seems convenient to label the two states by ‘satiated’ (state 1) and ‘hungry’ (state 2), respectively, though they do not necessarily correspond to the accepted meanings of those terms. Irrespective of how the motivational states are defined, most importantly, their delineation will provide us with an objective measure of the general motivational state, allowing us to explore what factors influence the transitions between activities (feeding/non-feeding) of the observed individuals. If the animal is in state i at time t, the probability of it being in state j at time t + 1 is:

(Example: is the probability that the animal will be hungry at time t + 1, given that it is satiated at time t.) As there is no a priori reason to assume that the occurrences of motivational states are homogeneously distributed over the night, we model the transition probabilities between motivational states as a function of time:

The logit link ensures that γ_ii^(t) is in [0,1]. It is straightforward to make this generalized linear model for γ_ii^(t) more flexible by considering quadratic or even cubic functions of the covariate t. However, for simplicity, and as we are primarily interested in whether there is any trend at all, we used a simple linear predictor here. From state i the process can only switch to state j or remain in state i, and so γ_ij^(t) = 1 − γ_ii^(t) for j ≠ i. Each integer time t refers to 1 min. At time t = 0—corresponding to 18.00 h in our application—the state is selected by an initial distribution δ = (ℙ(S_t = 1), ℙ(S_t = 2)).

The non-observable motivational states determine the distributions associated with the observed behaviour. We observe the behaviour X_t, where X_t = 0 if the animal does not feed at time t, and X_t = 1 if the animal does feed at time t. The model assumes that, given the motivational state at time t, the distribution of the behaviour X_t is independent of all previous states and observations. More precisely, we assume X_t to follow a Bernoulli distribution (i.e. a binomial distribution of size n = 1), where the parameter is driven by the (motivational) state the animal is in at time t:

As π₂ is associated with the animal being in the ‘hungry’ state, it will typically be relatively large, while π₁ (feeding probability when satiated) can be expected to be close to 0.

There are in total seven parameters to be estimated (one for the initial distribution, four for the transition probabilities and two for the state-dependent process). The model fitting exercise is usually carried out using numerical maximization of the likelihood function, which is given by a closed-form matrix product:

2.2

Here 1 ∈ ℝ² is a row vector of ones, T denotes the number of observations, Γ^(t) is the transition probability matrix at time t,

and ; for missing observations, this is replaced by the 2 × 2-identity matrix. For more details on the derivation and the numerical maximization of an HMM likelihood, we refer to chapters 2 and 3 in Zucchini & MacDonald [25]. Alternatively, one can apply the expectation-maximization (EM) algorithm [52]. In §2.3, we extend this basic HMM to capture the heterogeneity of multiple time series, associated with a population of grey mouse lemurs.

2.3. Stochastic model for a population of individuals

The class of HMMs provides several different strategies for dealing with populations of time series. For instance, one might impose the very restrictive assumption that the parameter set is common to all subjects. This strategy neglects any possible heterogeneity across subjects: two individuals, regardless of their sex, age, mass, personality, etc., would be assumed to act according to the same (stochastic) principles. Another extreme strategy assumes that each of the parameters is subject-specific, i.e. that each subject has its own set of parameters. This approach involves a significantly larger number of parameters and generally ad hoc comparisons between individuals. In between these two extreme options lies the compromise of assuming that some parameters—e.g. those determining the state-dependent process {X_t}—are common to all subjects, while the others are subject-specific. An important special case of the latter is to assume that the subject-specific parameters—the random effects—are drawn from a common distribution. This approach substantially reduces the number of parameters to be estimated. HMMs incorporating random effects were considered, for example by [13,53,54]. Random effects can be understood as explaining the individuality (or personality) of the different animals. Unfortunately, their implementation is very demanding in terms of computational effort [39]. A computationally less expensive way of accounting for possible heterogeneity across subjects is to incorporate subject-specific covariates in the model [55–57]. Such a model may explain heterogeneity across subjects, but it requires that suitable covariates are available. In the given application, we will consider sex and body mass of individuals, covariates that may help to explain individual differences, but only to a limited extent. We will thus also incorporate random effects in the model, in this way combining the benefits of both approaches. More precisely, for animal m, m = 1, … ,M, we assume the motivational state-transition probabilities at time t to be determined by:

2.3

(i = 1,2, t = 0,1,2, …), where sex^(m) is 1 if the mth individual is a male (and 0 otherwise) and where mass^(m) denotes the body mass of the mth individual in grams. In comparison to the HMM for a single animal, we have additionally included the subject-specific covariates ‘sex’ and ‘body mass’. The former divides the population into two groups—females and males—while the latter takes account of possible heterogeneity across individuals of different body mass. If that was the only extension of the basic HMM considered above, the model would still have one crucial limitation: it would not allow for different individualities or personalities of the animals. Indeed, it would assume that animals of the same sex and the same body mass act according to exactly the same stochastic principles. As this appears to be unrealistic and too restrictive, we further increase the flexibility of the model by incorporating random effects. To be specific, we assume that the parameters of the state-dependent process, π₁ and π₂, are not fixed across subjects, but that each of them is randomly distributed on the interval [0,1], independently and identically across subjects:

2.4

for i = 1 and i = 2, respectively, where ℬ(a_i, b_i) denotes a beta distribution with shape parameters a_i > 0 and b_i > 0, and π_i,m denotes the probability of feeding, given motivational state i, for the mth individual. Note that we can, in principle, also model correlation between the random effects, e.g. by using a bivariate Gaussian distribution for the vector of the logit–transformed parameters π₁ and π₂; for the sake of simplicity and readability we did, however, not attempt this in the current work.

Our model belongs to the flexible class of the so-called mixed HMMs [39]. The inclusion of random effects offers an elegant and plausible way for modelling ‘personality’—in a broad sense–of individuals. On the other hand, the presence of random effects unfortunately renders the evaluation and maximization of the likelihood very challenging: in our case with two random effects, the likelihood function involves a twofold integral:

2.5

Here x_t,m denotes the observation made at time t for lemur m and f_i denotes the probability density function of the ℬ(a_i, b_i)-distribution. The other ingredients are defined analogously as in equation (2.2); in particular, Γ^(t,m) is the matrix comprising the state-transition probabilities at time t for the mth individual. For simplicity, the likelihood here is given for the case with only one time series associated with each lemur; indeed, we have more than that. As the different series were recorded on different days, they can reasonably be assumed to be independent, and thus the corresponding extension of the formula given in equation (2.5) is straightforward; see also Altman [39]. Owing to the multiple integral, this likelihood cannot be evaluated directly. We applied numerical integration, i.e. we approximated each of the two integrals by a sum based on partitioning the integration interval into a number of bins and then approximating the integrand within each bin; see the appendix for more details on the type of approximation we applied. Maximization of the likelihood was carried out using nlm() in R. Numerical integration is computationally expensive, and as the computational burden increases exponentially with the number of random effects, this method can only be applied when there are few random effects. A more sophisticated alternative, which is, however, less accessible to practitioners, is given by Monte Carlo EM methods. For a comprehensive discussion of the existing approaches for estimating HMMs that incorporate random effects, see [39]. A computationally less intensive alternative uses discrete distributions for the random effects [58].

3. Results

The parameter estimates for the mixed HMM, defined by equations (2.3) and (2.4), and associated 95% confidence intervals (CIs)¹ are given in table 1. For the sake of better interpretability, each of the (beta) random effects' distributions, ℬ(a_i, b_i), i = 1,2, has been reparameterized in terms of a mean (μ_i) and a standard deviation (σ_i) parameter².

Table 1.

Estimated parameters with 95% CIs for the mixed HMM.

parameter	associated with	estimate	CI
δ1		0.869	(0.820, 0.906)
β0,1		2.177	(1.348, 3.005)
β0,2		1.961	(1.055, 2.867)
β1,1	time	0.0041	(0.0031, 0.0051)
β1,2	time	0.0003	(−0.0007, 0.0013)
β2,1	sex	−0.402	(−0.603, −0.202)
β2,2	sex	−0.317	(−0.538, −0.097)
β3,1	body mass	0.0139	(−0.0023, 0.0302)
β3,2	body mass	−0.0065	(−0.0240, 0.0111)
μ1	random effect π1	0.015	(0.013, 0.017)
σ1	random effect π1	0.004	(0.001, 0.020)
μ2	random effect π2	0.925	(0.892, 0.949)
σ2	random effect π2	0.058	(0.030, 0.112)

For each of the covariates—‘time’, ‘mass’ and ‘sex’—we conducted a likelihood ratio test of the simplified model (without the respective covariate) against the full model as defined by equations (2.3) and (2.4). At a 5 per cent significance level, the simplified models were rejected in favour of the full model for the covariates ‘time’ and ‘sex’, respectively, while the simplified model without covariate ‘mass’ could not be rejected; the p-values are <0.001, <0.001 and 0.127, for ‘time’, ‘sex’ and ‘mass’, respectively.

To gain some insight into the goodness-of-fit of the model, we conducted the following simple predictive check: first, for each of the 54 different covariate combinations (corresponding to the 54 different mouse lemurs observed), we simulated series from the fitted model with exactly the same lengths and placements in time as the corresponding observed ones. Figure 2 displays histograms of the subject-specific ratios ‘number of feeding events/number of events in total’ (i.e. the proportion of observations that correspond to ‘feeding’), for the observed data and for one typical set of simulated series (repetitions did not indicate any significant mismatch). This check suggests that the model captures the observed variability in these proportions reasonably adequately, but note that this covers only one arbitrarily chosen aspect of the data—the meaningfulness thus is limited.

Figure 2.

Predictive check: histograms of the ratios ‘number of feeding events/number of events in total’, for the (a) observed and (b) simulated data.

The various aspects concerning the (motivational) state process are illustrated in figure 3, which displays the transition probabilities in dependence of the covariates ‘mass’, ‘sex’ and ‘time of night’. In the following, we list predictions for the feeding behaviour made by the fitted model. Considering the influence of the time of night, mouse lemurs are more likely to switch between the ‘satiated’ and the ‘hungry’ motivational state at the beginning of the night than towards the end of the observation period. The behaviour of female grey mouse lemurs is more persistent as reflected by both (stochastically) longer feeding and non-feeding periods, whereas male mouse lemurs change their activity more frequently. Furthermore, mouse lemurs with a high body mass stay (stochastically) longer satiated and exhibit shorter hungry periods (i.e. they feed less often than light ones). Notably, a female with the lowest body mass of 33 g would still have a more persistent feeding behaviour at a given observation time than a male with the highest body mass of 59 g.

Figure 3.

Probabilities of motivational state consistency in dependence of the covariates ‘sex’ ((a) females, (b) males), ‘mass’ (solid lines: mass = 59 g, dashed lines: mass = 33 g) and ‘time of night’ (x-axes); P(i → j) refers to ℙ(S_t+1 = j|S_t = i).

We now consider the state-dependent distributions. The joint distribution of the random effects, π₁ and π₂, is displayed in figure 4. Notice the scales: the distribution of π₂—the feeding behaviour in the ‘hungry’ motivational state—is wider, meaning that the differences across individuals are larger for that parameter. For most animals, π₂ is around the mean, 0.925, but the density of π₂ has significant mass (≈0.27) even below 0.9. As regards the probability of feeding in the ‘satiated’ state, there is much less variability across individuals; according to the fitted model for more than 99 per cent of the lemurs that probability is smaller than 0.03. Therefore, between individuals of the same sex and same body mass, feeding behaviour in the hungry state is much more variable than in the satiated state. When compared with a model with no random effects (i.e. constant π₁ and π₂ across subjects), the Akaike information criterion (AIC) selects the model presented here, i.e. the one that includes random effects (ΔAIC = 29.7).

Figure 4.

Fitted joint probability density function of π₁(=x) and π₂(=y).

4. Discussion

Building integrative models is an important step when studying the relationship between proximate behavioural processes and the environment in free-ranging animals [29]. We developed a statistical model with high relevance for the study of behavioural processes and underlying motivational aspects.

4.1. Evaluation of the modelling approach

HMMs have proved to be very useful for dealing with unmeasured state processes; cf. Zucchini & MacDonald [25] for numerous examples. They are immensely flexible and can be applied to different kinds of behaviours, giving them great potential in statistical ecology. They provide increased interpretive capabilities by allowing us to identify transitions in underlying hidden states, even if these transitions are not obvious from observations [59]. A (recently increased) number of publications on animal movement took advantage of the flexibility of HMMs to analyse the processes related to individual movement [14,17,60]. However, ecological applications of HMMs are still relatively rare and have focused mostly on modelling the behaviour of single individuals separately; an exception is Zucchini et al. [13].

Including subject-specific covariates in the model enables factors that drive differences in behavioural dynamics across individuals to be identified. HMMs, and in particular such that incorporate covariates, can facilitate detecting differences in behaviour which are not directly obvious from observations; e.g. the same absolute time devoted to actual behaviours might be reached by quite different motivational state sequences. Possible future directions for extending our model are numerous, but perhaps most fruitful will be the inclusion of covariates with more explanatory power, such as measures of physical condition (body mass does not reflect physical condition per se) or a combination of spatial and behavioural data.

Another important aspect of our model is the inclusion of random effects (individuality). Including individuality seemed useful to us for two different reasons. First, the included covariates ‘sex’ and ‘body mass’ are not likely to explain all behavioural differences between individuals. Second, individual reactions make the model more realistic, since it is unlikely that individuals react in the same manner. Animal behaviour is usually characterized by a combination of a certain degree of flexibility in behavioural responses on the one hand and consistent differences in behaviours between individuals, the so-called animal personalities, on the other hand. The awareness of this paradox is highlighted by the growing interest in animal personalities [61]. By using mixed HMMs, behavioural ecologists might be able to identify how behavioural flexibility and personality differences interact and lead to differences in behavioural sequences.

Important possible extensions of the model we considered here include the relaxation of the first-order assumption concerning the state process (which often will be unrealistic). Technically, it is not difficult to fit HMMs with higher order dependencies in the state process (see [25], §8.3), or to consider more flexible distributions for the state dwell times (i.e. the times spent in the motivational states, which under the first-order assumption are geometrically distributed, see [62]).

4.2. Influence of covariates on feeding behaviour of grey mouse lemurs

The results of our model offer new views and hypotheses for future analyses of mouse lemur behaviour. According to the present model, state-switching probability changed with advancing time of night. At the beginning of the night, individuals changed more often between the states associated with either hunger or satiation. It makes intuitive sense to assume that individuals should be hungry at the beginning of their activity period. But why do grey mouse lemurs switch states more often? We know that about 85 per cent of the diet was composed of tree exudates during the observation period (see the electronic supplementary material). Gum has been defined as a slowly depleting, monopolizable resource [45,63]. Gum trees seem to be most profitable at the beginning of the night because gum production can accumulate bigger drops during the day, whereas gum is regularly harvested during the night. The yield per visited tree is therefore probably higher at the beginning of the night. Possibly, mouse lemurs adjust their behaviour not only to their physiological needs, but also to the availability of the resource. In other words, mouse lemurs switched more often between the ‘satiated’ and ‘hungry’ state because they switched their whereabouts more often to patrol the gum trees in their home range.

Regarding the covariate body mass, the model indicated that heavy individuals fed for shorter periods and had longer non-feeding bouts than lighter individuals (but note that this effect was not found to be significant). Body condition has been found to play an important part in the life of grey mouse lemurs. It influences, for example, mating success of males [64], but also activity patterns of individuals on other temporal scales [46]. Potentially, heavier individuals monopolized recourses of higher quality, or they have a generally reduced activity because of higher energetic reserves or as an antipredator strategy.

The sex effect on the consistency of feeding patterns might be related to dominance structures. Studies from captivity suggest that female grey mouse lemurs are dominant over males [65]. If females are truly dominant over males, they might be much less often displaced from feeding sites than males or monopolize trees of higher productivity. Another possibility for these differences in feeding duration between sexes could be the fact that, following natal dispersal, most males are not living in their natal habitat anymore. Unfamiliarity with their new habitat might force males (temporarily) to feed on whatever resource regardless of the quality. Including data on movements, social interactions or number of feeding trees and food availability for a given individual could be useful to untangle the reasons for the observed sex effect. The application of the model to data on adult individuals or data from subadults in different seasons might also be worthwhile.

5. Conclusion

Based on the evaluation and application of our model, we highlighted the usefulness and advantages of HMMs, in general, and mixed HMMs in particular, for statistical analyses of (multiple) behavioural sequences and the generation of further testable hypotheses, in this case about the feeding behaviour of mouse lemurs and their determinants. Mixed HMMs can help us to derive general organizational mechanisms of behavioural processes and to understand how they influence the ecological dynamics of populations and thus whole ecological environments.

Appendix A. Details on the numerical integration

The details concerning the numerical integration that has been applied to evaluate the likelihood equation (2.5) are as follows. After initial experiments that aimed at identifying the ‘essential range’ of the two random effects' distributions (i.e. the range where almost all mass of the distributions is in) each of these ranges, for the two random effects' distributions as indexed by r, r = 1,2, was split into q equally sized intervals W_i,r = (w_i−1,r, w_i,r), i = 1, … ,q. Let w_i,r^* denote the midpoint of W_i,r, and let h(π, m) denote the likelihood for given π = (π₁, π₂) and individual m:

The likelihood equation (2.5) can then be approximated as follows:

where F_i denotes the cumulative distribution function associated with the density f_i, i = 1, 2. There are two sources of approximation: first, the replacement of the intervals [0, 1] by the respective essential ranges (second line above), and second, the replacement of the function h(π, m) by the constant value of that function evaluated at the midpoints of the respective intervals (fourth line above). The former is not necessary in the present application (since we are dealing with the bounded interval [0,1]), however, as long as the essential range is chosen to be sufficiently large, it improves the approximation since the intervals W_i,r become narrower and the discretization thus finer (and note that this step is necessary in applications where the integration intervals are unbounded). Note that this is by no means the only way in which the integrals can be approximated: one may, for example, apply more sophisticated methods such as Gauss–Legendre quadrature.