Entropy analysis in gait research – methodological considerations and recommendations

Abstract

The usage of entropy analysis in gait research has grown considerably the last two decades. The present paper reviews the application of different entropy analyses in gait research and provides recommendations for future studies. While single-scale entropy analysis such as approximate and sample entropy can be used to quantify regularity/predictability/probability, they do not capture the structural richness and component entanglement characterized by a complex system operating across multiple spatial and temporal scales. Thus, for quantification of complexity, either multiscale entropy or refined composite multiscale entropy is recommended. For both single- and multiscale-scale entropy analyses, care should be made when selecting the input parameters of tolerance window r, vector length m, time series length N and number of scales. This selection should be based on the proposed research question and the type of data collected and not copied from previous studies. Parameter consistency should be investigated and published along the main results to ensure transparency and enable comparisons between studies. Furthermore, since the interpretation of the absolute size of both single- and multiscale entropy analyses outcomes is not straightforward, comparisons should always be made with a control condition or group.

Introduction

Entropy is defined as the loss of information in a time series. Based on what is known about the current state of a time series, entropy will quantify the probability of the next state of the system.⁸⁵ First introduced in the 1800s in the field of thermodynamics, the quantification of entropy still remains based on probability. Approximate entropy was introduced as a mathematical tool for quantifying probability in biological data in 1991.⁶³ The first alteration to the method aimed to correct a bias in the calculation was sample entropy.⁶⁹ Multiscale entropy²⁰ was later introduced in 2002 to examine entropy over multiple time scales. The introduction of correlation entropy, permutation entropy, increment entropy, symbolic entropy, von Neumann entropy, fuzzy entropy, variations of multiscale entropy, and cross entropy algorithms has increased the difficulty of interpreting and comparing data across literature. The use of entropy analysis on human gait data was first published in 1998.⁶⁰

Furthermore, each algorithm takes parameter inputs, and if selected improperly can lead to the reporting of incorrect findings. The majority of entropy algorithms need three inputs for analysis. First, the length of the time series—usually abbreviated as N. Second, is a tolerance radius used to determine whether or not patterns within the time series are similar—usually abbreviated as r. Third, is the length of data that will be compared across the entire time series to determine the conditional probabilities—usually abbreviated as m. This parameter has also been called embedding dimension, vector length, pattern length, segment length, or pattern window.

An example of how the selection of input parameters greatly affects reported findings can be seen as to whether older or young adults have a more probable gait pattern. The issue of results changing due to parameter input was first documented by Karmakar et al. (2007) in which they demonstrated that changing the tolerance for comparison led to different interpretations between young and older adults’ gait.³⁸ Findings have indicated that older adults’ joint angle patterns while walking are less probable, indicating more random patterns, compared to young adults.⁴⁶ Conversely, it has been reported that age increases probability of gait, indicating a more rigid walking pattern, when using accelerometry data.^{3, 79} The effect of parameter input on findings between older versus younger adults’ walking was further emphasized in a 2013 paper by Yentes et al.⁸⁷ Likely, some of these issues with different findings between age or pathological groups could be due to the type of data used or sampling rates during data collection.^{57, 68} Other factors that may affect results include data treatment of outliers or nonstationary of data.²²

Hence, the inconsistency between studies has led to conflicting results, making data interpretation challenging. Further complicating the issue is the interpretation of the underlying theory of entropy, specifically, whether these algorithms represent complexity. Therefore, the purpose of this paper is to review methods of entropy analysis in human gait data and provide guidance for best practices moving forward (pun intended). Both single- and multi-scale entropy algorithms are reviewed, including challenges with calculation and interpretation. A review on the use of entropy for gait analysis and sources of conflict between studies are provided. Lastly, best practices include information on types of data, sampling rates, data treatment, and parameter selection.

Single-Scale algorithms

This section will discuss algorithms that are designed to estimate the entropy on one time scale. As entropy is related to probability, the greater the probability of the time series, the less new information you receive from the next states of the system, and vice versa. An entropy value close to 0 represents a system that is highly probable with little new information gained from the next states of a system. On the other hand, a higher value of entropy indicates less probability and a greater amount of new information that is gained from the next data points in the time series. A rigid, or systematic, time series would result in low entropy, whereas a time series that is highly random would result in high entropy.⁸⁵ The standard unit of each of these algorithms is bits, as in bits of information.

Pincus introduced approximate entropy in 1991 as a way to quantify entropy in biological data, as previous algorithms relied on infinite and/or mathematical equation data.^{63, 64} It is important to remember approximate entropy is an estimate of the actual entropy based on a time series of limited length. Data length becomes an important consideration.

To perform the calculation of approximate entropy, the time series of length N is divvied up into vectors of length m, sometimes referred to as the embedding dimension. Each vector is compared to every other vector within the time series, including itself. As the vectors are compared, the number of vectors that are similar to each other are counted. To determine if two vectors are similar, one compares each element in the vector with the corresponding element in the comparison vector. Determining similarity between two vectors is done using a distance function such as Chebyshev (maximum distance between points) and Euclidean (distance of a straight line between points). The maximum distance between the two vectors, Chebyshev distance, is typically used.^{22, 63, 69} The two vectors are considered to be similar if the comparison elements are all within the tolerance window, r. Take for example a certain vector of length m = 3, A = [ 6 2 9 ], that is being compared to Vector 1 = [ 1 3 5 ] and 2 = [ 5 3 8 ] where r = 1. Therefore, for a vector to be counted as similar to vector A, the elements in the comparison vectors would need to be within the following tolerance window [ 5–7 1–3 8–10]. Vector 2 is counted as similar; however, Vector 1 is not considered similar. The process is repeated with a vector length of m+1. To extend the example, take vector B = [ 6 2 9 1 ] where r = 1. For a vector to be counted as similar to vector B, the elements in the comparison vectors would need to be within [ 5–7 1–3 8–10 0–2].

The similar vectors of length m are counted, and the log average found. Then similar vectors of length m+1 are counted, and log average found. The total conditional probability of vector length m+1 is subtracted from the total conditional probability of vector length m to calculate the approximate entropy. Values of approximate entropy will range from 0 to ~2 bits, where 0 bits represents no entropy or a perfectly repeatable timeseries, and 2 bits represents a perfectly random timeseries. For a detailed description of the approximate entropy algorithm, please see Yentes (2016).⁸⁵

Approximate entropy was found to suffer from a lack of relative consistency,⁶² being highly dependent on data length,^{17, 87} and biased toward a more probable outcome as it always includes a self-match for each vector to avoid taking the logarithm of zero.²⁷ To offset these issues, sample entropy, was developed.⁶⁹ Sample entropy eliminates the counting of self-matches and takes the logarithm of the sum of conditional probabilities rather than the logarithm of each individual conditional probability. Sample entropy is defined as the “negative natural logarithm of the conditional probabilities that two sequences similar for m points remain similar at the next point”.⁶⁹ Sample entropy values theoretically range from 0 to converging toward infinity.⁶⁹ This upper limit is in the extreme case that no matches between vectors have been made, a case in which all data points are independent and uncorrelated ^{47 (Appendix B)}. A perfectly repeatable time series would have a sample entropy of ~0 and a perfectly random time series would have a sample entropy converging toward infinity. This is unlikely when using gait or physiological data sets. As stated in ⁶⁹, sample entropy is bounded by 2[(N − m − 1)(N − m)]⁻¹ to ln (N − m) + ln (N − m − 1) − ln (2) where N is the data length and m is the vector length.

Over the years, several entropy algorithms have been introduced, including permutation and increment entropy. These two algorithms are different as they consider the temporal order of the data within the comparison vectors. Permutation entropy compares neighboring values and identifies patterns of increasing or decreasing values.^{8, 71} It is thus, “the Shannon entropy of the distribution of ordinal patterns in the time series.”^{42 (page1)} Given that similarity is determined based on sequence order, permutation entropy does not consider how the elements in a vector are different or equal. The parameter needed to calculated permutation entropy is the vector length (embedding dimension), m. For example, vector 1 = [ 23 57 ] and 2 = [ 1026 1027 ] would be converted to [ 1 2 ] and [ 1 2 ], respectively, as the first number in each vector is smaller than the second number. These two vectors are considered similar because both vectors demonstrate an increase in sequence, or the same permutation. However, the values of [ 23 57 ] are not similar to the values of [ 1026 1027 ], nor is the difference between consecutive values within the vector. If you consider vector 3 = [ 87 43 1010 0 ] and vector 4 = [ 91 801 1873 24 ] they would become [ 3 2 4 1 ] and [ 2 3 4 1 ] based on their permutations. They would not be considered similar. Permutation entropy is calculated from the conditional probabilities using Shannon entropy (Equation 1)

$$H=-\sum _{i=0}^{N-1}{p}_{\pi }{log}_2{p}_{\pi }$$ (1)

where H is the Shannon entropy and p is the probability of each unique permutation’s relative frequency, π.^{8, 71}

Increment entropy was introduced after permutation entropy.⁵⁴ Similar to permutation entropy, each increment of the time series is transformed into a word with two letters. The sign (positive, negative, or zero) and magnitude of the increment is coded into the word. Increment entropy has the “ability to detect either energetic change or structural changes”.^{54 (page 2)} The parameters needed to calculated increment entropy include the vector length, m, and the resolution level. To illustrate, consider a time series x(i) = [ 2 9 5 7 0 ] that is constructed into an increment time series y(i), where y(i) = x(i + 1) – x(i); therefore, y(i) = [ +7 −4 +2 −7 ]. To construct the word, the first letter is the sign, positive, negative, or no sign. The second letter is the magnitude, which is quantified by the increment’s standard deviation and the resolution level. The relative frequency of each unique word is quantified by dividing the total amount of any unique word by the length of the time series minus m. The increment entropy is found by calculating the Shannon entropy of each word’s relative frequency.

Calculation of each of these algorithms is not as straightforward as presented in the previous examples. Parameters cannot be selected arbitrarily. Consider first the length of the time series. Early on, approximate entropy was found to be a highly useful measure as it was reported it could be used on time series of 75–100 or 1000 data points.^{62, 63, 66} In later years, it has been noted that approximate entropy is highly influenced by the length of the time series.^{17, 87} Yentes et al (2013) demonstrate that approximate and sample entropy appear to stabilize around 2000 data points derived from the logistic map;⁸⁷ although, sample entropy has been reported to be unaffected by data length.⁷⁰ However, extending time series length may introduce drift into the data and could affect the calculation.²²

The tolerance window may have the greatest influence on the calculation of entropy.^{17, 19, 47, 53, 55, 75, 86} Selecting a tolerance window too small will limit the number of matches found and on the other hand, selecting too large of a window could lead to too many matches being found and increase the probability. The tolerance window is typically calculated as r times the standard deviation of the time series.⁶³ Yet, fixed tolerance windows, r, have been used.^{75, 86} The selection of r should depend on the research question being asked as well as the relative consistency of the outcome (see Best Practices below for a detailed description).

Additional considerations to calculation should include the presence of outliers,⁵⁸ sampling rate,^{2, 26, 57, 68} and the type of data being collected.⁵⁷ First, outliers will alter the standard deviation of the time series and have a direct influence on the tolerance window used for comparison. Second, increasing a sampling rate to extend the time series length is not appropriate. In part, entropy is a measure of information and increasing the sampling rate only adds redundant data to a time series. It has been shown that increasing a sampling rate, lowers the sample entropy value, as data become redundant, probability increases.^{57, 68} It has been shown that the number of data points within each stride influences the sample entropy value, more so than the number of cycles within the time series,⁶⁸ and that discrete data are more robust to sampling rate changes than are continuous data.⁵⁷ Selection of sample rate should be done a priori.⁷⁷ Third, entropy output, and interpretation of the outcome, is dependent on the type of data, continuous versus discrete. Take for example a step length time series versus knee joint angle time series sampled at 120 Hz. With an m = 2, this would mean that the comparison vectors are two and three (m + 1) consecutive data points long. The step length sample entropy interpretation would be the probability of two and three consecutive step length values occurring again within the data set. For the knee joint angle at 120 Hz, this would be two and three consecutive points 0.0083 seconds apart and interpreted as the probability of these point in the joint angle occurring again in the gait cycle or multiple gait cycles. However, if one would want to know the probability of a larger portion of the knee joint angle or if the entire angle repeats itself again in the time series, a much larger m value would need to be selected.⁵⁷ Depending on the research question, the data type and m would be selected.

Another solution has been to introduce a time lag into the construction of vectors m.³⁰ Instead of selecting consecutive data points, a time lag can be introduced, essentially skipping data points. To illustrate, consider a time series x(i) = [ 27 81 13 52 94 45 79 56 33 ]. Using a time lag of 1, the vectors constructed would be [ 27 81 ], [ 81 13 ],…, [ 56 33 ]. When using a time lag of 2, the vectors would become [ 27 13 ], [ 81 52 ],…, [ 79 33 ], and a time lag of 3 [ 27 52 ],…,[ 45 33 ]. If data have been over sampled, consecutive data points are redundant, meaning that they share a high amount of mutual information.²⁹ The amount of information shared from point-to-point decreases as you increase the lag. Identifying the time lag that would construct vectors with minimal mutual information²⁹ may overcome redundancies in a time series.

Consideration must be given to the input parameters, mainly due to the relative consistency of each algorithm. Approximate entropy lacks relative consistency or simply, the stability of the measure.⁶² And in some instances, so does sample entropy.⁸⁷ Simply, if the input parameters are changed, the direction of results may change. As a hypothetical example, assume one has used the input parameters of N = 1000, m = 2, and r = 0.2 times the standard deviation of the time series to compare stride length of older and younger adults. The results indicate that the younger adults are more regular than are the older adults. Upon altering the input parameters slightly by using r = 0.25, same N and m, the results are different, the older adults are more regular than young. One has conflicting results and could inappropriately select the parameters that produce results fitting closest to their hypothesis. Issues regarding relative consistency and its potential effect on inappropriate conclusions has been shown with spatiotemporal gait data^{86, 87} and joint angles⁵⁷. Based on one study,⁴ gait center of pressure appears to have fairly stable consistency.

To ensure interpretations in data are not an artifact of parameter selection, reporting of several combinations of parameters in supplemental data has been encouraged.⁸⁷ Confirming that the parameter selection used and reported demonstrated consistent, directional results across nearby parameters is important. The primary concern here is to ensure the direction of differences between groups are the same, not the magnitude. It is emphasized that using parameters others have reported in the literature is no longer appropriate. Parameter selection should depend upon the data, and confirmation should be provided that this selection was appropriate. Reporting additional parameter combinations will increase transparency of parameter selection and ensure that reported results are not the artifact of parameter choice.⁸⁷

Entropy measures have historically been considered a quantification of complexity.^{62, 63, 65} Defining complexity is not straightforward and definitions vary. For this paper, complexity is defined in accordance with the principles outlined in Delignières and Marmelat (2012) and Costa et al (2005).^{22, 28} We define complexity in human movement as a system that 1) originates from a deterministic origin, 2) cannot be broken down into fundamental components —“infinitely entangled”²⁸, and 3) “operates across multiple spatial and temporal scales”²² — structural richness. Entropy cannot provide information regarding whether or not the system is from deterministic or stochastic sources.²² A time series with a large entropy value can only indicate that a time series is more random than one with a smaller entropy value. Richness or entanglements of the data are not quantified using these algorithms. A system that is complicated should not be considered complex.²⁸ As defined by Delignières and Marmelat (2012), a complicated system is one that has a large number of components, the “interactions between components are not of prior importance, and knowledge about the parts is sufficient for understanding the whole”.^{28 (page1)} White noise is thus, complicated, not complex. Consequently, a time series with a large entropy value should be considered complicated, as single-scale entropy cannot quantify the three parts of the definition of complexity above.

The question becomes, how can single-scale entropy algorithms be defined, as probability, predictability, or regularity? Entropy algorithms are based off a measure of probability. Stating that one group has a higher probability to have a certain walking pattern appear again compared to another group is appropriate. Since prediction is related to probability, this interpretation may be inferred as correct; however, caution must be taken as prediction of the next state has technically not occurred in calculation. Regularity is a term that has been used to describe gait data as well. Costa et al (2005) state that these measures “quantify the degree of regularity of a time series by evaluating the appearance of repetitive patterns”.²² The key to this statement is the ‘appearance of repetitive patterns’ or the structural form of a time series. Consideration of regularity in this manner may be presumed as appropriate. Hence, probability, predictability, and regularity may be used for interpretation depending on the research question and hypothesis.

Multi-Scale algorithms

This section will discuss algorithms which quantifies entropy on multiple time scales. Based on the idea that complexity of biological systems arises from the interaction of components on multiple scales, Costa et al. in 2002 introduced multiscale entropy as a novel measure of complexity in physiological time series.²⁰ The authors presented an algorithm which creates multiple new time series from one single time series recorded from the biological system in question and sample entropy is then calculated on each new time series.^{20, 22} In several studies, Costa and colleagues applied multiscale entropy to heart rate interbeat intervals in various populations.^{20, 21, 24, 37, 49} In agreement with the ‘loss of complexity’ theory proposed by Lipsitz and Goldberger,⁵² the authors observed that patients with heart related diseases had lower complexity compared to healthy controls.^{20, 21, 24}

While widely used in cardiovascular research, multiscale entropy is far less used in biomechanical gait research. Costa et al. (2003) was the first to apply multiscale entropy to quantify complexity of human gait.²³ Subsequently, multiscale entropy has been applied to trunk accelerations signals,^{34, 72–74} stride time intervals time series,^{51, 67} and lower limb muscle activity signals³⁶ in various individuals and under various conditions. Although, numerous studies have presented new versions of the original algorithm to overcome potential methodological drawbacks,³² the performance of the different algorithms have rarely been investigated. Raffalt et al. (2018) compared the performance of four different alternative multiscale entropy algorithms to the original version when applied to both theoretical signals and stride time intervals time series of healthy individuals during treadmill and overground walking.⁶⁷ The four alternative algorithms included the refined composite multiscale entropy,⁸² intrinsic mode entropy,⁶ multiscale fuzzy entropy,⁸⁴ and generalized multiscale entropy²⁴ which have been proposed to overcome limitations of the original multiscale entropy algorithm. Three criteria were formulated to evaluate the performance of the algorithms. An appropriate algorithm 1) should be able to distinguish between three theoretical signals with different temporal structure, 2) should consistently return the correct level of complexity in accordance with the expected complexity of theoretical signals (i.e. high sensitivity), and 3) display relative high parameter consistency (i.e. that change in parameters does not result in large changes in outcome — see previous section). Prior to the analysis, it was proposed that pink noise signals would exhibit a high level of complexity, white noise signals would exhibit an intermedium level, and a sine wave with added white noise would exhibit the lowest level of complexity. It was observed that only the original multiscale entropy algorithm and the refined composite multiscale entropy algorithm could confirm this expectation while at the same time, contained high sensitivity and high parameter consistency.⁶⁷ Thus, it was concluded that these two algorithms are the most appropriate to use when quantifying complexity. In the following, only these two algorithms are discussed.

Calculation of multiscale entropy and refined composite multiscale entropy

In both the original algorithm and the refined composite multiscale entropy algorithm, multiple time series are constructed according to the following procedure. First, for a one-dimensional discrete time series, [ x_{1, …} , x_i, … , x_N ], coarse-grained time series, [ y^(τ) ], are constructed determined by the scale factor, τ, following equation 2:

$$y_j^{\left(\tau \right)}=\frac{1}{\tau }\sum _{i=\left(j-1\right)\tau +1}^{j\tau }{x}_i,1\le j\le \frac{N}{\tau }$$ (2)

This creates non-overlapping windows of length τ, where data points within each window are averaged. The first scale [ y¹ ], equals the original time series and the length of each subsequent coarse-grained time series is equal to the length of the original time series divided by the scale factor τ.^{20, 22} In the original multiple entropy algorithm, sample entropy is then calculated on each coarse-grained time series with the same input parameters, m and r, for all scales. The tolerance window is calculated as the r times the standard deviation of the original time series. Finally, the outcome is plotted as a function of scale.²⁰ To achieve a quantification of the complexity of the signal in question, the area below the sample entropy versus time scale curve is calculated.^{20, 23}

For refined composite multiscale entropy, sample entropy is calculated differently than in multiscale entropy. First, for each coarse-grained time series corresponding to the scale factor of τ, the number of matched vector pairs, $n_{k,\tau }^{m+1}$ and $n_{k,\tau }^m$, is counted. Secondly, the mean of $n_{k,\tau }^{m+1}$ and $n_{k,\tau }^m$ for 1 ≤ k ≤ τ is calculated as ${\overline{n}}_{k,\tau }^m$ and ${\overline{n}}_{k,\tau }^{m+1}$. Finally, refined composite multiscale entropy (RCMSE) is defined as the logarithm of the ratio between ${\overline{n}}_{k,\tau }^m$ and ${\overline{n}}_{k,\tau }^{m+1}$ (equation 3).

$$RCMSE=-\ln \frac{{\overline{n}}_{k,\tau }^{m+1}}{\overline{{\overline{n}}_{k,\tau }^m}},\text{where}{\overline{n}}_{k,\tau }^{m+1}=\frac{1}{\tau }\underset{k=1}{\overset{\tau }{\Sigma }}{n}_{k,\tau }^{m+1}\text{and}{\overline{n}}_{k,\tau }^m=\frac{1}{\tau }\underset{k=1}{\overset{\tau }{\Sigma }}{n}_{k,\tau }^m$$ (3)

Accordingly, the refined composite multiscale entropy algorithm quantifies entropy more accurately and with lower probability of undefined entropy values at different scales.⁸²

As with single-scale entropy, multiscale entropy and refined composite multiscale entropy require selection of input parameters which can affect the outcome. In addition to the tolerance limit, vector length, and time series length, an appropriate scale number should be chosen. This is a crucial choice, as the length of the subsequent coarse-grained time series will equal the length of the original time series divided by the scale. Thus, using 10 scales on time series of 10,000 data points will result in a reduction of data point for the 10^th coarse-grained time series with a factor 10. As mentioned previously, the length of the investigated time series can influence the outcome of sample entropy calculations significantly.^{86, 87} Therefore, care should be made to investigate the potential bias of the decreasing length of the coarse-grained time series.

As discussed earlier, entropy measures on a single-scale quantifies probability, predictability, and regularity and not complexity of the time series in question. In contrast, multiscale entropy embraces the structural richness of a complex system by including multiple time scales. However, while the time series of single-scale entropy measures has a clear representation of the investigated biological system (e.g. joint angle or step time intervals), this is not the case with the coarse-grained time series in multiscale entropy and refined composite multiscale entropy. With increasing scale, the coarse-grained time series contain less of the detailed information present in the original time series but the fundamental patterns are preserved. Thus, liking a specific scale with underlying control mechanisms is not straightforward.

Entropy analysis of gait

With the rapid increase of entropy usage over the past two decades, the literature has exploded as well. The purpose of this section is to provide an overview of general findings with regards to regularity of walking patterns associated with aging and task demands. It is not meant to be a comprehensive review.

AGING

In general, it appears that walking patterns increase in regularity with aging; yet, this does not seem to always hold true. Using data from 10-meters of walking, walking patterns become more regular with age and more so with the presence of impairments.³ A more regular walking pattern was also present in sedentary older adults compared to healthy young adults and older adults that regularly participated in Nordic walking.⁹ On the other hand, an early report showed that older adults had a more irregular joint angle range of motion compared to younger adults.⁴⁶ Mediolateral acceleration of movement at the lower trunk demonstrated, increased irregularity with increasing speed in adults over the age of 65 years.³¹ Whereas, in the vertical direction it was most regular at the fastest speed. ³¹

Increasing task demands, typically through dual task paradigms, does appear to increase irregularity of walking patterns in older adults. Mediolateral center of pressure displacement was more irregular during dual task walking compared to single task walking, and irregularity increased as persons aged.^{4, 5} Completing a letter fluency task while walking increased irregularity of anteroposterior acceleration compared to walking only in a group of older adults.⁴⁸

With maturation and into aging, complexity (quantified by multiscale entropy) may demonstrate a U-shaped pattern. A steady decrease in complexity of the trunk acceleration was present when age increased from childhood into early adulthood.^{10, 11} However, complexity increased again from early adulthood into middle-age and older adulthood.¹⁰ When walking overground versus on a treadmill, differences in trunk acceleration complexity between conditions were found within the older adult group, but were not significantly different from young adults.¹²

FALLS

Minimum toe clearance was more regular in older adults at risk for falls compared to healthy young and elderly groups;³⁸ however, when compared between those with fall risk and a control group, another study reported increased irregularity in minimum toe clearance in those at risk for falls.⁴³ Fallers have increased irregularity in acceleration while walking at a maximum walking speed. This irregularity is also moderately correlated with the maximum walking speed.⁴⁵ When walking overground at a self-selected walking speed, no difference in acceleration of the lower trunk complexity was found between the fallers and non-fallers.¹³ This may emphasize the need to increase task demands beyond comfortable walking to alter complexity in some walking parameters. Studies have suggested that complexity measures may have predictive ability of falls.^{33, 74}

INCREASED TASK DEMANDS

Perturbing the sensory system does appear to increase the irregularity in walking patterns. Bilateral mastoid vibration while walking led to a more irregular walking pattern as compared to walking with no mastoid vibration.¹⁸ Perturbations to the visual system may result in more irregular gait patterns compared to perturbations of the somatosensory system.²⁵ Furthermore, optic flow perturbations (faster or slower than walking speed) increased irregularity of joint angle range of motion as compared to no optic flow.³⁹ Walking was shown to be more irregular as one walked on a surface with less stiffness compared to one with high stiffness.¹⁵ Although, when walking in unstable shoes, center of mass velocity was more regular compared to walking in flat shoes.¹⁴

Alterations to the task demands of walking may increase the regularity of walking patterns. Walking at a pace slower than your comfortable walking speed increased regularity of stride time and stride length.⁷⁸ Stride rate when walking slower, faster, or walking to a metronome was less regular compared to preferred walking speed.¹ When walking backward, joint angle range of motion was significantly more regular as compared to forward walking.⁴⁰ The same was true when walking uphill compared to no incline and downhill walking.⁸⁰ Hip joint range of motion was more regular when carrying a load while walking than walking alone.⁵⁹

Best practices for use

The best practices for use are not as straightforward and simple as one may hope. In this section, we outline the majority, if not all, of the considerations that should be made before applying an entropy algorithm to your dataset. We highly encourage researchers to be transparent about these considerations and to publish what they can in the supplemental data. Far too many of the considerations below can lead to spurious results and the reporting of findings that are due to an artifact of parameter or data choice.

First, consider your research question. Which algorithm will best answer your question? If you are looking for complexity, a multiscale algorithm would be best. Second, before collecting data, consider your data collection set up. It is an unfortunate circumstance to collect an entire experiment, only to realize that the data collected was not long enough. The following list will outline considerations that should be made before collecting data.

1. What type of data do you need to answer your question? Will you need discrete or continuous data (e.g., spatiotemporal data or joint angles)?

   1. If it is spatiotemporal data, should you use stride or step data? There are reasons to select one or the other. Step data allows you to quantify the fluctuations from step-to-step directly. When using stride data, the contralateral limb’s fluctuations are indirectly quantified.

   2. Are the data inherently cyclic such as joint angles? The repetitive nature of such signals would dominate the entropy estimation and could mask subtle differences between groups or walking conditions. However, if used it should be kept in mind that this type of data has been shown to be more sensitive to the sampling frequency (i.e. the number of data points within a gait cycle) and less to the number of gait cycles in total.^{4, 57, 68} Further, it has been shown that cyclic data, like joint angles and center of pressure displacement, are sensitive to the selection of m and r.^{4, 57}

   3. Are you considering using a phasic signal such as electromyography or a force on/off signal? These signals are inherently tricky. The on/off phases of the signal will affect the calculation of entropy. What does it mean biologically to have sections of the time series read zero, and matches being counted for these time periods? A few papers have used these types of signals but it is still unknown as to whether they are appropriate for entropy analysis.^{36, 41}

2. What is the sampling rate that you are using?

   1. Perform a power spectral analysis on pilot data to determine the correct sampling frequency. Stergiou (2001 Chapter 9) has written a clear explanation of how to do this.⁷⁷

      1. If using spatiotemporal data, the highest resolution in determining heel strikes or other gait events is extremely important. Data may need to be sampled at a higher rate to determine these gait events accurately.⁸⁶

      2. When using continuous data, it is important to keep in mind that the nervous system does not have infinite resolution. Reflexes and muscle activity modulations are controlled on a millisecond level^{50, 76} and sampling data beyond this frequency (1000Hz) may lead to redundant information. A power spectral density done a priori should alleviate this concern.

   2. If data are oversampled, you run the risk of capturing data that is redundant and lack new information for the algorithm to quantify. This leads to low entropy values, biasing the results toward predictability.

      1. If data are oversampled, consider down sampling data to the appropriate rate. Another option is to use a lag when creating comparison vectors.³⁰

3. Are you trying to capture predictability within or between cycles? This is related in part to the section on selecting parameters below. What is the biological interpretation of m for your study? For example, the entropy of step length time series with an m of 2, is interpreted as the predictability of the lengths between three steps. Do these lengths between three steps occur again and again in the time series? What then if you have a time series of joint angles and an m of 2? What if the joint angle was sampled at 100 Hz? You have three points from a joint angle curve that are each 10 milliseconds apart. Thus, you are determining the predictability of these three data points, within 20 milliseconds of each other, happening over and over again. What does this mean biologically? Possibly nothing. Potentially you would rather know the predictability of the joint angle cycle. If the gait cycle is normalized to 100 points from heel strike to toe off, possibly an m of 99 or 100 would be more appropriate. To our knowledge, this has yet to be done for gait studies and would need to be tested in dedicated future works.

4. If you are planning on using a multiscale algorithm, how many scales will you need? Previously, 6 has been used in gait research but is that the appropriate number to answer your research question? What impact will the choice of scales have on the coarse-grained time series length and subsequent sample entropy calculation?

5. Will you be able to collect enough data in one continuous trial? Many researchers are interested in concatenating short trials together to make one long time series.^{44, 56, 61} In pathological populations, collecting a long enough continuous trial can be difficult, mainly due to fatigue. Equally, in clinical settings it is tempting to concatenate straight-line walking data from multiple trials of back-and-forth walking in a corridor while removing data from turning steps. Whether or not this is appropriate is open for debate. When concatenating trials, you must consider how this will affect the calculation of entropy. The assumption of the algorithm is to quantify fluctuations from one step to the next; however, when concatenating trials, you will have fluctuations in one trial that are not due to fluctuations from the previous trial(s). It has been found that sample entropy was sensitive to concatenation of gait trials.⁶¹

6. All of these considerations should lead the investigator to determine the length of data to be recorded.

Third, after you have collected your data, here are considerations that should be made:

1. First thing, plot your data. Look for visual outliers and spikes.

2. Quantify the stationarity of the data. To do this, examine the mean and standard deviations of different sections of the time series. Does the mean or standard deviation change across the trial? If so, you may need to consider detrending the data or removing outliers.

   1. Detrending of non-stationary signals can be done using various techniques such as cascade moving average filter,¹⁶ ensemble empirical mode decomposition,⁸³ or wavelet filtering.⁷

   2. If you have an outlier, consider if the point is biological or the result of methodological error? If the point is biological, should it remain?

   3. Removing an outlier must take thought as well. Removing the data point completely could influence the quantification of fluctuations. It may be possible to replace the outlier with an estimated data point; however, the methods in which to do this have yet to be fully investigated.

3. Related to removing outliers or detrending your data, should you filter your data? There is no good answer to this question. If you have considered all items above before collecting data, your time series should be the best reflection of the system behavior in question. Filtering data may then remove small fluctuations that are in fact, biologically important. On the other hand, due to the nature of a data type (i.e., electromyography), possibly filtering of the data is warranted. If the choice is to filter, cutoff frequencies should be determined using a known method such as Winter’s⁸¹ or Jackson’s³⁵ method. Using previously reported cutoff frequencies may not be appropriate for your data.

4. Selecting the parameters is important as these selections may have the greatest effect on the eventual entropy value.

   1. Consideration of m is highly important. An m of 2 is the most popular, in part due to the original suggestions by Pincus^{63, 64, 66}. However, as mentioned in the section above, what does the m mean biologically? Narrow down the m values you consider further and critically think about what selection is most appropriate. This may mean trying different m values in combination with multiple r values to determine the best relative consistency.

   2. Another parameter that must be closely considered is that of r, the tolerance for considering similar vectors. If your r value is too small, vectors will not have many matches; yet, if the r value is too large, almost all vectors will be considered a match.

      1. We advocate for researchers to try multiple r values and to publish their findings in the supplementary material. If similar studies have used an r value of 0.2 times the standard deviation, then one should examine the relative consistency of r = 0.15, 0.25, and 0.30 times the standard deviation as well. Plotting the entropy values for groups versus the r values on the x-axis will allow you to examine the relative consistency. The goal would be to have relative consistency for an r value just below and just above your selection.

      2. Multiple methods have been proposed regarding how to select r. It may be selected based on the minimization of the maximum entropy relative error or the maximum value of entropy.^{19, 53, 55} Or possibly using a fixed tolerance, whether or not r should be multiplied by the standard deviation.^{75, 86} The choice of how to select r should relate to the research question at hand.

      3. If a fixed tolerance is not used and r is multiplied by the standard deviation, any factor that affects variance will affect the tolerance. This includes but is not limited to: data length, nonstationarity of the data, spikes, and outliers.

      4. The discussion of a recent paper⁸⁶ also presented an argument that potentially distribution of between-data differences in the time series may affect the tolerance. Consider two time series: S1 = [1 3 2 3 1 2 2] and S2 = [1 2 3 2 1 2 3] with the same mean = 2 and same standard deviation = 0.817. When calculating the between-data point differences: D1 = [−2 1 −1 2 −1 0] and D2 = [−1 −1 1 1 −1 −1] with a rectified mean of D1 = 1.167 and of D2 = 1. D1 has a wider range of directional changes and D2 is dominated by small directional changes. When using a relative small r multiplied by the standard deviation as tolerance, more similar vectors is likely detected in S2 because of the relative small changes in vector size. However, as r increases, this bias potentially shifts in favor of S1. When comparing subject groups or experimental conditions, differences in the distribution of between-data point differences could potentially bias the results.

Based on the individuality of entropy analysis to data sets, one may wonder if comparing entropy findings across papers is valid. We argue that in certain cases, it is. If an author has been transparent about their data treatment and parameter selection, readers will be assured that the findings are not an artifact of the algorithm. Additionally, if the author has presented their entropy results within the context of the appropriate control groups, these relative differences between groups should be valid. As researchers, we must make every effort that we are comparing valid results to valid results when writing manuscripts. We must also be honest regarding the limitations of our studies.

Conclusion

Quantification of single scale and multiscale entropy are intriguing methods to assess regularity/predictability and complexity in both basic and clinical gait research. What then are normative entropy values? This question may never be able to be answered. Entropy algorithms are sensitive to their inputs and we expect that all studies may have different values, even for their control groups. It may be too early in our studies of entropy of gait to be able to compare our findings to a set of normative values. In contrast, we should work to describe the directional differences between groups and compare our findings to directional differences reported previously. If our work is done with intent and thought, findings from entropy studies may be considered a true finding and not an artifact of parameter selection.