Effect of parameter selection on entropy calculation for long walking trials

Abstract

It is sometimes difficult to obtain uninterrupted data sets that are long enough to perform nonlinear analysis, especially in pathological populations. It is currently unclear as to how many data points are needed for reliable entropy analysis. The aims of this study were to determine the effect of changing parameter values of m, r, and N on entropy calculations for long gait data sets using two different modes of walking (i.e., overground versus treadmill). Fourteen young adults walked overground and on a treadmill at their preferred walking speed for one-hour while step time was collected via heel switches. Approximate (ApEn) and sample entropy (SampEn) were calculated using multiple parameter combinations of m, N, and r. Further, r was tested under two cases r*standard deviation and r constant. ApEn differed depending on the combination of r, m, and N. ApEn demonstrated relative consistency except when m=2 and the smallest r values used (rSD=0.015*SD, 0.20*SD; rConstant=0 and 0.003). For SampEn, as r increased, SampEn decreased. When r was constant, SampEn demonstrated excellent relative consistency for all combinations of r, m, and N. When r constant was used, overground walking was more regular than treadmill. However, treadmill walking was found to be more regular when using rSD for both ApEn and SampEn. For greatest relative consistency of step time data, it was best to use a constant r value and SampEn. When using entropy, several r values must be examined and reported to ensure that results are not an artifact of parameter choice.

INTRODUCTION

The use of entropy methods to calculate regularity or predictability in a time series has vastly increased over the last twenty years. Claude Shannon was the first scientist to introduce an algorithm for calculating information entropy within a time series[1]. Modifications to the original Shannon entropy algorithm have been made, including approximate[2,3], sample[4,5], multiscale[6,7], increment[8], permutation[9], and multiscale permutation[10] entropy, just to name a few. The use of entropy analysis has been wide ranging from investigations in financial markets to weather patterns to thermodynamics to biology.

Each entropy algorithm requires parameter selection that may or may not affect the calculation of entropy[11,12]. The m parameter, is the number of data points that are to be compared, sometimes called the window or vector of data. The parameter r, is the set tolerance, sometimes called radius, that is utilized to determine if two vectors are considered similar. The last parameter, N, is the length of the entire data set.

The two most popular entropy algorithms utilized for human movement analysis are approximate entropy (ApEn) and sample entropy (SampEn). ApEn was originally developed as a method to measure regularity within a time series[2,3,13]. However, a limitation of ApEn is that it is biased toward regularity because it includes a self-match count to avoid taking the logarithm of zero[4,5]. Furthermore, it requires fixed parameters when comparing data[13] and lacks relative consistency[13], a term used to describe how stable the output of the algorithm is when input parameter selection is changed slightly. Our previous work has demonstrated that ApEn is more prone to inconsistent output as compared to SampEn when utilized for time series of 200 data points or less[12]. However, both algorithms were sensitive to certain combinations of parameters. When reporting entropy results, it is of highest priority that the results are not an artifact of parameter choice. In addition, due to the sensitivity of ApEn to parameter choice, comparisons between studies and data can only be done if the parameter choices are fixed. This is very difficult, as each data set requires careful selection of parameters based upon that unique set of data. SampEn was developed to overcome the limitations of ApEn[4,5]. The calculation of SampEn does not include a self-match and the logarithm of the sum of conditional probabilities is taken as compared to ApEn. As stated by Richman and Moorman[5], ApEn calculates probabilities “in a template-wise fashion” whereas, SampEn calculates “the negative logarithm of a probability associated with the time series as a whole” (pp H2042).

One of the most difficult parameters to select is that of the tolerance, r, value. If the tolerance level is large compared to differences in values of sequential points, the probability values used to calculate entropy will be high and vice versa. Therefore, selection of the tolerance has a crucial impact on the outcome. There have been many proposed methods to determine r, including using the standard deviation (SD) of the entire time series[2,13], standard error of the entropy values[4], and using fixed tolerance values[14,15].

The length of the data set is a parameter of concern for human movement scientists. When dealing with pathological populations, it is sometimes difficult to obtain long enough data sets in order to perform analysis. Data sets less than 200 points appear to be too short for entropy analysis, yet, it may take up to 2000 data points for stabilization of entropy values[12]. It is currently unclear as to how many data points are needed for reliable entropy analysis.

Another limitation of entropy analysis is the need for collection of uninterrupted data[2,3]. To collect uninterrupted data, continuous or discrete, and the amount of data required for analysis, many researchers have subjects walk on a treadmill versus walking overground. This allows the researcher to collect uninterrupted steps without the concern of space and/or equipment constraints. However, the treadmill could be considered a constraint as it limits speed fluctuations in walking that are normally present in overground walking. In addition, there are physiological and biomechanical differences between overground and treadmill walking[16–25]. Nonlinear measures have shown conflicting results regarding the difference in the structure of variability between treadmill and overground walking[26–30].

Thus, the purpose of this research was to determine the effect of changing parameter values on entropy calculations for long gait data sets (i.e., step time) using two different modes of walking. In addition, to understand the effect of changing parameters on entropy calculations, an examination of tolerance, r, was completed. It was hypothesized that SampEn would maintain relative consistency across all data lengths and be resistant to changes in parameter values.

MATERIALS AND METHODS

Twenty-one subjects participated in this research study. Foot switch data was collected from subjects, but subjects whose foot switch data contained any signal dropout were excluded. Therefore, 14 subjects’ data from the original cohort were included in analysis (7 males; 24.9±4.2 years; 1.71±0.12 m; 69.3±16.8 kg). All participants were in excellent health, had no conditions that would inhibit their ability to walk for one hour, and reported physical activity at or above the currently recommended level[31]. The University’s Institutional Review Board approved all procedures and subjects provided consent prior to participation.

Participants were instrumented with force-sensitive resistors on each heel and second metatarsal head (Trigno 4-channel footswitch sensor, Delsys Inc., Natick, MA; 148 Hz). The foot switches were secured to the skin of the foot using tape before the participant placed their own sock and shoe on their foot. Cables were then secured to the top of the participants’ shoes with tape. Data streamed wirelessly to a data acquisition unit (Delsys Inc., Natick, MA).

During visit one, participants walked for one-hour at a self-selected speed around an indoor track (201.2 meters/loop). Speed was not recorded or monitored. Study personnel followed them within five meters to keep the wireless unit within signal range. Participants returned for a second visit and were asked to walk for one-hour at a self-selected speed on a treadmill (1.08±0.11m/s). Participants wore the foot switches similar to visit 1. After selection of their speed, participants rested a minimum of five minutes before the hour-long data collection commenced. Participants were asked not use the handrails of the treadmill unless they needed to for safety. No safety harness was worn.

Step time was calculated from the force-sensitive resistors worn during both trials. Data were processed using custom MATLAB (Mathworks, Inc., Natick, MA) codes. Data were inspected for spikes (a value greater than three SDs), removed, and filled using a cubic spline. Auto detection of the leading and trailing edge of the force-sensing resistor square wave was visually confirmed. Event timings for left and right heel strikes and toe offs were saved. Step time was calculated as the time interval between subsequent right and left heel strikes (Figure 1).

Figure 1.

Data from a representative subject for overground (top) and treadmill (bottom) step times.

Both ApEn and SampEn were quantified from the step time time series generated from both trials. Entropy analysis has been described previously[12]. ApEn and SampEn were calculated using all combinations of the parameters m=2 and 3; r=0.15, 0.20, 0.25, and 0.30*SD (rSD); and N=500, 1000, 1500, 2000, 2500, 3000, 3500, 4000, 4500, and 5000 (supplementary data). As part of a secondary analysis regarding tolerance selection methods on the step time data, ApEn and SampEn were calculated using the above combinations of the parameters m, and N; however, r was not multiplied by the SD. The r values selected were r=0, 0.0033, 0.0068, 0.0136 (rConstant). These r values were chosen based off the sampling rate, 148 Hz, 0, 0.5, 1, and 2 times the sampling interval.

A linear mixed model with compound symmetry covariance structure was conducted in SAS (SAS Institute Inc., Cary, NC) to determine the effect of condition, r, m, and N on ApEn and SampEn. Mode (overground and treadmill) was included in the model to adjust for differences between conditions. Significance was determined at alpha <0.05 level. Residual plots were examined to confirm residuals were approximately normally distributed.

RESULTS

rSD

A significant interaction was found between r and N (p=0.006) and the interaction between N and m was marginally significant (p=0.05), meaning ApEn differed depending on the combination of r, m, and N (Figures 2&3). Treadmill walking was more regular than overground walking.

Figure 2.

ApEn (diamond) and SampEn (square) as a function of step time data length for r=0.15 (A), 0.20 (B), 0.25 (C), and 0.30 (D) multiplied by the standard deviation for overground (gray) and treadmill (black) walking when m=2.

Figure 3.

ApEn (diamond) and SampEn (square) as a function of step time data length for r=0.15 (A), 0.20 (B), 0.25 (C), and 0.30 (D) multiplied by the standard deviation for overground (gray) and treadmill (black) walking when m=3.

For SampEn, there was a significant effect of r (p<0.001); as r increased, the value of SampEn decreased. When the r was increased to a critical level, the tolerance was large enough to allow for additional matches to be counted, thus lowering the entropy (Figure 4). In our data, enough time series demonstrated a critical change in tolerance, adding additional matches between r=0.2 and 0.25*SD, thus causing the “flip” in results between conditions (Figure 4B). Treadmill walking was more regular than overground walking.

Figure 4.

SampEn as a function of tolerance are plotted for all overground (gray) and treadmill (black) step time time series (m=2, N=5000). The top row represents rSD entropy values where the bottom row represents rConstant values. Treadmill and overground trials for each subject are plotted (A & C) as well as the group average (C & D). When r is multiplied by the SD (A), the tolerance for comparisons is too small for like-matches to be found within each time series. When the tolerance reaches a critical level, the number of matches increases allowing the entropy value to drop in the treadmill trials. This causes a “stair step” decrease in entropy across r values in some trials. Enough of the subjects’ time series reached this critical level when r increased from 0.20 to 0.25*standard deviation (B) and overground entropy then became greater than treadmill entropy. When r is held constant (C & D) and not multiplied by the standard deviation of the time series, the SampEn value decreases with every increase in r.

rConstant

ApEn was different depending on the combination of r, m, and N (p=0.002; Figures 5&6). For SampEn, there was a significant effect of r (p<0.001); as r increased, SampEn decreased. Overground was more regular treadmill walking.

Figure 5.

ApEn (diamond) and SampEn (square) as a function of step time data length for r=0.0 (A), 0.003 (B), 0.007 (C), and 0.013 (D) for overground (gray) and treadmill (black) walking when m=2.

Figure 6.

ApEn (diamond) and SampEn (square) as a function of step time data length for r=0.0 (A), 0.003 (B), 0.007 (C), and 0.013 (D) for overground (gray) and treadmill (black) walking when m=3.

DISCUSSON

The overall objective of this research study was to determine the effect of changing parameter values of m, r, and N on entropy calculations for long data sets using two different walking conditions. It was hypothesized that SampEn would maintain relative consistency across all data lengths and be resistant to changes in parameter values. Our hypotheses were partially supported.

Relative consistency was defined by Richman and Moorman[5], “if [entropy] of one data set is higher than that of another, it should, but does not, remain higher for all conditions tested” (pp H2039). The ApEn has been known to fluctuate within a range of m and r values[13], however, the direction of difference between two data sets should be consistent within that fluctuation. When m=2, ApEn lacked relative consistency at smaller r values for both rSD and rConstant. Treadmill entropy moved from less regular to more regular at greater data lengths compared to overground (Figures 2A&B and 5A&B). At higher r values, ApEn relative consistency was maintained for both rSD and rConstant. When m=3, relative consistency was maintained in all combinations of r and N except r=0.25*SD and N=500 for both rSD and rConstant. Overall, ApEn was sensitive to the parameter choice selected, especially combinations of r and N, and could have an effect on reported findings if parameters are not closely inspected.

Opposite of our hypothesis, we found that SampEn also lacked relative consistency. When r=0.2*SD and m=2 or 3, overgound and treadmill step time entropy switch at an N value of 3000. In this case, treadmill walking was more regular as compared to overground and became less regular as N increased, >3000. Of particular interest was the lack of global relative consistency across rSD values. When r=0.15*SD and 0.20*SD (and greater N values for the latter), treadmill walking was less regular (greater SampEn values) than overground walking. However, when r=0.25*SD and 0.30*SD, a global shift occurred and the inverse was found. After further investigation, it was determined that this reversal was due to the tolerance (rSD) being close to precision of the data. When r was multiplied by the SD of the time series, the tolerance was very small, only allowing exact matches to be counted. When r was increased to a critical level, the tolerance was large enough to allow for additional matches to be counted, lowering the entropy (Figure 4).

Another potential explanation in the shift of SampEn overground and treadmill values may be due to the distribution of step time corrections (i.e., the differences between one step time and the following step time). Consider the following two time series: S1=[1 3 2 3 1 2 2] and S2=[1 2 3 2 1 2 3]. They have the same mean=2 and same SD=0.8165, but the distribution of between-data point differences in not the same: D1=[−2 1 −1 2 −1 0] and D2=[−1 −1 1 1 −1 −1]. When rectified and averaged, D1=1.1667 and D2=1. While D2 is dominated by small directional changes, D1 has a wide range of directional changes. When rSD is used with a small r, it is more likely that more similar vectors are detected in S2 because of the relative small changes in vector size. However, as r increases, this bias could potentially shift to favor S1. If a similar difference in the distribution of step correction exists between the two walking conditions, the data sets with smaller changes would be biased toward lower entropy values at small rSD, while the data sets with larger changes would be equally biased at a larger rSD.

In this particular study, the rConstant values were comparable to the tolerance values used for rSD. The average SD for the overground condition was 0.016 and for treadmill was 0.021. When multiplied by r, tolerance values ranged from 0.002–0.005 and 0.003–0.006 for overground and treadmill, respectively. Additionally, it is important to consider that when using rSD, different tolerances were used for treadmill walking compared to overground but when using rConstant, the tolerances were the same. The probability of finding a match (and perhaps a lower SampEn) may be greater for groups with a lower SD if rConstant is used.

Although, the use of rConstant with SampEn lead to high relative consistency, different results were found. Under every combination of parameters, overground walking was more regular than treadmill, opposite of the findings when using rSD. Treadmill walking was more regular using rSD for both ApEn and SampEn. And no difference between conditions when rConstant for ApEn. This differential pattern between rConstant and rSD is similar to the findings of Forrest et al[15]. When examining the effect of parameter selection on ApEn for increasing isometric force outputs, they reported that using a fixed r (based off of the measurement noise in the system) led to a different pattern in ApEn as compared to rSD. As effort increased, force output became increasingly less regular when using the fixed r. An inverse pattern was found for rSD, regularity increased with increasing effort. The authors concluded that the use of multiple r values may be needed[15]. The current study is the first to use a rConstant with SampEn. Important to note is using a rConstant eliminates the scale-invariant property of the entropy algorithm. Caution should be displayed when using this method.

The influence of self-matches on the calculation of ApEn was also demonstrated in this experiment. ApEn continued to increase as N increased and did not plateau until 2500–3000 data points when m=2. As N increases, the effect of self-matches on the entropy value decreases as the algorithm is more likely to find matches beyond the self-match with larger data sets. However, when m=3, the ApEn value never appeared to plateau or stabilize. Similar to how self-matches have less influence on time series of longer data lengths, it is easier to match a pattern length of two than a pattern length of three.

There are several limitations to this study. First, this study only examines one aspect of gait, a temporal parameter. This was due to the limitation in methodologies to collect one-hour of uninterrupted data overground. Other methods may be possible for future studies including inertial movement sensors, accelerometers, or wireless based sensors. Second, these results may be biased by the resolution of the methodology chosen. Step time data were calculated to five significant digits. Having more or less significant digits may affect the number of matches identified by the algorithms. Third, it is possible that our healthy young participants became fatigued by the protocol. One hour of self-paced walking could induce muscular fatigue, causing a change in walking patterns. However, muscular fatigue was not measured and it is unknown as to whether fatigue was a factor. However, none of the subjects reported that fatigue affected their walking during the trials. A visual inspection of the plots for all trials did not reveal a drifting or clear change in the step time fluctuation during the end of the trials (Figure S1). Furthermore, we had no objective measure of mental fatigue/boredom and cannot exclude it to influence our subjects. Fourth, walking speed potentially differs between the two conditions. In studies using unconstraint overground walking as an experimental setup, the natural fluctuation in walking speed is inherent in the recorded data. In contrast, the use of treadmill will constrain these fluctuations. The objective of the study was not a direct comparison of entropy measures between overground and treadmill walking, rather to investigate the parameter consistency when applying ApEn and SampEn to long data sets.

As stated above, the current study investigated only one temporal aspect of walking, step time. This is a discrete, temporal event. Future studies may examine the effect of entropy calculations on discrete, spatial events as well as continuous variables. Based on the underlying pattern matching of both algorithms, it is possible that entropy calculations respond differently to discrete versus continuous or cyclical data types.

CONCLUSION

Overall, selecting a proper combination of r, m, and N should be done to provide relative consistency. For greatest relative consistency of step time data, it was best to use a constant r value and SampEn. We suggest when using a constant r, examine r values of 0 to n*1/sampling rate by iterations of 1/sampling rate, with n being however the different values to be tested. Due to relative consistency issues for both algorithms, we encourage all future work using entropy analysis to publish supplemental data with multiple parameters to ensure that the results reported are not an artifact of parameter choice.

Highlights.

• Parameter selection influences both approximate and sample entropy

• Multiplying r times the standard deviation can influence relative consistency

• Several parameters must be examined and reported