Criterion and Construct Validity of Prosthesis-Integrated Measurement of Joint Moment Data in Persons with trans-tibial Amputation

Abstract

Prosthesis-integrated sensors are appealing for use in clinical settings where gait analysis equipment is unavailable, but accurate knowledge of patients’ performance is desired. Data obtained from load cells (inferring joint moments) may aid clinicians in the prescription, alignment and gait rehabilitation of persons with limb loss. Purpose of this study was to assess the accuracy of prosthesis-integrated load cells for routine use in clinical practice.

Level ground walking of persons with transtibial amputation was concurrently measured with a commercially-available prosthesis-integrated load cell, a 10-camera motion analysis system, and piezoelectric force plates. Ankle and knee flexion/extension moments were derived and measurement methods were compared via correlation analysis.

Pearson correlation coefficients ranged from 0.661 for ankle pronation/supination moments to 0.915 for ankle flexion/extension moments (p<0.001). Root mean squared errors between measurement methods were in the magnitude of 10% of the measured range and were explainable.

Differences in results depicted differences between systems in definition and computation of measurement variables. They may not limit clinical use of the load cell, but should be considered when data are compared directly to conventional gait analysis data. Construct validity of the load cell (i.e., ability to measure joint moments in-situ) is supported by the study results.

Introduction

Gait analysis is an important tool in biomechanics research. A typical motion analysis laboratory includes multiple imaging cameras and floor-embedded force platforms. Use of these instruments allows accurate measurement of whole-body kinematics and kinetics. This information is essential to understanding healthy and impaired human movement, and to assessing how movement may be affected by therapeutic or experimental interventions. In persons with lower limb amputation, gait analysis has been used to examine kinematic asymmetry and loading balance¹, function and efficacy of prosthetic components², and quality of prosthesis alignment³. Typical variables of interest include tri-axial joint moments. Loading patterns and joint kinetics that can be derived from this information are typically used to describe prosthesis utilization and gait stability^4–6.

Conventional gait analysis, as described above, has been established as the standard means of studying gait biomechanics. However, it is not without limitations. Conventional gait analysis is most notably constrained by a relatively small capture volume whose dimensions are determined by the size of the laboratory, the number of available cameras, and the number of force plates. Furthermore, conventional gait analysis can only be conducted within a well-controlled physical environment (traditionally a gait laboratory) where the required equipment is available and calibrated for the intended purpose. As a consequence, it is challenging to study performance of physical activities that usually occur in a non-laboratory setting and/or require a larger space than is available in a laboratory setting. An extreme example of such an activity would be downhill skiing. But even common activities, such as indoor walking, may be considered difficult to measure with conventional gait analysis as they typically comprise of multiple consecutive steps and are performed across a variety of environments⁷.

Because of the limitations present in conventional gait analysis, mobile data collection methods are of interest when studying human motion. Devices such as wearable goniometers⁸, gyroscope and accelerometer arrays⁹ and instrumented shoe insoles¹⁰ have been proposed and used for general activity monitoring¹¹, activity classification¹², and gait analysis¹³. These mobile devices have helped overcome the limitations of laboratory-based gait analysis by expanding the volume and duration of data collection. However, their advantages may come at the expense of measurement accuracy, as wearable devices are often prone to motion artifacts¹⁴.

In persons with lower limb amputation, motion artifacts from wearable sensors “(e.g., accelerometers, gyroscopes, or goiniometers)” may be avoided by using sensors that are directly integrated into a prosthetic structure. This unique capability allows analysis of gait across a variety of activities and settings. Analyzing gait is an objective in amputee research¹⁵ as well as in clinical practice¹⁶. The concept of equipping artificial limbs with sensor technology is not new^17,18. Similar sensors are integrated in microprocessor controlled knee- or ankle-components^2,19. However, only recently have stand-alone sensors (e.g., load cells) that are purposefully designed to be integrated into prosthesis become commercially available^20,21. Their rugged designs, user-friendly software interfaces, and standardized prosthetic adapters are motivated by their potential application as clinical tools. Prosthetists can (temporarily) install the device in a patient’s prosthesis to collect data and subsequently use the obtained data to inform decisions about prosthetic prescription, fit, and/or alignment.

One example of a commercially available sensor device designed for integration into lower limb prostheses is the iPecs Lab (College Park Industries, Warren, MI). The iPecs Lab is a compact (1.8″H × 2.8″W × 3.2″D), six degree-of-freedom (i.e., three forces and three moments) strain gauge-based, wireless, portable force sensor. It can be installed by replacing or shortening the standard pylon adapter of any endoskeletal prosthesis. The device includes 32 strain gages that are configured in eight Wheatstone bridge circuits and positioned around the central pylon (Figure 1).

Figure 1.

Strain gage location and orientation in the iPecs sensor.

Forces and moments applied to the sensor are computed from measured bridge voltages and a manufacturer-provided calibration matrix. Software provided with the iPecs allows the user to identify adjacent joints (e.g., knee and ankle in trans-tibial prostheses) and their positions with respect to the center of the iPecs device, where 3-axial moment and forces are directly measured. Joint moments are automatically derived using these inputs. Since iPecs data, like data obtained through conventional gait analysis, is intended to be interpreted by a researcher or practitioner, validity of iPecs moment measurements as measured against the gold standard of conventional gait analysis is of interest.

Assessing the accuracy of prosthesis-integrated sensor measurements is a prerequisite for efficiently utilizing these technologies in research or clinical practice and for facilitating interpretation of obtained outcomes in context to those published in the literature. Given the inherent differences in spatial arrangement and reference frames between force plates and load cells, deviations may be present and should be documented. Additionally, the improved understanding of the capabilities and limitations of prosthesis-integrated load cells will help determine how and for what purposes this equipment is ideally used or avoided. The purpose of this study was, therefore, to compare joint moment data obtained by the iPecs Lab to that obtained using conventional gait analysis in persons with trans-tibial amputation, investigating the hypothesis that joint moments obtained with both methods are identical. To investigate the hypothesis, first, agreement between entire curves was determined. Secondly, curve peak values were compared across measurement methods. Thirdly, findings were also represented as plots for visual assessment of curve agreement. Results of this study were expected to provide evidence of criterion validity of and to aid prosthetics clinicians and researchers in its interpretation.

Methods

Persons with lower limb loss were recruited by flyers distributed to local prosthetists’ offices and support group meetings sites and by postings to online message boards. Inclusion criteria were trans-tibial amputation, proficiency in prosthesis use (defined as being able to walk constantly for 30 minutes without discomfort), and age between 18 and 80 years. Exclusion criterion was use of a prosthesis that did not allow for installation of the iPecs Lab (e.g., an exoskeletal prosthesis, permanent cosmesis, or insufficient clearance between the distal end of the socket and the prosthetic foot).

Ten persons with trans-tibial amputation were enrolled in the study. Data from seven subjects were included in this analysis as complete video recordings allowed for accurate synchronization of iPecs and conventional gait analysis data. Mean age of the remaining seven subjects (Table 1) was 50.9 years (SD=13.4 years), mean weight was 91.1 kg (SD=20.1 kg), and mean height was 180 cm (SD=9 cm). All subjects had unilateral amputations, and regularly used sockets in a modified patellar-tendon-bearing design with elastic roll-on liners and non-articulated energy-storage-and-return feet.

Table 1.

Demographic and anthropometric information on study subjects

Subject number	Sex	Age (years)	Weight (kg)	Height (cm)	Residual limb length (cm)	Time since fitting (years)	preferred walking speed (m/s)
1	F	46	64	168	14	0.5	1.45
2	M	29	81	179	17	5	1.28
3	M	59	118	188	22.5	2	1.13
4	M	59	82	190	18	5	1.42
5	M	60	91	173	15	8	1.27
6	M	38	84	173	23	2	1.35
7	M	65	118	189	23	3	1.52

All procedures were reviewed and approved by the University of Wisconsin – Milwaukee institutional review board (IRB). Informed written consent was obtained from all subjects.

Prior to data collection, study subjects’ prostheses were equipped with the iPecs Lab by a certified prosthetist. Initial alignment of each subject’s prosthesis was documented before removing the prosthetic pylon and installing the iPecs Lab and a shorter pylon so that the original prosthesis length and alignment was preserved. Subjects were allowed as much time as they desired to familiarize themselves with walking with the modified prostheses. Generally, the familiarization period did not exceed five minutes.

Retroreflective markers were located on anatomical landmarks to acquire joint centers of the lower limbs for the conventional gait analysis. A modified Cleveland Clinic marker configuration (Motion Analysis, Santa Rosa, CA) with excluded arm markers was used. It consists of marker triads placed on the lateral aspects of thigh and shank segments²², as well as markers at the lateral and medial sides of knee and ankle axes, at the back of the heel and the front of the toes. In cases where the prosthetic foot shell did not have defined malleoli, the prosthetic ankle marker location was estimated to match the position of the respective lateral and medial malleoli on the contralateral leg with respect to the heel and toes, in an effort to be as consistent as possible. The prosthetic side knee axis was estimated based on the geometry of the socket, and placed two centimeters proximal of the inferior aspect of the patella in the frontal plane, and at a 60/40 (anterior/posterior) division of the knee diameter in the sagittal plane²³, consistent with clinical practice.

Following marker placement, the iPecs unit was prepared according to the manufacturer’s instructions. Distances from the iPecs Lab center to the adjacent leg joints (knee and ankle) were measured to 1 mm with a ruler and input into the iPecs software. Each subject was then asked to stand and lift the donned prosthesis off the ground, by slightly bending their hip and knee. This position was maintained while the investigator zeroed the sensor. Sampling rate was set to 250 Hz and was comparable to rates used in previous studies of prosthesis-integrated^20,21 or implanted²⁴ force sensors. The iPecs unit was set to stream data wirelessly to a nearby personal computer.

Conventional gait analysis was conducted as subjects walked at self-selected speed while wearing their load cell-equipped prostheses. Kinematic data were collected with a 10-camera motion analysis system (Cortex ®, Motion Analysis Corporation, Santa Rosa, CA) at 100 Hz. Kinetic data were simultaneously acquired with 3 force plates (AMTI, Watertown, MA) at 1000 Hz. Camera and force place sampling rates were similar to those used in studies of persons with limb loss⁶. Data were post processed to extract joint moment information based on inverse dynamics calculations²⁵ using OrthoTrak 6.5 (Motion Analysis Corp., Santa Rosa, CA, USA), consistent with methods previously described by other investigators²⁶. Subjects were asked to walk back and forth across the force plates at self-selected speed until four prosthetic-side foot strikes were recorded.

On average, data collection sessions lasted for about 30 minutes once the marker preparations and prosthesis modifications had been completed. An investigator filmed the subject’s prosthesis throughout the session with a digital video camera (Sony Cybershot BSC/W30, Tokyo, Japan). The video recording data was used to synchronize load cell and force plate data (Figure 2).

Figure 2.

Sample of an iPecs recording of axial force data. Axial force data were used to synchronize video recordings with iPecs data. Subjects were asked to stand with equal weight on both legs for ten seconds prior to every pass over the force plates, then initiate gait with a prosthesis step (shown in the plot at approximately 0:00:39). Video data showed that in this example the subject stepped on the force plate with their fifth step. This step was identified in the axial force curve by manual count (highlighted above) and the corresponding iPecs data was subsequently aligned with the concurrently recorded force plate data.

Individual steps were extracted from the load cell data record using an experimentally-determined axial force threshold of 7 N. This threshold provided reliable definition of stance phase (i.e. no missed events, no false positives) and was confirmed by visual inspection of the data. Steps were identified from the kinematic data based on the trajectories of heel- and toe markers, and the corresponding moment data were analyzed. Paired step data collected by iPecs and conventional gait analysis were normalized to 100 samples per step cycle in order to directly compare measurement methods. This down-sampling was assumed to have minimal effect on the analysis, as the frequency content of gait is dominated by frequencies under 10 Hz²⁷.

Statistical analysis was twofold, including a measure of overall curve agreement as well as curve landmark comparisons to investigate the hypothesis that iPecs data are identical to concurrently obtained conventional gait analysis data.

First, knee- and ankle moments derived through each method were compared via correlation analyses. This technique is consistent with those used in comparable studies^28,29. Root-mean-squared-error (RMSE) between data sets was additionally computed to account for offsets or linear errors between iPecs and conventional gait analysis measurement methods (e.g., what might be attributed to improper calibration of the load cell).

Secondly, magnitude and timing of minimal and maximal ankle moment peaks were compared across measurement methods by means of bivariate correlation and paired-samples t-tests. The variation in kinetic patterns for the knee joint disallowed the extraction of respective peak values, a problem that has been described before³⁰.

Analyses were conducted with SPSS 19.0 (IBM, Armonk, New York). A critical alpha of 0.05 was set prior to analysis.

Results

Analysis of joint moments obtained by the iPecs and conventional gait analysis showed an average Pearson coefficient R of 0.807 (p < 0.001), and a root mean squared error of 5.85 Nm/kg across all variables when correlated over the entire step cycle (Tables 2–4). Differences in ankle flexion/extension moments are visually represented in a plot (Figure 3). The correlation here is similar to the group average, whereas the RMSE is above the group average.

Table 2.

Results of correlation analysis on joint moments measured by iPecs and CGA for the entire gait cycle. Coordinate axes refer to the anatomical (local) reference frame. Mx describes the flexion/extension moment, My pronation/supination moment, and Mz axial torsion moment.

	R	R2	n	RMSE(Nm/kg)	Range(Nm/kg)	RMSE %
Mx ankle	0.942	0.887	28	10.37	157.04	6.60
My ankle	0.694	0.482	23	3.06	21.93	13.96
Mz ankle	0.833	0.694	27	1.19	24.92	4.79
Mx knee	0.750	0.563	26	10.42	79.79	13.06
My knee	0.873	0.763	24	5.65	60.05	9.41
Mz knee	0.794	0.631	27	1.71	19.28	8.89

Table 4.

Results of correlation analysis on joint moments measured by iPecs and CGA for swing phase of the gait cycle only.

	R	R2	n	RMSE(Nm/kg)	Range(Nm/kg)	RMSE %
Mx ankle	0.561	0.315	14	0.82	7.68	10.70
My ankle	0.530	0.281	13	0.18	0.92	19.57
Mz ankle	0.593	0.352	19	0.28	1.67	16.63
Mx knee	0.907	0.823	26	4.04	34.42	11.74
My knee	0.689	0.475	23	0.65	5.85	11.11
Mz knee	0.621	0.386	15	0.29	1.29	22.19

Figure 3.

External ankle flexion/extension moment of a sample step, measured by conventional gait analysis (solid line) and prosthesis-integrated sensor (dashed line). In this case, the correlation coefficient was computed as R = 0.934 (p <0.001) and the RMSE was 14.407 Nm/kq or 10.52% of the total range.

A marked deviation from expected knee moments obtained from the iPecs sensor was observed across the sample. The investigators attributed the observed difference to a sign error in the iPecs proximal moment transformation matrix. Therefore, the iPecs-generated knee moments were also derived manually by the investigators using the iPecs force data, the measured distance from the knee axis to the iPecs center, and a sign-modified transformation matrix. The corrected knee moments were more consistent with expected data (Figure 4).

Figure 4.

Sample comparison of external knee flexion/extension moment as computed conventional gait analysis (solid line), the integrated sensor (dashed line), and manual correction of the iPecs data (dotted line). The Pearson correlation coefficient between the externally-calculated knee moment and that derived by conventional gait analysis was R = 0.753 (p < 0.001) and RMSE was 1.60 Nm/kq, compared to R = −0.230 (p < 0.001) and RMSE of 8.38 Nm/kq when the iPecs output was considered instead of the externally-calculated moment.

Magnitudes and timing of peak ankle flexion/extension moments obtained from the iPecs were similar to those derived from conventional gait analysis. After eliminating two outliers from the 28 data sets, where untypical curve shapes disallowed proper identification of data minima and maxima, correlation coefficients between 0.46 and 0.92 were found. Correlations between measurement methods were lower for timing of peaks than for magnitudes, even though the peak times differed by, at most, 9% of the overall step period. The mean difference in peak times between measurement methods was 3.64% (± 2.57%, p < 0.001) for the time of plantar flexion moment maximum and 2.6% (± 2.03%, p < 0.001) for time of dorsi-flexion moment maximum. At mean stride durations of 1.05 seconds, those differences represent deviations between measurement methods of less than 40 milliseconds.

Discussion

This study investigated the hypothesis that joint moment data are identical when measured with prosthesis integrated load cells and conventional gait analysis, in an effort to find evidence for concurrent criterion validity of data measured with the iPecs Lab in persons with trans-tibial amputation. To the authors’ knowledge, this study represents the first comparison of kinetic outcomes measured both with the iPecs Lab and conventional gait analysis. As no existing peer-reviewed literature on the validity of the iPecs unit is available, the present study provides the first step towards developing that evidence base. The iPecs manufacturer reports an instrument accuracy of 1 to 1.5% and a non-linearity of less than 0.5%. Neither results from a previous study with the iPecs Lab³¹ nor information provided in unpublished works indicated any issues with the reliability or validity of data produced by the integrated load cell.

This study independently examined the validity of the iPecs Lab, showing that the joint moment measurements obtained by the prosthesis-integrated load cell are comparable to those obtained through conventional methods (e.g., conventional gait analysis), but are not entirely identical. Thus, the hypothesis was rejected. Moments reported by the iPecs Lab software and those derived from conventional gait analysis are estimated from different input variables. Joint moments derived from conventional gait analysis require direct measurement of lower-limb kinetic and kinematic data as well as knowledge of joint segments’ physical dimensions and inertial properties. Moments reported by the iPecs Lab are derived from measured forces and moments at the center of the sensor unit, and spatial distances between the sensor and adjacent joints. These differences in input variables likely explain the differences in joint moments identified in this study.

In this context, it may be noteworthy that the iPecs moment calculations are based on fewer variables, and thereby require fewer assumptions than the conventional inverse-dynamics calculations utilized in conventional gait analysis. In both methods, it is assumed that body segments are rigid and that the residual limb does not move within the prosthetic socket, but only the inverse dynamics approach necessitates information on segments’ mass, center of gravity, and joint friction. Accurately determining those parameters³² is time consuming and is therefore often foregone in favor of approximations, including the assumption that joints are frictionless and that inertial properties of artificial limbs are identical with normal limbs^33–35.

Differences also exist in the joint axes definitions upon which each measurement method is based. Data provided by the iPecs Lab originate from a coordinate system located at the center of the sensor unit. As the iPecs Lab is integrated into the prosthesis, this coordinate system rotates and translates with movement of the prosthesis. Joint axes are defined solidly within this system and move with respect to the global coordinate system. Differences in defining joint axes in both systems may entail that iPecs-based data describes slightly different quantities than data that are derived from the static, global coordinate system used by conventional gait analysis. The Cleveland Clinic marker set has been noted to have poor reliability in estimating (frontal plane) ankle moments^36,37, possibly due to the particular method of defining the foot segment. The post processing of conventional gait analysis data often involves gap filling (from marker dropouts) and trajectory smoothing, which introduces another source of differences between both methods. This limits direct comparability between measurement methods, but also poses the possibility that prosthesis-integrated load cells may more robustly measure the construct of joint moments in trans-tibial prosthesis^38,39. The value of measuring joint moments and forces in alternative reference frames, even when “…they represent subtly different biomechanical quantities” (p.449) has been suggested⁴⁰. The authors concur that measurement in alternative reference frames may have merit. As an example, data derived from the mobile coordinate system may hold valuable clinical and scientific information about bilateral weight distribution, foot placement, and utilization of prosthetic components. Additionally, specific research outcomes may be more readily measured with an integrated load cell as opposed to deriving them from externally-measured kinetic and kinematic data.

A technical finding of this study demonstrated that the proximal (i.e., knee) moment reported by the iPecs software may not well reflect the moment estimated through conventional gait analysis. Manual calculation of the knee moment, using the acquired force data, produced knee moment data more closely correlated to the conventional gait analysis-derived moments than the iPecs software. The manufacturer has been informed of this finding and the software has since been updated to a corrected version.

As the application of the iPecs unit in this study was similar to how it might be used in a clinical setting, limitations in the accuracy of its individual alignment and calibration with respect to the reference frame are noteworthy. Realizing a perfect orientation of the iPecs sensor axes parallel to the anatomical joint axes is hampered by the restricted adjustability of endoskeletal adapters and the possible circumstance that the joint axes of knee and ankle are not parallel to each other. Likewise, calibration errors may be introduced when prostheses are held in a slightly angled position during the zero-setting. Furthermore is the definition of ankle joint centers in prosthetic feet without a mechanical ankle hinge always subject to inevitable assumptions, much like the center of knee joint rotation cannot be accurately defined as a static axis. Assumed, that those deviations from parallelism are within a range of 20 degrees, the thus introduced error would be 1-cos(10 deg) = 1.5%. By most standards, those errors are likely too small to warrant the effort that would be needed to correct them in clinical applications.

The size and composition of the subject sample might be suggested as a limitation. Only active users of unilateral prostheses with energy-storage-and-return feet components participated in this study and only data from 28 steps (i.e., 4 steps per subject and 7 subjects) across the study sample were evaluated. This may limit the transferability of our findings to populations with different anthropometrics, demographics, or prosthetic prescriptions when the collected data cover a range outside the here observed one. However, there is also little to suggest these factors would make a difference. A limitation of the study is that a standard anthropometric model without correction of segment properties for the prostheses was used for the conventional gait analysis part. Finally, it should be noted that only one iPecs unit was used in this study. While the investigators have assumed that other units will be identical in performance, generalization to other units should be made with caution until comparison among units can be made. Although the device has been validated by standardized tests at the manufacturers laboratories, as well as at independent facilities prior to being introduced in the market⁴¹, a standardized test of reliability and accuracy of the iPecs output data may be recommended for subsequent studies.

Despite limitations to use the load cell as described in this study, the investigators believe that direct measurement of outcomes using prosthesis-integrated load cells provides information that may not be easily attainable through conventional motion analysis. In response to the found differences in data from the load cell and the gold standard conventional gait analysis, one could posit that the load cell method is more direct and thus potentially more valid in the sense of construct validity. The construct of joint moment, that is the product of applied forces and their distance from the joint axis, is not directly measurable, but resembles the moments within the segment that are directly measured by the integrated load cell.

For example, the integrated load cells can be used to acquire and compare consecutive prosthetic-leg steps. Step-to-step variability is recognized as an indicator of gait stability^34,42 and may allow researchers or clinicians to assess prosthetic users’ for fall-risk, sub-optimal alignment, or inadequate prosthetic suspension. Since larger numbers of samples (i.e., steps) can generally be obtained with an integrated instrument than with conventional force plate experiments, data collected with an integrated load cell may better represent inherent gait patterns. It also seems plausible that longer sample periods achievable with the integrated load cell may allow for detection of rare gait events, such as tripping or stumbling. Integrated load cells also offer the potential to enhance single-subject studies⁴³ of prosthetic interventions where subtle variations between experimental interventions (e.g., prosthetic components, designs, or alignments) may be more readily measured. The ability to acquire multiple, sequential steps in a short period of time may allow researchers or clinicians to better assess changes that occur over short periods of time (e.g., acclimation to a new intervention) and avoid testing effects (e.g., fatigue) that may affect outcomes when they are studied in a conventional motion analysis laboratory.

In summary, joint moment data obtained with the iPecs Lab are similar, but not identical to those obtained with conventional gait analysis. Differences in measured outcomes between measurement methods are attributed to differences in joint axes definitions, input variables, and assumptions of the models used to derive selected outcomes (e.g., joint moments). The noted limitations with the iPecs instrument may not prohibit use of this device (or similar devices) in the clinic, where such tools may be applied for the monitoring and documentation of prosthesis performance. For these purposes, objective measurement data as provided by prosthesis-integrated gait analysis can effectively supplement traditional gait assessment methods that rely on visual observation and subjective feedback obtained from clinical patients.

Table 3.

Results of correlation analysis on joint moments measured by iPecs and CGA for stance phase of the gait cycle only.

	R	R2	n	RMSE(Nm/kg)	Range(Nm/kg)	RMSE %
Mx ankle	0.947	0.897	28	11.62	142.30	8.16
My ankle	0.649	0.421	23	3.35	12.91	25.94
Mz ankle	0.814	0.662	26	1.27	17.76	7.17
Mx knee	0.788	0.621	27	10.28	60.53	16.98
My knee	0.767	0.588	28	6.05	46.70	12.95
Mz knee	0.763	0.582	26	1.86	11.85	15.68