Validity and reliability of the XSENSOR in-shoe pressure measurement system

Daniel Parker; Jennifer Andrews; Carina Price

doi:10.1371/journal.pone.0277971

. 2023 Jan 17;18(1):e0277971. doi: 10.1371/journal.pone.0277971

Validity and reliability of the XSENSOR in-shoe pressure measurement system

Daniel Parker ^1,^*, Jennifer Andrews ¹, Carina Price ¹

Editor: Peter Andreas Federolf²

PMCID: PMC9844836 PMID: 36649238

Abstract

Background

In-shoe pressure measurement systems are used in research and clinical practice to quantify areas and levels of pressure underfoot whilst shod. Their validity and reliability across different pressures, durations of load and contact areas determine their appropriateness to address different research questions or clinical assessments. XSENSOR is a relatively new pressure measurement device and warrants assessment.

Research question

Does the XSENSOR in-shoe pressure measurement device have sufficient validity and reliability for clinical assessments in diabetes?

Methods

Two XSENSOR insoles were examined across two days with two lab-based protocols to assess regional and whole insole loading. The whole insole protocol applied 50–600 kPa of pressure across the insole surface for 30 seconds and measured at 0, 2, 10 and 30 seconds. The regional protocol used two (3.14 and 15.9 cm² surface area) cylinders to apply pressures of 50, 110 and 200 kPa to each insole. Three trials of all conditions were averaged. The validity (% difference and Root Mean Square Error: RMSE) and repeatability (Bland Altman, Intra-Class Correlation Coefficient: ICC) of the target pressures (whole insole) and contact area (regional) were outcome variables.

Results

Regional results demonstrated mean contact area errors of less than 1 cm² for both insoles and high repeatability (≥0.939). Whole insole measurement error was higher at higher pressures but resulted in average peak and mean pressures error < 10%. Reliability error was 3–10% for peak pressure, within the 15% defined as an analytical goal.

Significance

Errors associated with the quantification of pressure are low enough that they are unlikely to influence the assessments of interventions or screening of the at-risk-foot considering clinically relevant thresholds. Contact area is accurate due to a high spatial resolution and the repeatability of the XSENSOR system likely makes it appropriate for clinical applications that require multiple assessments.

Introduction

In-shoe pressure measurement systems are used in clinic and research to quantify contact area and pressure at the foot-shoe and foot-insole interface [1]. Plantar pressure data may be used to quantify in-shoe pressures during specific tasks [2], in specific populations [3], or to optimise interventions such as therapeutic footwear for diabetic populations [4]. More recently clinical practice guidelines have taken the step to recommend the use of therapeutic footwear with a demonstrated plantar pressure relieving effect [5], reinforcing the need for effective, valid and repeatable (within- and between-day) methods of measurement.

Several in-shoe pressure measurement systems are commercially available. Their suitability to specific research or clinical use is determined by sensor type, size, resolution, and distribution and how these align with the data-collection protocol and intended application. Capacitive sensors, for example, succumb to creep when exposed to extended periods of loading, which may require correction in a static or fatiguing protocol [6]. This would make them less suitable as devices to assess static balance tasks, however these may be perfectly suitable for use within a study that exclusively observes dynamic tasks. Devices may also be selected for their ability to identify small changes in pressure at particularly high- or low-pressure ranges, for example, when modifying ulceration risk within an at-risk foot or observing contact area changes in response to orthotic or footwear intervention [4, 7]. Assessments of intervention effectiveness typically focus on analysis of load distribution and peak or mean pressure reduction, across discrete foot regions, averaged over multiple mid-gait steps [8]. Contact area, calculated as the number of active sensors allows evaluation of changes to load distribution driven by the geometry of the intervention [8]. Peak plantar pressure, normally calculated as the absolute highest pressure for a given region during stance allows for assessment of exposure to potential risk [9]. While mean plantar pressure, calculated as the average pressure across sensors within a discrete region during stance, incorporates changes to contact area and magnitude to evaluate the broader effect on load distribution [10].

Owing to the requirement for assessment of intervention effectiveness, it is crucial that systems are demonstrated to perform in a repeatable way. The validity and reliability of some in-shoe measurement devices have been investigated utilising both bench-top [11, 12] and in-situ methods [13] through protocols with varying methodologies. High repeatability with the Pedar in-shoe system has been demonstrated between days [13, 14] and the measurement of midfoot pressure and contact area variables also demonstrate moderate to high intra class correlation coefficients (ICC = 0.556–0.947 and 0.529–0.921 respectively) between trials [15]. The Tekscan system has been reported to have low durability and to demonstrate significant creep (19% within 15 min) and hysteresis, high variability between and within sensors and low overall repeatability [16]. A more recent publication with their updated F-Scan in-shoe system however demonstrated reliability values that were higher, with contact area in particular having high ICCs (0.91–0.98) and the forefoot and toe region also being highly repeatable values for pressure time integral and average and peak pressure (ICC = 0.83–0.98) [17]. This was consistent with findings with the same system in a diabetic cohort where peak pressure coefficient of variations (CV 0.150, 0.155) and ICCs (0.755 and 0.751) of the metatarsal heads had the best indices of reliability [18]. Most more recent literature demonstrates systems produce high correlations or strong relationships between two days or measures.

These studies highlight that consideration of appropriate technical specification of the in-shoe pressure system is required prior to selecting a system for use in clinic and for research purposes. To evaluate a plantar pressure system for clinical use and benchmark performance it is important to consider the expected range, the observed change with intervention and the minimally important difference (MID) across repeated measures [19, 20]. MID and observed change in plantar pressure measurement is most clearly established in the context of offloading the diabetic foot and international guidelines for prevention of foot ulcers provides a strong recommendation for reductions of 30% peak plantar pressure for effective intervention based on moderate quality of evidence from the literature [5].

In 2016, Price et al., published an assessment of the validity and repeatability of three widely used in-shoe pressure systems (Medilogic (T&T Medilogic, Medizintechnik Gmb Schönefeld, Germany), Pedar (Novel, Munich, Germany) and Tekscan (Tekscan Inc., Boston, USA)) for a range of pressure magnitudes and durations [21]. Combining this prior protocol for objective pressure system evaluation with the known requirements for 30% reduction to detect clinically meaningful differences for intervention in diabetic patients facilitates a benchmark for appropriateness for clinical assessments. The aim of this study is to assess the validity and between-day reliability of the XSENSOR X4 system (XSENSOR Technology Corporation, Calgary, Canada) with the aim of determining its suitability for clinical assessments in diabetes.

Method

The XSENSOR X4 Insole System is a wireless in-shoe pressure measurement system, available in UK sizes 4–15 (Fig 1). Insoles have a standard sampling rate of 75 Hz and 230 capacitive sensels (resulting in a resolution: 7–9 mm), reportedly recording pressures from 7–883 kPa with ±5% accuracy of the full-scale calibration. Two XSENSOR insoles (UK 4/5 and 9/10), hereafter referred to as S4 and S10, underwent the same protocol as previously described to assess three other commercially available insole systems [21]. As is standard with capacitive systems, sensel number is consistent across insoles and therefore sensel area of the XSENSOR insoles varied between the two insole sizes (S4 = 0.49 cm² and S10 = 0.67 cm²). Insoles were supplied with a calibration file for use in XSENSOR software. A whole insole and a regional protocol were applied to the sensors and throughout both protocols, sensels which registered >10 kPa were deemed active sensels and included in analysis.

Whole insole protocol

The TruBlue calibration device (Novel, Munich, Germany) was used to load each insole evenly across its surface at a range of pressures (50, 100, 200, 300, 400, 500 and 600 kPa). Each pressure was applied manually as quickly as possible using a hand release valve to inflate the internal bladder. The start of the measurement was when the target pressure was achieved (monitored by a pressure gauge; VDO Instruments, Germany).

Each pressure was applied for 30 seconds (maintained at ±2% target pressure, monitored by a pressure gauge). In each of the seven target pressure conditions three measurements were recorded, and an average mean and average peak pressure were calculated across the insole. Based on manufacturer guidance some sensels on the periphery of the insole were excluded from analysis due to a stepped border enclosing sensel cabling which was expected to affect the local application of even pressure.

Regional protocol

Two cylinders were used to apply regional loads in this study. Each cylinder was made of rigid plastic and had a 1mm layer of PORON at its base. Cylinders were applied to the rearfoot of each insole, where we would anticipate the heel to be making contact (Fig 1a). Cylinder areas were chosen to represent anatomical sites on the foot, specifically the calcaneus (surface area 15.9 cm²) and the metatarsal head (surface area 3.1 cm²). Cylinders were loaded by the application of weights to their top surface and supported by a vertical linear bearing to ensure load through their centre to generate realistic pressures for these anatomical regions during stance of 50, 110 and 200 kPa. For each of the six conditions (2 areas x 3 pressures) three measurements were recorded of 30 seconds.

Variables

Variables were calculated for the regional and whole insoles protocol using custom-written scripts in Python (3.7.3, Numpy 1.18.1, Pandas 0.25.3).

Within the regional protocol average contact area was calculated as the cumulative area of active sensels at T0, T2, T10 and T30 of the three 30 second trials. T0 reflected a measure at the first instant at which the required pressure had been applied and the other T values added to this. Validity was established by calculation of root mean square error (RMSE) when compared to the known surface area of each cylinder (3.1 and 15.9 cm²), defined as area error (AErr). Time dependence was assessed by calculating the change in area over time. Repeatability between day was calculated using intra-class correlation coefficients (ICC; two-way mixed absolute agreement) for values recorded at 2 seconds.

Within the whole insole protocol the repeatability and validity of the held load at 0, 2, 10 and 30 seconds were outcome variables (T0, T2, T10 and T30). Peak and mean pressure measurement errors (MErr) were calculated based on the absolute value versus the target pressure, also presented as a percentage of the target pressure. Validity was established by calculation of RMSE across all active sensels to the target pressures applied in the TruBlue device. Consistency of the measured pressure across the insole was assessed by calculating the percentage of sensors within 5% of the all sensel mean. Time dependence was assessed by calculating the change in peak and mean pressure over time. Reliability was explored for the data by comparing day one to day two for each pressure (between day) and both days to the target value. These were then considered for each pressure separately then a combined dataset across all target pressures. Correlation was used to identify agreement, paired t-test to assess statistically significant bias and Bland-Altman plots with 95% limits of agreement to display combined systematic bias and random error between tests [20], defined as Reliability error (RErr). To assess limits of agreement the absolute difference and mean were calculated for each measure, difference data was assessed for normality (Shapiro-Wilk) and heteroscedastic error (correlation coefficient of absolute difference vs mean) as per the method of Nevill and Atkinson (1997) [22]. Due to the presence of heteroscedastic error in all cases, natural logarithms were taken for measurement data, differences between log transformed measures were then confirmed as normally distributed. Systematic bias was calculated as the average difference between days and an antilog was taken to calculate the mean bias ratio. Limits of agreement were calculated as agreement ratios between which 95% of difference between days can be calculated [22]. Using the 30% reduction guideline [5] we have defined an analytical goal of 15% reliability error (RErr) (combined systematic bias and random error) such that each days measured value should not be affected by more than half of the objective reduction. All statistical analysis was performed in Excel (V2108, Microsoft, USA) and Statistical Software for the Social Sciences (SPSS; V20, IBM, USA).

Results

Regional protocol

Measured contact area had absolute AErr mean ± S.D. of 0.86 ± 0.41 cm² for S4 and 0.68 ± 0.57 cm² for S10 across both cylinder sizes (Fig 2). When assessed based on cylinder size, the large cylinder absolute AErr was 0.91 ± 0.56 cm² while the small cylinder was 0.63 ± 0.40 cm². Over the 30s duration contact area showed either no change or a minimal increase (0 to 2%) with the exception of one measure day 1, S4, Small cylinder at 50 kPa which exhibited a 6% reduction in measured area. The contact area between-day repeatability demonstrated high values for S4 (ICC = 0.972) and S10 insoles (ICC = 0.939).

Whole insole protocol

Peak pressure was higher than the target pressure in S4 (Absolute MErr Mean ± SD and % Error across all pressure levels: 20.64 ± 13.69 kPa; 7%) and S10 (23.09 ± 15.92 kPa; 9%). Percentage MErr were highest at 50 kPa (13% and 20% respectively) with this latter value in the S10 insole on day two being the only MErr which exceeded our analytical goal of 15% (Fig 3; Table 1). Peak pressure error increased slightly in all trials (0.6 to 2.8%) over the 30s. Mean pressure MErr was negative for S4 (7.42 ± 7.97 kPa; -2%) and S10 (15.21 ± 11.01 kPa; -4%) (Fig 3). Mean pressure MErr as a percentage was largest at 600 kPa pressure (-4% and -6%), but no values exceed our analytical goal (Fig 3; Table 2). Over the 30s duration mean pressures increased slightly in all trials (0.4 to 3%). RMSE across all sensels was lowest in S4 (mean 11.99 kPa, max 27.74 kPa) compared to S10 (mean 18.35 kPa, max 40.46 kPa) and increased in both insoles with target pressure (Fig 3).

Fig 3 — Where circles mark the mean pressure data, error bars denote the Root Mean Square Error (RMSE) across all individual sensels about the mean recorded value and the triangle marks the peak pressure data.

Table 1. Average peak pressure at each target pressure at 2 seconds for Day 1 and Day 2 the differences, and the “ratio limits of agreement” alongside the t-test between the measures of day 1 and 2.

	Measured Values				Log Transformed Calculations
Peak Pressure	Day 1		Day 2		T-Test Day1 Day 2	Ratio Limits
	Mean±SD	%MErr	Mean±SD	%MErr	T-Test Day1 Day 2	Bias limits	Upper	Lower
Size 4 (S4)
50 kPa	53.59±1.04	7%	56.31±0.98	13%	0.01	1.05*/÷1.02	1.07	1.04
100 kPa	102.44±5.20	2%	106.27±2.27	6%	0.33	1.04*/÷1.08	1.13	0.96
200 kPa	204.05±3.83	2%	215.11±4.42	8%	0.09	1.05*/÷1.05	1.10	1.01
300 kPa	303.19±3.88	1%	318.30±4.57	6%	0.01	1.05*/÷1.01	1.06	1.04
400 kPa	413.75±2.78	3%	421.35±1.67	5%	0.04	1.02*/÷1.01	1.03	1.01
500 kPa	515.90±6.69	3%	528.05±5.40	6%	0.23	1.02*/÷1.04	1.06	0.99
600 kPa	626.71±6.91	4%	649.09±2.80	8%	0.03	1.04*/÷1.02	1.05	1.02
Size 10 (S10)
50 kPa	56.55±2.45	13%	60.13±0.18	20%	0.20	1.06*/÷1.10	1.17	0.97
100 kPa	102.10±1.20	2%	105.57±1.94	6%	0.02	1.03*/÷1.01	1.05	1.02
200 kPa	200.02±2.61	0%	211.52±1.85	6%	0.02	1.06*/÷1.02	1.08	1.03
300 kPa	302.83±6.07	1%	315.33±3.43	5%	0.03	1.04*/÷1.02	1.06	1.02
400 kPa	416.53±5.77	4%	428.24±2.42	7%	0.06	1.03*/÷1.02	1.05	1.01
500 kPa	524.70±1.33	5%	539.00±6.54	8%	0.12	1.03*/÷1.03	1.06	1.00
600 kPa	627.11±8.82	5%	651.83±2.56	9%	0.03	1.04*/÷1.02	1.06	1.02

Open in a new tab

Measured values were transformed using natural logarithms to obtain ratio limits of agreement. %MErr: Measurement error, percentage error from the target pressure, Normality: Significance value of Shapiro-Wilk, T-Test: two tailed, paired samples

Table 2. Average mean pressure at each target pressure at 2 seconds for Day 1 and Day 2, the differences, and the “ratio limits of agreement” alongside the t-test between the measures of day 1 and 2.

	Measured Values				Log Transformed Calculations
Mean Pressure	Day 1		Day 2		T-Test Day1 Day 2	Ratio Limits
	Mean±SD	%MErr	Mean±SD	%MErr	T-Test Day1 Day 2	Bias limits	Upper	Lower
Size 4 (S4)
50 kPa	46.13±0.95	-8%	49.36±0.90	-1%	0.02	1.07*/÷1.02	1.10	1.04
100 kPa	94.70±4.72	-5%	98.25±1.40	-2%	0.34	1.04*/÷1.09	1.13	0.96
200 kPa	189.91±4.06	-5%	199.98±3.54	0%	0.10	1.05*/÷1.05	1.11	1.00
300 kPa	283.61±3.25	-5%	297.84±4.17	-1%	0.01	1.05*/÷1.01	1.06	1.04
400 kPa	380.95±2.14	-5%	389.60±1.84	-3%	0.01	1.02*/÷1.01	1.03	1.02
500 kPa	472.52±5.62	-5%	485.38±3.73	-3%	0.16	1.03*/÷1.03	1.06	0.99
600 kPa	560.25±6.93	-7%	577.64±2.16	-4%	0.06	1.03*/÷1.02	1.05	1.01
Size 10 (S10)
50 kPa	46.89±2.07	-6%	49.16±0.14	-2%	0.28	1.05*/÷1.10	1.15	0.96
100 kPa	92.99±0.83	-7%	94.90±1.17	-5%	0.02	1.02*/÷1.01	1.03	1.01
200 kPa	180.42±2.03	-10%	190.67±2.85	-5%	0.03	1.06*/÷1.03	1.09	1.03
300 kPa	275.56±6.81	-8%	287.13±3.20	-4%	0.05	1.04*/÷1.03	1.07	1.01
400 kPa	372.30±4.64	-7%	381.46±2.21	-5%	0.06	1.02*/÷1.02	1.04	1.01
500 kPa	460.20±1.94	-8%	475.44±6.42	-5%	0.12	1.03*/÷1.03	1.07	1.00
600 kPa	542.41±6.99	-10%	564.77±3.20	-6%	0.02	1.04*/÷1.01	1.06	1.03

Open in a new tab

Pressure data from day one and day two is reported for peak (Fig 4; Table 1) and mean (Table 2) pressures alongside reliability measures for the between days comparison assessed at each magnitude of applied pressure (Data is reported in full in S1 File). In all cases a positive systematic bias was observed for measurements between days along with a significant difference between days at many target pressure levels, the bias ratio ranged from 1.02 to 1.06 in peak pressure and 1.02 to 1.07 in mean pressure data. Considering the reliability of the data across all target pressures, for S5 (correlation post log transform of measurement error and mean -0.57, t-test of day 1 and 2 p < .001) and s10 (correlation post log transform of measurement error and mean -0.65, t-test of day 1 and 2 p < .001).

Applying the 95% ratio limits of agreement to evaluate against the MID, for a peak pressure assessed at a 200 kPa level on day 1, it is possible that the XSENSOR system could obtain an estimate as low as 200x1.01 = 202 kPa or as high as 200x1.10 = 220 kPa on day 2 (S4, Table 1). This represents a maximum RErr of 20 kPa, or 10%, which did not exceed our analytical goal and was considerably lower than the MID of 60 kPa or 30% at 200 kPa, for S10 the maximum RErr was lower at 8%. Assessment of the worst-case scenario within mean pressure measures can be made at the 50 kPa level (Table 2). Assuming 50 kPa on day 1 the lower and upper bounds of reassessment would be 50x0.96 = 48 kPa and 50x1.15 = 57.5 kPa on day 2, the maximum RErr was equal to our analytical goal at this level.

The sensors across the insole demonstrated high consistency, particularly for the S4 insole (Table 3). At pressures above 50 kPa the number of sensels within 5% of the mean pressure value was >80% for both insole sizes (Table 3).

Table 3. Percentage of sensels within 5% of mean value for all pressures and times and each insole size and day 1 and 2.

Target pressure		Size 4								Size 10
		Day 1				Day 2				Day 1				Day 2
Magnitude (kPa)	Duration (sec)	0	2	10	30	0	2	10	30	0	2	10	30	0	2	10	30
50		78	77	78	79	80	81	81	81	76	77	77	78	77	76	76	77
100		86	86	87	86	88	88	87	87	84	84	84	84	84	84	85	85
200		87	88	88	88	89	90	89	89	84	84	83	84	83	84	83	84
300		94	95	96	96	96	96	97	97	90	90	91	92	91	91	92	91
400		94	95	95	95	94	94	93	94	87	88	88	88	89	89	89	89
500		93	94	93	94	92	92	93	92	88	88	88	88	89	89	88	89
600		93	93	93	92	90	91	90	91	89	90	90	89	90	89	89	90

Open in a new tab

Discussion

Peak pressure is routinely used within clinical evaluation to establish high-risk regions, however is sensitive to error from single sensels within a discrete region [23]. The grand average measurement error across target loads and days for mean pressure was 4% in S4 and 6% in S10 and for peak pressure was 5% in S4 and 7% in S10. This grand average using the XSENSOR system was comparable to grand average measurement errors reported for Pedar (mean 4.5% and peak 4.8%) and lower than for Medilogic (mean 10.7% and peak 46.2%) and Tekscan (mean 60.5% and peak 193.3%) using the same protocol [21]. Contrasting the results for other insole systems in the prior study [21], mean pressure error increased with increasing target pressure, with very good agreement (<5% error) at low pressures (50–200 kPa) which could improve accuracy of contact area variables. The RMSE mean (12–18 kPa) and max (28–40 kPa) was higher than reported for Pedar (2.5, 4.7 kPa) but comparable to Medilogic (mean 28.5, max 45.7 kPa) and Tekscan (25.5, 41.8 kPa).

Within this study the analysis has extended to assessment with a clinically relevant objective. Within a clinically relevant range (100–400 kPa) [9] XSENSOR insoles recorded peak pressure measurement errors of 1 to 8% and mean pressure measurement errors in the range -10 to 0%. Systematic bias between days was within 2 to 6% for peak and mean pressure while Reliability error (including random error between days) increases this to an upper limit between 3 to 10% for peak pressure and 3 to 11% for mean pressure. Between day reliability showed significant differences when assessed across the whole operating range however at individual pressure levels these differences were not meaningful when assessed against the analytical goal. The overall error observed with XSENSOR is not expected to bias evaluations of efficacy, given that an effective intervention is expected to reduce pressure by more than 30% [5]. Applying this to a case from the literature; Preece et al 2017 observed a 37% reduction from an initial peak plantar pressure level of ~260 kPa by optimising apex position on rocker soled footwear [24]. Using XSENSOR in this setting, a maximum Reliability error (RErr) of 10% would not alter the interpretation of the assessment of these interventions. Mean pressure however consistently underestimated the target pressure, with a larger underestimation at higher pressures, which points to the importance of peak pressure when measuring at-risk feet [1, 23]. However, peak pressures at high pressures (>400kPa) have higher absolute errors, particularly in the S10 insole. Although pressures of this magnitude may be rare in standard shod walking, the increased variability of peak pressure measurement in this range should be considered. Tasks which include more dynamic activities, higher velocities as well as those with low contact areas such as high heels produce peak pressures within these ranges [25]. This is particularly relevant with the trend to real-world data collection which is arising in more recent literature, where peak pressures may exceed those recorded in level walking [26]. The number of sensels reading within 5% of the mean pressure (Table 1), an indicator of the insole’s consistency, was low at 50 kPa (75–80%), but increased to an optimal level (>90%) at 300 kPa target pressure, thereafter reducing but maintaining high numbers of sensels (>87%). The size 10 insole showed lower consistency, which likely underpins the greater variability observed in mean and peak pressure for this insole.

A good representation of applied contact area was observed with the large cylinder for all pressures, with less than 11% error across all pressures. Representation of the small cylinder showed greater surface area error (max 37%) with a tendency to overestimate contact area, suggesting the involvement of additional peripheral sensels. For context in a test with the larger cylinder addition or removal of 1 peripheral sensel would represent 3% error in S4 and 5% error in S10 and as such an 11% error could be said to represent 3–4 sensels. For the small cylinder a 1 sensel change has a much greater effect with errors of 16% in S4 and 24% in S10, representing 2–3 sensels. For comparable conditions (Large 50, Large 200 and Small 200) XSENSOR demonstrated better area estimation than prior systems assessed with the same protocol [21]. This improved accuracy is underpinned by the high spatial resolution of the XSENSOR system with a minimum of 1.5 sensors per cm² (1.5 / cm² for S10, 2.0 / cm² for S4). This represents a higher density than Medilogic (0.79 / cm²) and Pedar (0.57–0.78 / cm²) systems, but a reduced density when compared to Tekscan (3.9 / cm²). The area error is relevant for assessment of localised regions of interest, such as prior ulceration sites, which are typically small plantar areas with high associated pressures. This could impact on both contact area calculation and the application of average pressure measurement. However, these data should be considered alongside the limitations of the protocol and the application of an isolated rigid cylinder which is not a realistic representation of foot anatomy and pressure distribution. The interaction of pressure from multiple contact points between the foot and the ground during gait may negate the activation of boundary sensels and their influence on measured contact area within this regional protocol. However, the observed involvement of these boundary sensels surrounding regions of high pressure suggests a potential to influence measured pressure in neighbouring anatomical regions. Between-day reliability of the system was high when considering regional loading (at 2s) for both cylinder and insole sizes assessed by the ICCs.

The whole insole loading (pressures compared across days) demonstrated measurement error and reliability error within our analytical goals. The regional loading demonstrate high between-day reliability and lower area errors than evident with other pressure systems. This therefore, suggests that the XSENSOR system is appropriate for repeated measures visits which aim to identify localised changes in pressure or compare between interventions across clinical visits for example. Also, that this reliability would not be substantially influenced by the insole size used. These data however, have the limitation that they were collected within controlled laboratory conditions and the reliability of the system for clinical use should be explored in-situ with relevant patients and relevant interventions also.

Limitations

The whole insole protocol utilised a pressurised bladder to apply an even load to the insole surface, however this does not account for the structure of the XSENSOR insole which required some omission of data post-collection. The target pressure varied slightly but was always within 2% of the desired load. The effect of this on the outcomes cannot be isolated. The contact surfaces used to produce the regional loading responses were flat and solid and as such did not fully represent the loading applied by the soft tissues of the foot or the interaction with footwear materials that would be evident in-shoe. The influence of in-shoe factors such as temperature and bending were not considered and may affect the systems differently. The reliability of the system during a gait assessment should be explored.

Supporting information

S1 File. Additional between day figures.

This file contains figures for peak pressure measures between days at each magnitude of applied pressure using the mean of the three trials for each time point (T0, T2, T10, T30).

(DOCX)

Click here for additional data file.^{(1.1MB, docx)}

S2 File. Data underlying findings.

This excel file contains the dataset used for analysis. 5 tabs are presented in including summary data tables for: 1) Day 1 data for the whole insole, 2) Day 2 data for the whole insole, 3) Calculations for peak pressure reliability, 4) Calculations for mean pressure reliability, 5) Regional analysis of contact area.

(XLSX)

Click here for additional data file.^{(73.9KB, xlsx)}

Acknowledgments

XSENSOR Technology Corporation (Calgary, Canada) provided the insole system for testing but had no involvement in the study design, collection, analysis or interpretation of data or the manuscript writing or submission.

Data Availability

All relevant data are within the manuscript and its Supporting information files.

Funding Statement

The author(s) received no specific funding for this work.

References

1.Cavanagh P. R., Hewitt F. G., and Perry J. E., “In-shoe plantar pressure measurement: a review,” Foot, vol. 2, no. 4, pp. 185–194, Dec. 1992. [Google Scholar]
2.Hennig E. M. and Nilani T. L., “In-shoe pressure distribution for running in various types of footwear,” J. Appl. Biomech., vol. 11, no. 3, pp. 299–310, Aug. 1995. [Google Scholar]
3.Milani T. L., Hennig E. M., Meyering S., Diehl R. R., and Berlit P., “Foot pressure patterns of patients with neuromotor disorders,” Gait Posture, vol. 2, no. 4, pp. 246–247, 1994. [Google Scholar]
4.Bus S. A., Haspels R., and Busch-Westbroek T. E., “Evaluation and optimization of therapeutic footwear for neuropathic diabetic foot patients using in-shoe plantar pressure analysis,” Diabetes Care, vol. 34, no. 7, pp. 1595–1600, Jul. 2011. doi: 10.2337/dc10-2206 [DOI] [PMC free article] [PubMed] [Google Scholar]
5.Bus S. A. et al., “Guidelines on the prevention of foot ulcers in persons with diabetes (IWGDF 2019 update),” Diabetes. Metab. Res. Rev., vol. 36, no. S1, Mar. 2020. [DOI] [PubMed] [Google Scholar]
6.Arndt A., “Correction for sensor creep in the evaluation of long-term plantar pressure data,” J. Biomech., vol. 36, no. 12, pp. 1813–1817, Dec. 2003. doi: 10.1016/s0021-9290(03)00229-x [DOI] [PubMed] [Google Scholar]
7.Telfer S., Woodburn J., Collier A., and Cavanagh P. R., “Virtually optimized insoles for offloading the diabetic foot: A randomized crossover study,” J. Biomech., vol. 60, pp. 157–161, Jul. 2017. doi: 10.1016/j.jbiomech.2017.06.028 [DOI] [PubMed] [Google Scholar]
8.Orlin M. N. and McPoil T. G., “Plantar Pressure Assessment,” Phys. Ther., vol. 80, no. 4, pp. 399–409, Apr. 2000. doi: 10.1093/ptj/80.4.399 [DOI] [PubMed] [Google Scholar]
9.Owings T. M. et al., “Plantar pressures in diabetic patients with foot ulcers which have remained healed,” Diabet. Med., vol. 26, no. 11, pp. 1141–1146, Nov. 2009. doi: 10.1111/j.1464-5491.2009.02835.x [DOI] [PubMed] [Google Scholar]
10.Telfer S., Abbott M., Steultjens M., Rafferty D., and Woodburn J., “Dose-response effects of customised foot orthoses on lower limb muscle activity and plantar pressures in pronated foot type,” Gait Posture, vol. 38, no. 3, 2013. doi: 10.1016/j.gaitpost.2013.01.012 [DOI] [PubMed] [Google Scholar]
11.McPoil T. G., Cornwall M. W., and Yamada W., “A comparison of two in-shoe plantar pressure measurement systems,” Low. Extrem., vol. 2, no. 2, pp. 95–103, Jan. 1995. [Google Scholar]
12.Hsiao H., Guan J., and Weatherly M., “Accuracy and precision of two in-shoe pressure measurement systems,” Ergonomics, vol. 45, no. 8, pp. 537–555, Jun. 2002. doi: 10.1080/00140130210136963 [DOI] [PubMed] [Google Scholar]
13.Putti A. B., Arnold G. P., Cochrane L., and Abboud R. J., “The Pedar^® in-shoe system: Repeatability and normal pressure values,” Gait Posture, vol. 25, no. 3, pp. 401–405, Mar. 2007. doi: 10.1016/j.gaitpost.2006.05.010 [DOI] [PubMed] [Google Scholar]
14.Ramanathan A. K., Kiran P., Arnold G. P., Wang W., and Abboud R. J., “Repeatability of the Pedar-X^® in-shoe pressure measuring system,” Foot Ankle Surg., vol. 16, no. 2, pp. 70–73, Jun. 2010. doi: 10.1016/j.fas.2009.05.006 [DOI] [PubMed] [Google Scholar]
15.Murphy D. F., Beynnon B. D., Michelson J. D., and Vacek P. M., “Efficacy of plantar loading parameters during gait in terms of reliability, variability, effect of gender and relationship between contact area and plantar pressure,” Foot Ankle Int., vol. 26, no. 2, pp. 171–179, 2005. doi: 10.1177/107110070502600210 [DOI] [PubMed] [Google Scholar]
16.Woodburn J. and Helliwell P. S., “Observations on the F-Scan in-shoe pressure measuring system,” Clin. Biomech., vol. 11, no. 5, pp. 301–304, Jul. 1996. doi: 10.1016/0268-0033(95)00071-2 [DOI] [PubMed] [Google Scholar]
17.K. Patrick and L. Donovan, “Test–retest reliability of the Tekscan^® F-Scan^® 7 in-shoe plantar pressure system during treadmill walking in healthy recreationally active individuals,” https://doi-org.salford.idm.oclc.org/10.1080/14763141.2017.1355010, vol. 17, no. 1, pp. 83–97, Jan. 2017. [DOI] [PubMed]
18.J. H. Ahroni, E. J. Boyko, and R. Forsberg, “Reliability of F-Scan In-Shoe Measurements of Plantar Pressure:,” http://dx.doi.org.salford.idm.oclc.org/10.1177/107110079801901004, vol. 19, no. 10, pp. 668–673, Jun. 2016. [DOI] [PubMed]
19.Bunting K. V., Steeds R. P., Slater L. T., Rogers J. K., Gkoutos G. V., and Kotecha D., “A Practical Guide to Assess the Reproducibility of Echocardiographic Measurements,” J. Am. Soc. Echocardiogr., vol. 32, no. 12, pp. 1505–1515, Dec. 2019. doi: 10.1016/j.echo.2019.08.015 [DOI] [PubMed] [Google Scholar]
20.Atkinson G. and Nevill A. M., “Statistical methods for assessing measurement error (reliability) in variables relevant to sports medicine,” Sports Med., vol. 26, no. 4, pp. 217–238, 1998. [DOI] [PubMed] [Google Scholar]
21.Price C., Parker D., and Nester C., “Validity and repeatability of three in-shoe pressure measurement systems,” Gait Posture, vol. 46, 2016. doi: 10.1016/j.gaitpost.2016.01.026 [DOI] [PubMed] [Google Scholar]
22.Nevill A. M., Atkinson G., Nevill A. M., and Atkinson G., “Assessing agreement between measurements recorded on a ratio scale in sports medicine and sports science.,” Br. J. Sports Med., vol. 31, no. 4, pp. 314–318, Dec. 1997. doi: 10.1136/bjsm.31.4.314 [DOI] [PMC free article] [PubMed] [Google Scholar]
23.Rosenbaum D. and Becker H.-P., “Plantar pressure distribution measurements. Technical background and clinical applications,” Foot Ankle Surg., vol. 3, no. 1, pp. 1–14, 1997. [Google Scholar]
24.Preece S. J., Chapman J. D., Braunstein B., Brüggemann G.-P., and Nester C. J., “Optimisation of rocker sole footwear for prevention of first plantar ulcer: comparison of group-optimised and individually-selected footwear designs,” J. Foot Ankle Res., vol. 10, no. 1, p. 27, Dec. 2017. doi: 10.1186/s13047-017-0208-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
25.R. E. Snow, K. R. Williams, and G. B. Holmes, “The Effects of Wearing High Heeled Shoes on Pedal Pressure in Women:,” http://dx.doi.org.salford.idm.oclc.org/10.1177/107110079201300206, vol. 13, no. 2, pp. 85–92, Aug. 2016. [DOI] [PubMed]
26.Hulshof C. M., Van Netten J. J., Dekker M. G., Pijnappels M., and Bus S. A., “In-shoe plantar pressure depends on walking speed and type of weight-bearing activity in people with diabetes at high risk of ulceration,” bioRxiv, 2022. [DOI] [PubMed] [Google Scholar]

PLoS One. doi: 10.1371/journal.pone.0277971.r001

Decision Letter 0

Peter Andreas Federolf

21 Jan 2022

PONE-D-21-30139Validity and repeatability of the XSENSOR in-shoe pressure measurement systemPLOS ONE

Dear Dr. Parker,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please attend carefully to both reviewers' comments and address particularly the concerns of reviewer 2.

Please submit your revised manuscript by Mar 07 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Peter Andreas Federolf

Academic Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf”

2. Please note that in order to use the direct billing option the corresponding author must be affiliated with the chosen institute. Please either amend your manuscript to change the affiliation or corresponding author, or email us at plosone@plos.org with a request to remove this option.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: No

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for the opportunity to review the article titled Validity and repeatability of the XSENSOR in-shoe pressure measurement system. This manuscript describes a technical study of an in-shoe pressure measurement system to validate its measurement capabilities. The results of the study indicate that the XSENSOR system provides reasonable estimates of pressure and provide a characterization of bias and reliability.

General Comments

Overall, this is a well-designed and executed study.

The authors’ approach to quantifying accuracy is well founded and executed. However, I think the approach to day-to-day reliability is not as rigorous. I am generally skeptical of ICCs and their use in quantifying reliability as they have many issues that have been identified in literature (Muller and Buttner, 1994). My suggestion would be to reframe the problem of reliability as a hypothesis test with respect to a minimally important difference (MID).

The authors have provided a reference that suggests clinically important changes in pressure would be on the order of 30%. Using this (or other prior work as appropriate), we can define an MID that we would expect to see in a clinical study. If we wish to be conservative, we could use the lower end of the typical range of pressure values, 100 kPa, and the reference of 30% gives us an MID of 30 kPa. We can then take the measurements for day 1 and day 2, across all pressure conditions, and treat these as repeated measures observations. In doing so, we can calculate the distribution of the differences between days and conduct a single-sided hypothesis test with respect to an MID of 30 kPa (the alternate hypothesis being that our estimated mean of differences exceeds 30 kPa).

The advantage of this approach is that it provides a concrete benchmark for evaluating whether the insoles will measure pressure reliably. In my opinion, this is more informative than ICCs, and a more statistically rigorous test of insole reliability. I also suspect that the findings would favour the insoles as being highly reliable.

A table that provides comparative values for the XSENSOR system, similar to Table 1 from Price et al., 2016, might be of interest to the reader.

Specific Comments

The introduction is a bit meandering and some of the information is related, but tangential to the main purpose of the article. There are somewhat contradictory statements as well. It is stated that comparing between different studies/systems is very difficult, while the aim of the current study is to provide such a comparison between XSENSOR and other systems. Yet, this study does not address this difficulty at all; it only provides similar quantifications for the XSENSOR as has already been done for other systems in earlier work. I’d suggest a revision of the introduction to make it more focused on the purpose of the article: evaluating the XSENSOR system for clinical use.

Throughout the manuscript there are no spaces between numbers and units and these should be added throughout (e.g., 400kPa should be 400 kPa).

Line 53: capitalization of diabetic not necessary

Line 60: single quotes around creep not required; this is a standard physics term

Line 106: single quotes around calibration file not required

Line 108: single quotes around active sensels not required

Line 126: this sentence is a little unclear as it may read as though there were two separate cylinders with different bases; perhaps consider rewording

Line 130: “For [each of] the six conditions…”

Line 138 and 144: As mentioned in my general comments, ICCs are not the ideal representation of reliability

Line 144: suggest “performed” instead of “undertaken”

Line numbers stopped after page 10

Page 11, First paragraph, the peak pressures for 100-400 kPa measurements seem to be within 1-8 % of the target measurements, unless there is some averaging done between day 1 and 2.

Page 11, First paragraph, “… interact with evaluations of efficacy…” might be clearer as “… bias evaluations of efficacy…”

Page 11, First paragraph, while some of the mean measurements did indeed show less error than peaks, this was not universally true; I suggest rephrasing this statement

Page 13, Second paragraph, given that the testing in this study was done under very controlled laboratory conditions, the evaluation of reliability should reflect the caveat that calibration and measurement under in situ conditions might be less reliable than what has been shown

Figure 1, the scales for the grids in this figure is not obvious, and if I am correct, the scales are not identical between S4 and S10, even though they appear to be equal in this figure. I would suggest either adding scales, or at least clarifying that the graphical representations of contact area are not to scale.

Figure 2, superscript required for ‘cm2’ in the y-axis label.

Figure 3, the vertical alignment of all plots makes reading the results for the 500 and 600 kPa tests difficult. I would suggest staggering the results slightly left and right to make reading them easier to read.

Supplementary file, the Excel file could use a little formatting/clean-up to make it easier to interpret; for example, the headers for each section could be bolded/formatted as table headings; some headers are bolded already, but many aren’t

Reviewer #2: General comments:

This manuscript investigates the validity and day-to-day repeatability of an XSensor in-shoe pressure-sensing insoles during standardized loading protocols in comparison to known pressures and contact areas.

Overall, I agree with the authors on the importance for independent research groups to confirm the validity and reliability of commercial measurement systems. I also get the impression that the measurements were carried out carefully. In an original research article, however, the generated data should be presented and discussed in a way that provides value to other researchers. In my opinion, the submitted manuscript is lacking a clear structure, the investigated outcome variables are not well justified, and there are no criteria for what constitutes sufficient validity and reliability. My main critical points are:

1) The outcome variables are not well prepared and justified in the introduction. The authors talk about mean and peak pressure over varying duration, contact areas at different pressures, creep, insole consistency, etc.. It is unclear, why each variable is of interested and in which context each variable is relevant. The aim statement should clearly state the variables that the authors plan to use to determine the insoles’ validity and repeatability. Then, the variables should be described in the methods. Particularly, the authors should avoid to report results that were not prepared in the methods (creep, insole consistency).

2) What constitutes sufficient validity and repeatability? The authors talk about various applications of pressure-sensing insoles, e.g. running biomechanics [2] or in patients with neuromotor disorders [3-5]. Comparing pressures between slightly modified running shoes during running OR comparing pressures between healthy vs. diseased patients during standing or walking will lead to very different requirements of validity and repeatability for a pressure-sensing system. Therefore, the authors cannot make a sweeping statement in the abstract to say that “the Xsensor system is appropriate for clinical assessment that require multiple assessments”.

Therefore, the criteria that indicate sufficient validity and repeatability need to be developed carefully in the context of a certain research area and are only relevant in this context. Those criteria should not only be based on relative measures (e.g. relative error or ICC). Such measures certainly have their strengths but also weaknesses (e.g. Koo, Terry K., and Mae Y. Li. 2016. “A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research.” Journal of Chiropractic Medicine 15 (2): 155–63.)

The authors should consider including absolute measures such as Bland-Altman limits of agreement (see Atkinson, G., and A. M. Nevill. 1998. “Statistical Methods for Assessing Measurement Error (reliability) in Variables Relevant to Sports Medicine.” Sports Medicine 26 (4): 217–38.). Limits of agreement would provide an intuitive way of determining whether the measurement error grows with increasing pressure or measurement duration.

Finally, the authors should come up with clinically relevant or meaningful changes in pressure in absolute physical units (or maybe relative to body weight) that occur in a certain research context and use those values to build a framework for validity and repeatability.

3) The authors criticize previous studies for their limited external validity. In this experiment however, there is no information on the loading rate. I assume that the pressure was built up slowly and was not comparable to a pressure profile to running or walking. Is there anyway for your system to simulate a dynamic loading profile? Currently, I would argue that the results of this manuscript are relevant for standing only.

Specific comments:

Line 72-73: ICCs of around 0.5 are not usually considered to show “high intra class correlation”.

Line 73-81: The relevance of previous findings on the Tekscan system is unclear for this study on the XSensor system. It would seem more relevant to explain if some measurement systems have a larger error than clinically relevant underfoot pressure changes in clinical populations.

Line 95: It is unclear how the “suitability for clinical and biomechanical assessment” is defined. There should be a-prior standards about the validity and repeatability for their use in clinical research and biomechanical research.

Line 99: When looking at the XSensor webpage (https://xsensor.com/), the available systems are “Intelligent Insoles Clinical” and “Intelligent Insoles Pro”. Please explain how the XSensor X4 system relates to these two available systems.

Line 115: How long were the measurements?

Line 117: Where exactly were those sensels and how many were excluded?

Line 118: sensel underlined?

Line 125: Please briefly describe the testing device to generate pressure during the regional protocol. How accurate is this device?

Line 129: How long were the measurements in the regional protocol?

Line 130: Where was the regional pressure applied?

Line 138: This is the first time that a between-day study design is mentioned. This must be mentioned in the aims of this study and at the beginning of the methods. In general, while it is appreciated that the previously developed protocol is cited [reference 18], the most important aspects of this protocol must still be mentioned. Like how many days passed between the measurements, etc.? Why were the durations 0,2, 10, 30 seconds selected? What is a duration of 0 seconds anyway? How exactly were relative and absolute errors calculated (e.g. the ones in Table 2)?

Lines 134-144: This section is difficult to follow. For example: What happened to the different pressures in the regional protocol (50, 110, 200 kPa)? Why were peak pressures investigated? Is “load” equal to pressure? Why did the authors only investigate ICCs as a relative measure of reliability? In my opinion, absolute measures of repeatability and validity such as Bland-Altman limits of agreement would be much more intuitive in their interpretation and it would be easier to observe changes in agreement with the known measures as a function of held duration or applied pressure. This section needs to be structured better and the analysis approach should be justified.

Line 143: Please explain exactly what type of ICC model was used and how the individual values were combined into the respective results. Different ICC models can lead to different results given the same data.

Figure 1: Please add a color scale so the reader can assess changes in pressure. Please indicate where in Figure 1a the regional pressures were applied.

Line 154: Why is there no analysis of the measured pressures in the regional protocol? This type of regional loading may affect the insole’s validity in measuring the correct mean and peak pressure.

Table 1: There is no description of this analysis in the methods.

Lines 160-167: It is unclear where those values come from? (e.g. S10 (23.09±15.92kPa; 9%)). Are those averages?

Line 164-165: There has been no mention of a time-dependent analysis of peak pressure in the methods section.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Maurice Mohr

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Attachment

Submitted filename: Reviewer comments.pdf

Click here for additional data file.^{(148.9KB, pdf)}

PLoS One. 2023 Jan 17;18(1):e0277971. doi: 10.1371/journal.pone.0277971.r002

Author response to Decision Letter 0

22 Jul 2022

We would like to thank the reviewers for their detailed and considered comments on our manuscript. We have made significant changes to the manuscript in response to comments which we believe has improved the quality and clarity of the data presented. Within this revision we have identified key changes which required reprocessing of the data. To prevent addition of similar and repeating figures or tables we have replaced prior versions with a more appropriate presentation of the data. We have also expanded the supplementary data which provides additional information to complement the results.

We have two sections which respond to key points raised by the reviewers and are addressed below. We also have referred to these in the comments where they are relevant rather than repeating ideas and responses.

Reliability response:

We appreciate the detailed review and critique from the reviewers and their constructive suggestion for improvements in our assessment of reliability. Following from this we have utilised a further assessment to assess reliability of peak and mean pressure values from the whole insole protocol, which allows us to assess these potential errors against a clinically relevant value (please see diabetes response below).

For the regional protocol we do not have a similar threshold of acceptable error as we do not have a known value where contact area error has clinical relevance. Further to this the data collected in this study does not allow assessment across multiple participants and so random error in this case is determined by a fixed count of sensors which overlap the indenter perimeter. This has led to us maintaining our approach with ICC for the assessment of reliability in the regional protocol, which has not been updated in the manuscript. The whole insole approach however is now described in detail in the methods with new comparisons of day 1 and day 2 presented in the results.

As the purpose of this manuscript is to assess the suitability of the XSENSOR system for clinical measurement a clinically relevant analytical goal was deemed more suitable than a formal hypothesis test for the assessment of the measurement errors acceptability.

The approach described below and in the methods is taken from:

- G. Atkinson and A. M. Nevill, “Statistical methods for assessing measurement error (reliability) in variables relevant to sports medicine,” Sports Med., vol. 26, no. 4, pp. 217–238, 1998.

- A. M. Nevill, G. Atkinson, A. M. Nevill, and G. Atkinson, “Assessing agreement between measurements recorded on a ratio scale in sports medicine and sports science.,” Br. J. Sports Med., vol. 31, no. 4, pp. 314–318, Dec. 1997.

• Day one to day two was compared to each other and both were compared to the known pressure we applied through the TruBlue device. A percentage error was calculated.

• A T-test and correlation compared bias between day 1 and day 2 measurements.

• Bland-Altman plots with 95% agreement displayed disagreement between tests.

• This data included heteroscedastic error and therefore natural logarithms were taken.

• Bias ratio was calculated based on the antilog of the mean difference between days.

• Limits of agreement were calculated as agreement ratios between which 95% of difference between days can be calculated

• maximum reliability error (systematic bias and random error) was assessed against an analytical goal of 15%

This assessment of the data has enabled us to describe in the results and discussion the limits of agreement with reference to the MID and an analytical goal. Therefore, with relevance to the clinical assessment of patients with diabetes (please see response below).

Clinical assessment in Diabetes response:

We have narrowed down the aims of the research study to focus on diabetes and therefore have contextualised the reliability and validity of pressures systems with relevance to this application of the technology:

There are two IWGDF guidelines which refer to expectations of pressure reduction.

- 30% reduction is threshold for effectiveness in offloading footwear and interventions

- 200 kPa is a threshold for risk of recurrent ulceration

We have incorporated these into our paper introduction and described our method for application and implementation of these in the method. Using the 30% reduction guideline we have defined an analytical goal of 15% reliability error (systematic bias and random error) such that each days measured value should not be affected by more than half of the objective reduction.

Specific responses to reviewer comments (also provided in response to reviewers document):

Reviewer #1

R1C1. The authors’ approach to quantifying accuracy is well founded and executed. However, I think the approach to day-to-day reliability is not as rigorous. I am generally skeptical of ICCs and their use in quantifying reliability as they have many issues that have been identified in literature (Muller and Buttner, 1994). My suggestion would be to reframe the problem of reliability as a hypothesis test with respect to a minimally important difference (MID).

We thank the reviewer for their thorough and constructive reviews. Please see responses below that reference these points in more detail. In short: we have adopted this approach of defining a clinically relevant MID and testing the outcomes from our protocol against this.

R1C2. The authors have provided a reference that suggests clinically important changes in pressure would be on the order of 30%. Using this (or other prior work as appropriate), we can define an MID that we would expect to see in a clinical study. If we wish to be conservative, we could use the lower end of the typical range of pressure values, 100 kPa, and the reference of 30% gives us an MID of 30 kPa. We can then take the measurements for day 1 and day 2, across all pressure conditions, and treat these as repeated measures observations. In doing so, we can calculate the distribution of the differences between days and conduct a single-sided hypothesis test with respect to an MID of 30 kPa (the alternate hypothesis being that our estimated mean of differences exceeds 30 kPa).

An analytical goal approach was employed in place of a formal hypothesis test.

Please see the response to key points with more detail about the reliability assessment and the clinical relevance.

R1C3. A table that provides comparative values for the XSENSOR system, similar to Table 1 from Price et al., 2016, might be of interest to the reader.

We provide the details of the XSENSOR system within the first line of the method and haven’t included a table as there is nothing to compare this to within this paper.

We have expanded this content to include more technical information to make the information provided comparable to that in the prior paper.

R1C4. Introduction

There are somewhat contradictory statements as well. It is stated that comparing between different studies/systems is very difficult, while the aim of the current study is to provide such a comparison between XSENSOR and other systems. Yet, this study does not address this difficulty at all; it only provides similar quantifications for the XSENSOR as has already been done for other systems in earlier work. I’d suggest a revision of the introduction to make it more focused on the purpose of the article: evaluating the XSENSOR system for clinical use.

The content of the introduction and aims have been amended and altered as suggested. This is now to focus on clinical assessments in diabetes and appropriate standards of validity and repeatability to enable this. The detail of the introduction has been increased to more specifically relate to variables of interest in a clinical context and then, more specifically, in Diabetes. We have continued to include a range of variables (such as creep, consistency) in the introduction to provide context in terms of the work already done to validate pressure systems. The variables utilised within this work are identified more clearly as clinically relevant and then fully defined and described in the methods.

R1C5. Throughout the manuscript there are no spaces between numbers and units and these should be added throughout (e.g., 400kPa should be 400 kPa).

The manuscript has been amended as suggested

R1C6. Line 53: capitalization of diabetic not necessary

The manuscript has been amended as suggested

R1C7. Line 60: single quotes around creep not required; this is a standard physics term

The manuscript has been amended as suggested

R1C8. Line 106: single quotes around calibration file not required

The manuscript has been amended as suggested

R1C9.Line 108: single quotes around active sensels not required

The manuscript has been amended as suggested

R1C10. Line 126: this sentence is a little unclear as it may read as though there were two separate cylinders with different bases; perhaps consider rewording

Wording has been changed to:

“Two cylinders were used to apply regional loads in this study. Each cylinder was made of rigid plastic and had a 1mm layer of PORON at its base.”

R1C11. Line 130: “For [each of] the six conditions…”

Amended as suggested

R1C12. Line 138 and 144: As mentioned in my general comments, ICCs are not the ideal representation of reliability

Please see full reliability response above.

R1C13. Line 144: suggest “performed” instead of “undertaken”

Amended as suggested

R1C14. Line numbers stopped after page 10

We apologise for this error; line numbering has been corrected in the revised submission.

R1C15. Page 11, First paragraph, the peak pressures for 100-400 kPa measurements seem to be within 1-8 % of the target measurements, unless there is some averaging done between day 1 and 2.

This has been updated to reflect the range across both days.

“insoles recorded peak pressure measurement errors of 1-8%.”

R1C16. Page 11, First paragraph, “… interact with evaluations of efficacy…” might be clearer as “… bias evaluations of efficacy…”

The manuscript has been amended as suggested

R1C16. Page 11, First paragraph, while some of the mean measurements did indeed show less error than peaks, this was not universally true; I suggest rephrasing this statement

This paragraph has been reworded to more clearly represent the full range presented in figure 2 when observed across both days.

R1C17. Page 13, Second paragraph, given that the testing in this study was done under very controlled laboratory conditions, the evaluation of reliability should reflect the caveat that calibration and measurement under in situ conditions might be less reliable than what has been shown

Detail added to paragraph

“These data however, have the limitation that they were collected within controlled laboratory conditions and the reliability of the system for clinical use should be explored in-situ with relevant patients also.”

R1C18. Figure 1, the scales for the grids in this figure is not obvious, and if I am correct, the scales are not identical between S4 and S10, even though they appear to be equal in this figure. I would suggest either adding scales, or at least clarifying that the graphical representations of contact area are not to scale.

We have now revised figure 1 to include a clearer representation of the data.

Size 10 insoles each pixel represents 0.82cm x 0.82cm while Size 4 insoles each pixel size represents 0.7cm x 0.7cm.

To allow direct comparison the images have now been scaled to reflect the true dimensions. To further aid comparison to the target indenter area a scale outline of the indenter has been added centred on the middle of the active area.

R1C19. Figure 2, superscript required for ‘cm2’ in the y-axis label.

The figure has been amended as suggested

R1C20. Figure 3, the vertical alignment of all plots makes reading the results for the 500 and 600 kPa tests difficult. I would suggest staggering the results slightly left and right to make reading them easier to read.

Figure has been amended. Data has been staggered slightly such that it is offset for each pressure increment within each time of application to prevent the overlap of standard deviation bars and peak values.

R1C21. Supplementary file, the Excel file could use a little formatting/clean-up to make it easier to interpret; for example, the headers for each section could be bolded/formatted as table headings; some headers are bolded already, but many aren’t

Formatting has been improved throughout the document.

Reviewer #2: General comments:

R2C1. The outcome variables are not well prepared and justified in the introduction. The authors talk about mean and peak pressure over varying duration, contact areas at different pressures, creep, insole consistency, etc.. It is unclear, why each variable is of interested and in which context each variable is relevant. The aim statement should clearly state the variables that the authors plan to use to determine the insoles’ validity and repeatability. Then, the variables should be described in the methods. Particularly, the authors should avoid to report results that were not prepared in the methods (creep, insole consistency).

We have continued to include a range of variables (such as creep, consistency) in the introduction to provide context in terms of the work already done. The detail of the introduction has been increased to more specifically relate to variables that are of interest in a clinical context and then, more specifically, in Diabetes.

The plantar pressure variables utilised within this work are identified more clearly as clinically relevant and then fully defined and described in the methods, aligning each outcome measure with the purpose for it’s conclusion.

R2C2. What constitutes sufficient validity and repeatability? The authors talk about various applications of pressure-sensing insoles, e.g. running biomechanics [2] or in patients with neuromotor disorders [3-5]. Comparing pressures between slightly modified running shoes during running OR comparing pressures between healthy vs. diseased patients during standing or walking will lead to very different requirements of validity and repeatability for a pressure-sensing system. Therefore, the authors cannot make a sweeping statement in the abstract to say that “the Xsensor system is appropriate for clinical assessment that require multiple assessments”.

We have slightly reworded this sentence at the end of the abstract to say it is ‘likely’. The sentence was specific to the repeatability between day as described by the clinical assessments that require multiple assessments. We hope that the addition of the more detail exploration of reliability as recommended by the reviewers also further supports this statement.

R2C3. Therefore, the criteria that indicate sufficient validity and repeatability need to be developed carefully in the context of a certain research area and are only relevant in this context. Those criteria should not only be based on relative measures (e.g. relative error or ICC). Such measures certainly have their strengths but also weaknesses (e.g. Koo, Terry K., and Mae Y. Li. 2016. “A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research.” Journal of Chiropractic Medicine 15 (2): 155–63.)

Please see full reliability response.

R2C4. Finally, the authors should come up with clinically relevant or meaningful changes in pressure in absolute physical units (or maybe relative to body weight) that occur in a certain research context and use those values to build a framework for validity and repeatability.

As detailed above, we have added further detail from prior research relating to absolute measures that are relevant. We have also increased the specificity of the research question to relate to clinical assessment in diabetes to ensure that the framework for reliability, validity is clear and focussed on MID.

R2C5. The authors criticize previous studies for their limited external validity. In this experiment however, there is no information on the loading rate. I assume that the pressure was built up slowly and was not comparable to a pressure profile to running or walking. Is there anyway for your system to simulate a dynamic loading profile? Currently, I would argue that the results of this manuscript are relevant for standing only.

Loading was applied as quickly as possible but was done by hand (using a pressure release valve) so was not at a controlled rate. Detail has been added to the method to clarify this:

“Each pressure was applied manually as quickly as possible using a hand release valve to inflate a bladder.”

This was not at the same rate as unloading and loading the foot would be in stance for walking or running. But took approximately 1-2 seconds to be applied. From our prior studies (Price 2016) we would not expect drift in pressure systems over time to affect the measured pressure.

Using this device, it is not possible to simulate a dynamic loading profile. As we are applying load to the target rapidly over a short period of time (typically 2s) we believe that the instant of target pressure and post 2s of held load have relevance to gait

While we agree longer duration loading (10s and over) are relevant to standing the instant of and post 2s measures can be associated with gait in healthy and pathological cohorts. The 2s cohort has been selected for the main analysis as this is unlikely to be affected by any instant reaction to loading or by system drift over the short duration.

Collecting data over a longer duration also enables researchers to be aware of how protocols with more prolonged loading may influence the reliability and validity. This is relevant as more real world data collection occurs and the influence of other tasks (such as standing) potentially becomes a relevant aspect of potential intervention effectiveness tasks in the future.

R2C6. Line 72-73: ICCs of around 0.5 are not usually considered to show “high intra class correlation”.

Removed as per removal of ICC.

R2C7. Line 73-81: The relevance of previous findings on the Tekscan system is unclear for this study on the XSensor system. It would seem more relevant to explain if some measurement systems have a larger error than clinically relevant underfoot pressure changes in clinical populations.

We have added further detail relating to the requirements of systems for clinical assessment to the introduction. This includes values and numbers that offer context to these numbers for direct comparison.

R2C8. Line 95: It is unclear how the “suitability for clinical and biomechanical assessment” is defined. There should be a-prior standards about the validity and repeatability for their use in clinical research and biomechanical research.

This section has been rewritten with an analytical goal based on a relevant meaningful difference. This is now defined within the methods and presented within the results as per the suggestion of reviewer one.

R2C9. Line 99: When looking at the XSensor webpage (https://xsensor.com/), the available systems are “Intelligent Insoles Clinical” and “Intelligent Insoles Pro”. Please explain how the XSensor X4 system relates to these two available systems.

The XSENSOR X4 is the insole sensor version.

Both the clinical and pro systems supplied by XSENSOR use the same insole sensors used within this study. The difference between systems is the on-shoe hub device which has increased memory capacity in the pro version.

R2C10. Line 115: How long were the measurements?

Further detail has been added to the regional protocol section to define the measurements as 30 seconds long.

R2C11. Line 117: Where exactly were those sensels and how many were excluded?

Sentence reworded to include location as:

Based on manufacturer guidance some sensels on the insole periphery were excluded due to raised cabling enclosure which affected the even application of pressure.

R2C12. Line 118: sensel underlined?

Underline removed

R2C13. Line 125: Please briefly describe the testing device to generate pressure during the regional protocol. How accurate is this device?

The following details have been added

“Two cylinders were used to apply regional loads in this study. Each cylinder was made of rigid plastic and had a 1mm layer of PORON at its base.”

“Cylinders were loaded by the application weights to their top surface and supported by a vertical linear bearing to ensure load through their centre to generate realistic pressures for these anatomical regions during stance of 50, 110 and 200 kPa.”

Pressures applied were based on loads divided by contact area for each indenter. Mass applied by the cylinder was validated using a force plate. Measurement of absolute pressure was not the intended outcome of this assessment and the three load levels were chosen to represent low moderate and high load for each area.

R2C14. Line 129: How long were the measurements in the regional protocol?

The duration of recording for the regional protocol has been added to the method.

“three measurements were recorded of 30 seconds,”

R2C15. Line 130: Where was the regional pressure applied?

Detail has been added to describe the location of the device in the rearfoot, where we would anticipate the heel to be.

This has also been included in Figure 1.

R2C16. Line 138: This is the first time that a between-day study design is mentioned. This must be mentioned in the aims of this study and at the beginning of the methods. In general, while it is appreciated that the previously developed protocol is cited [reference 18], the most important aspects of this protocol must still be mentioned. Like how many days passed between the measurements, etc.? Why were the durations 0,2, 10, 30 seconds selected? What is a duration of 0 seconds anyway? How exactly were relative and absolute errors calculated (e.g. the ones in Table 2)?

The introduction now includes further detail relating to within- and between day in relevant sections including the aim.

The methods section has been made more detailed and includes further detail of the original protocol.

The testing was undertaken on consecutive days.

The duration of 0 seconds was a measure that was taken as soon as the pressure insole was loaded to the correct pressure magnitude for each measure. As described in response to the other reviewer comment, this was around 1- 2 seconds. This has now been described in the regional protocol section:

“The start point for measurement was when the target pressure was achieved (monitored by a pressure gauge “

and the variables section of the method.

“T0 reflected a measure at the first instant at which the required pressure had been applied and the other T values added to this.”

Within this manuscript to align to the focus of use for clinical interpretation of risk we have used the 2s time point as discussed in response to the comment above.

The longer durations (T10 & T30) were selected to reflect changes with the development in technology in the field, potable assessment of pressure is more viable and more common. Therefore, we undertook a data collection which facilitated generalising results to longer stance times than a walking gait, such as standing tasks which are also relevant to loading the at-risk foot. We refer to this literature in the discussion of the paper where we also address higher pressures for example:

This is particularly relevant with the trend in more real-world data collection which is arising in more recent literature, where peak pressures may exceed those recorded in walking

The calculation of the specific errors has been added to the method, variables section for clarity:

“Peak and mean pressure error were calculated based on the absolute value versus the target pressure, error was also presented as a percentage of the target pressure”.

R2C17. Lines 134-144: This section is difficult to follow. For example: What happened to the different pressures in the regional protocol (50, 110, 200 kPa)? Why were peak pressures investigated? Is “load” equal to pressure? Why did the authors only investigate ICCs as a relative measure of reliability? In my opinion, absolute measures of repeatability and validity such as Bland-Altman limits of agreement would be much more intuitive in their interpretation and it would be easier to observe changes in agreement with the known measures as a function of held duration or applied pressure. This section needs to be structured better and the analysis approach should be justified.

The methods section has been made more detailed and includes further detail of the original protocol. This details the assessment of mean and peak pressures.

Re ICC: Please see full reliability response.

R2C18. Line 143: Please explain exactly what type of ICC model was used and how the individual values were combined into the respective results. Different ICC models can lead to different results given the same data.

This text has been removed alongside other amendments and removal of ICC. Please see full reliability response.

R2C19. Figure 1: Please add a colour scale so the reader can assess changes in pressure. Please indicate where in Figure 1a the regional pressures were applied.

Figure 1 has been updated

For Figure 1 (a) has been updated to include an outline of the indenter location on each insole.

For Figure 1 (b) this shows the difference from the target pressure in % (which lies in the range of +/-15%)

For Figure 1 (c) a colour bar has been added showing the variation in load across the indenter area this is represented in % with full scale of +/-100%. As the number of active sensors and cumulative area is the outcome of interest in this image a to scale outline of the indenter has been added centred on the middle of the active area.

R2C20. Line 154: Why is there no analysis of the measured pressures in the regional protocol? This type of regional loading may affect the insole’s validity in measuring the correct mean and peak pressure.

The regional protocol is intended to assess loaded contact area under different applied pressures.

As described above pressures were produced by application of a mass to the cylinder to represent a low moderate and high load over the cylinder’s contact area.

The application of different pressures allows us to assess whether the activation or sensitivity of sensors overlapping the border of the cylinder is dependent on the applied load. Therefore whether the contact area measures of the insoles are reliant on the magnitude of the pressure being applied over the areas.

While in theory It would be possible to calculate a total force applied across the contact area this is not the purpose of this test.

R2C21. Table 1: There is no description of this analysis in the methods.

The analysis of the data to derive this data has been added to the method, in the variables section:

“finally, the percentage of sensors within 5% of the insole mean was calculated to allow assessment of variation across the test insole.”

R2C22. Lines 160-167: It is unclear where those values come from? (e.g. S10 (23.09±15.92kPa; 9%)). Are those averages?

We have defined these variables more precisely in the method, as previously described. Also we have improved he description in the sentence and added a reference to a figure and a table which both include the relevant data:

“Peak pressure was higher than the target in S4 (Mean ± SD and Error from target across all pressure levels: 20.64 ± 13.69 kPa; 7%) and S10 (23.09 ± 15.92 kPa; 9%) with errors from the target highest at 50 kPa (13% and 20% respectively) (Figure 3; Table 1).”

R2C23. Line 164-165: There has been no mention of a time-dependent analysis of peak pressure in the methods section.

The method and results have been restructured and further detail added with clear definition of main outcome variables and presentation of results.

Attachment

Submitted filename: Response to reviewers.docx

Click here for additional data file.^{(47.5KB, docx)}

PLoS One. doi: 10.1371/journal.pone.0277971.r003

Decision Letter 1

Peter Andreas Federolf

14 Sep 2022

PONE-D-21-30139R1Validity and reliability of the XSENSOR in-shoe pressure measurement systemPLOS ONE

Dear Dr. Parker,

Thank you for submitting your manuscript to PLOS ONE. We invite you to submit a revised version of the manuscript that addresses the remainig points raised during the review process. Both reviewers point out a number of small issues. Please adress them carefully.

Please submit your revised manuscript by Oct 29 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

We look forward to receiving your revised manuscript.

Kind regards,

Peter Andreas Federolf

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #2: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Reviewer #1: Thank you to the authors for their comprehensive response to the comments. I agree that the manuscript is much improved and I appreciate them taking the time to revise the content and analysis.

I have only a few minor remaining comments based on the revised version with changes highlighted

Line 76: “discrete” misspelled

Line 97: “that” seems superfluous

Line 98: wording is a bit unclear; is it supposed to be “highly correlated relationships”?

Line 108: capitalization of international not needed

Line 120: missing a period.

Lines 142-143: inclusion of pressure gauge company redundant with information in previous paragraph.

Line 145: change to “were calculated”

Lines 224-225: I am confused by the statement that no measurements exceeded the analytical goal, when one of the 50 kPa measurements was off by 20%.

Line 340: change to “these data”

Reference 11: misspelled “pressure”

Reviewer #2: General comments:

I would like to congratulate the authors on the extensive revisions to the manuscript. The inclusion of Bland-Altman limits of agreement and a clinically meaningful reliability target has significantly increased the value of this manuscript to the readership. Please consider a few remaining but minor comments/questions below. ^

Specific comments:

Line 69: discrete

Line 74: average pressure across sensors within a discrete region

Line 96-102: I am still under the impression that this part of the introduction does not add value to the manuscript. The importance of evaluating the contact area has been demonstrated further up already. I do not understand the meaning of “Additionally, that when considering repeated measures most systems demonstrate high relationships between the two days or measures”.

Line 173: intra-class correlation coefficient

Line 220-221: This sentence sounds like a contradiction. 20% > 15%

Figure 1: Is there any reason for the measurement at 100kPA showing the opposite trend of all other loading conditions? On average Figure 3 does not seem to show this opposite behavior at 100kPA.

Figure 4: (c) and (d) labels are flipped in the figure

Table 1&2: Does the correlation column represent the correlations before or after log-transformation? After transformation, the correlations should be much lower or rather non-existent. Also, there are strong negative and positive correlations (e.g. Table 1, s10, 100kPA vs. 500kPA). Do the authors have an explanation for this? I would expect a consistent trend where d2 shows higher pressures than d1 across all loading conditions?

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Maurice Mohr

**********

Attachment

Submitted filename: Reviewer comments_Round2.pdf

Click here for additional data file.^{(95.3KB, pdf)}

PLoS One. 2023 Jan 17;18(1):e0277971. doi: 10.1371/journal.pone.0277971.r004

Author response to Decision Letter 1

25 Oct 2022

We would like to thank the reviewers for their careful review of this manuscript.

please see response to reviewer document attached to this submission.

Attachment

Submitted filename: Response to Reviewers - PONE-D-21-30139.docx

Click here for additional data file.^{(27.1KB, docx)}

PLoS One. doi: 10.1371/journal.pone.0277971.r005

Decision Letter 2

Peter Andreas Federolf

8 Nov 2022

Validity and reliability of the XSENSOR in-shoe pressure measurement system

PONE-D-21-30139R2

Dear Dr. Parker,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Peter Andreas Federolf

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

PLoS One. doi: 10.1371/journal.pone.0277971.r006

Acceptance letter

Peter Andreas Federolf

25 Nov 2022

PONE-D-21-30139R2

Validity and reliability of the XSENSOR in-shoe pressure measurement system

Dear Dr. Parker:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Peter Andreas Federolf

Academic Editor

PLOS ONE

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

S1 File. Additional between day figures.

This file contains figures for peak pressure measures between days at each magnitude of applied pressure using the mean of the three trials for each time point (T0, T2, T10, T30).

(DOCX)

Click here for additional data file.^{(1.1MB, docx)}

S2 File. Data underlying findings.

(XLSX)

Click here for additional data file.^{(73.9KB, xlsx)}

Attachment

Submitted filename: Reviewer comments.pdf

Click here for additional data file.^{(148.9KB, pdf)}

Attachment

Submitted filename: Response to reviewers.docx

Click here for additional data file.^{(47.5KB, docx)}

Attachment

Submitted filename: Reviewer comments_Round2.pdf

Click here for additional data file.^{(95.3KB, pdf)}

Attachment

Submitted filename: Response to Reviewers - PONE-D-21-30139.docx

Click here for additional data file.^{(27.1KB, docx)}

Data Availability Statement

All relevant data are within the manuscript and its Supporting information files.

[pone.0277971.ref001] 1.Cavanagh P. R., Hewitt F. G., and Perry J. E., “In-shoe plantar pressure measurement: a review,” Foot, vol. 2, no. 4, pp. 185–194, Dec. 1992. [Google Scholar]

[pone.0277971.ref002] 2.Hennig E. M. and Nilani T. L., “In-shoe pressure distribution for running in various types of footwear,” J. Appl. Biomech., vol. 11, no. 3, pp. 299–310, Aug. 1995. [Google Scholar]

[pone.0277971.ref003] 3.Milani T. L., Hennig E. M., Meyering S., Diehl R. R., and Berlit P., “Foot pressure patterns of patients with neuromotor disorders,” Gait Posture, vol. 2, no. 4, pp. 246–247, 1994. [Google Scholar]

[pone.0277971.ref004] 4.Bus S. A., Haspels R., and Busch-Westbroek T. E., “Evaluation and optimization of therapeutic footwear for neuropathic diabetic foot patients using in-shoe plantar pressure analysis,” Diabetes Care, vol. 34, no. 7, pp. 1595–1600, Jul. 2011. doi: 10.2337/dc10-2206 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0277971.ref005] 5.Bus S. A. et al., “Guidelines on the prevention of foot ulcers in persons with diabetes (IWGDF 2019 update),” Diabetes. Metab. Res. Rev., vol. 36, no. S1, Mar. 2020. [DOI] [PubMed] [Google Scholar]

[pone.0277971.ref006] 6.Arndt A., “Correction for sensor creep in the evaluation of long-term plantar pressure data,” J. Biomech., vol. 36, no. 12, pp. 1813–1817, Dec. 2003. doi: 10.1016/s0021-9290(03)00229-x [DOI] [PubMed] [Google Scholar]

[pone.0277971.ref007] 7.Telfer S., Woodburn J., Collier A., and Cavanagh P. R., “Virtually optimized insoles for offloading the diabetic foot: A randomized crossover study,” J. Biomech., vol. 60, pp. 157–161, Jul. 2017. doi: 10.1016/j.jbiomech.2017.06.028 [DOI] [PubMed] [Google Scholar]

[pone.0277971.ref008] 8.Orlin M. N. and McPoil T. G., “Plantar Pressure Assessment,” Phys. Ther., vol. 80, no. 4, pp. 399–409, Apr. 2000. doi: 10.1093/ptj/80.4.399 [DOI] [PubMed] [Google Scholar]

[pone.0277971.ref009] 9.Owings T. M. et al., “Plantar pressures in diabetic patients with foot ulcers which have remained healed,” Diabet. Med., vol. 26, no. 11, pp. 1141–1146, Nov. 2009. doi: 10.1111/j.1464-5491.2009.02835.x [DOI] [PubMed] [Google Scholar]

[pone.0277971.ref010] 10.Telfer S., Abbott M., Steultjens M., Rafferty D., and Woodburn J., “Dose-response effects of customised foot orthoses on lower limb muscle activity and plantar pressures in pronated foot type,” Gait Posture, vol. 38, no. 3, 2013. doi: 10.1016/j.gaitpost.2013.01.012 [DOI] [PubMed] [Google Scholar]

[pone.0277971.ref011] 11.McPoil T. G., Cornwall M. W., and Yamada W., “A comparison of two in-shoe plantar pressure measurement systems,” Low. Extrem., vol. 2, no. 2, pp. 95–103, Jan. 1995. [Google Scholar]

[pone.0277971.ref012] 12.Hsiao H., Guan J., and Weatherly M., “Accuracy and precision of two in-shoe pressure measurement systems,” Ergonomics, vol. 45, no. 8, pp. 537–555, Jun. 2002. doi: 10.1080/00140130210136963 [DOI] [PubMed] [Google Scholar]

[pone.0277971.ref013] 13.Putti A. B., Arnold G. P., Cochrane L., and Abboud R. J., “The Pedar^® in-shoe system: Repeatability and normal pressure values,” Gait Posture, vol. 25, no. 3, pp. 401–405, Mar. 2007. doi: 10.1016/j.gaitpost.2006.05.010 [DOI] [PubMed] [Google Scholar]

[pone.0277971.ref014] 14.Ramanathan A. K., Kiran P., Arnold G. P., Wang W., and Abboud R. J., “Repeatability of the Pedar-X^® in-shoe pressure measuring system,” Foot Ankle Surg., vol. 16, no. 2, pp. 70–73, Jun. 2010. doi: 10.1016/j.fas.2009.05.006 [DOI] [PubMed] [Google Scholar]

[pone.0277971.ref015] 15.Murphy D. F., Beynnon B. D., Michelson J. D., and Vacek P. M., “Efficacy of plantar loading parameters during gait in terms of reliability, variability, effect of gender and relationship between contact area and plantar pressure,” Foot Ankle Int., vol. 26, no. 2, pp. 171–179, 2005. doi: 10.1177/107110070502600210 [DOI] [PubMed] [Google Scholar]

[pone.0277971.ref016] 16.Woodburn J. and Helliwell P. S., “Observations on the F-Scan in-shoe pressure measuring system,” Clin. Biomech., vol. 11, no. 5, pp. 301–304, Jul. 1996. doi: 10.1016/0268-0033(95)00071-2 [DOI] [PubMed] [Google Scholar]

[pone.0277971.ref017] 17.K. Patrick and L. Donovan, “Test–retest reliability of the Tekscan^® F-Scan^® 7 in-shoe plantar pressure system during treadmill walking in healthy recreationally active individuals,” https://doi-org.salford.idm.oclc.org/10.1080/14763141.2017.1355010, vol. 17, no. 1, pp. 83–97, Jan. 2017. [DOI] [PubMed]

[pone.0277971.ref018] 18.J. H. Ahroni, E. J. Boyko, and R. Forsberg, “Reliability of F-Scan In-Shoe Measurements of Plantar Pressure:,” http://dx.doi.org.salford.idm.oclc.org/10.1177/107110079801901004, vol. 19, no. 10, pp. 668–673, Jun. 2016. [DOI] [PubMed]

[pone.0277971.ref019] 19.Bunting K. V., Steeds R. P., Slater L. T., Rogers J. K., Gkoutos G. V., and Kotecha D., “A Practical Guide to Assess the Reproducibility of Echocardiographic Measurements,” J. Am. Soc. Echocardiogr., vol. 32, no. 12, pp. 1505–1515, Dec. 2019. doi: 10.1016/j.echo.2019.08.015 [DOI] [PubMed] [Google Scholar]

[pone.0277971.ref020] 20.Atkinson G. and Nevill A. M., “Statistical methods for assessing measurement error (reliability) in variables relevant to sports medicine,” Sports Med., vol. 26, no. 4, pp. 217–238, 1998. [DOI] [PubMed] [Google Scholar]

[pone.0277971.ref021] 21.Price C., Parker D., and Nester C., “Validity and repeatability of three in-shoe pressure measurement systems,” Gait Posture, vol. 46, 2016. doi: 10.1016/j.gaitpost.2016.01.026 [DOI] [PubMed] [Google Scholar]

[pone.0277971.ref022] 22.Nevill A. M., Atkinson G., Nevill A. M., and Atkinson G., “Assessing agreement between measurements recorded on a ratio scale in sports medicine and sports science.,” Br. J. Sports Med., vol. 31, no. 4, pp. 314–318, Dec. 1997. doi: 10.1136/bjsm.31.4.314 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0277971.ref023] 23.Rosenbaum D. and Becker H.-P., “Plantar pressure distribution measurements. Technical background and clinical applications,” Foot Ankle Surg., vol. 3, no. 1, pp. 1–14, 1997. [Google Scholar]

[pone.0277971.ref024] 24.Preece S. J., Chapman J. D., Braunstein B., Brüggemann G.-P., and Nester C. J., “Optimisation of rocker sole footwear for prevention of first plantar ulcer: comparison of group-optimised and individually-selected footwear designs,” J. Foot Ankle Res., vol. 10, no. 1, p. 27, Dec. 2017. doi: 10.1186/s13047-017-0208-3 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0277971.ref025] 25.R. E. Snow, K. R. Williams, and G. B. Holmes, “The Effects of Wearing High Heeled Shoes on Pedal Pressure in Women:,” http://dx.doi.org.salford.idm.oclc.org/10.1177/107110079201300206, vol. 13, no. 2, pp. 85–92, Aug. 2016. [DOI] [PubMed]

[pone.0277971.ref026] 26.Hulshof C. M., Van Netten J. J., Dekker M. G., Pijnappels M., and Bus S. A., “In-shoe plantar pressure depends on walking speed and type of weight-bearing activity in people with diabetes at high risk of ulceration,” bioRxiv, 2022. [DOI] [PubMed] [Google Scholar]

PERMALINK

Validity and reliability of the XSENSOR in-shoe pressure measurement system

Daniel Parker

Jennifer Andrews

Carina Price

Roles

Abstract

Background

Research question

Methods

Results

Significance

Introduction

Method

Fig 1.

Whole insole protocol

Regional protocol

Variables

Results

Regional protocol

Fig 2. Contact area for each insole and cylinder size for day two across the 2s trial.

Whole insole protocol

Fig 3. Peak and mean pressure values recorded over 0, 2, 10 and 30 s for the 7 target pressures for both insoles on day one.

Table 1. Average peak pressure at each target pressure at 2 seconds for Day 1 and Day 2 the differences, and the “ratio limits of agreement” alongside the t-test between the measures of day 1 and 2.

Table 2. Average mean pressure at each target pressure at 2 seconds for Day 1 and Day 2, the differences, and the “ratio limits of agreement” alongside the t-test between the measures of day 1 and 2.

Fig 4. Peak pressure values recorded at 2s for the 7 target pressures in both insoles between days.

Table 3. Percentage of sensels within 5% of mean value for all pressures and times and each insole size and day 1 and 2.

Discussion

Limitations

Supporting information

Acknowledgments

Data Availability

Funding Statement

References

Decision Letter 0

Peter Andreas Federolf

Roles

Author response to Decision Letter 0

Decision Letter 1

Peter Andreas Federolf

Roles

Author response to Decision Letter 1

Decision Letter 2

Peter Andreas Federolf

Roles

Acceptance letter

Peter Andreas Federolf

Roles

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases