Measurements of best, worst, and average socket comfort are more reliable than current socket comfort in established lower limb prosthesis users

Abstract

Objective:

To evaluate test-retest reliability and related measurement properties of items developed to assess best, worst, and average prosthetic socket comfort.

Design:

Methodological research to assess test-retest reliability of four individual socket comfort survey items. Socket comfort items were included in a self-report paper survey, which was administered to participants 2–3 days apart.

Setting:

General community.

Participants:

A minimum convenience sample of 63 participants was targeted for this study; 72 lower limb prosthesis users (>1-year post-amputation) completed the survey and were included in the final dataset.

Interventions:

Not applicable.

Main Outcome Measure:

The Expanded Socket Comfort Score (ESCS) was adapted from the original Socket Comfort Score (SCS). The original SCS is a single-item self-report instrument developed to assess a lower limb prosthesis user’s current socket comfort. Three additional items were designed to assess the user’s best, worst, and average socket comfort over the previous seven days.

Results:

Best, worst, and average socket comfort items demonstrated better reliability, as indicated by higher intra-class correlation coefficients. As such, these items also exhibited lower measurement error and smaller minimal detectable change values than the item that measured current socket comfort. However, test-retest coefficients for all four items were below the level desired for evaluation of within-individual changes of socket comfort.

Conclusions:

Items that assess best, worst, and average comfort provide a more stable measurement of socket fit than the existing SCS instrument. While administration of all four items may provide more comprehensive assessment of a lower limb prosthesis user’s socket fit, administrators should expect variations in scores over time due to the variable nature of the underlying construct over time. Future research should examine whether a multi-item socket comfort scale provides an improved overall assessment of socket fit.

Introduction

Socket fit quality is generally acknowledged as the most important aspect of a lower limb prosthesis.(1) While achieving a well-fitting, comfortable socket is necessary for a successful prosthetic outcome, there are few standardized outcome measures available to assess socket comfort.(2–4) One of these instruments, the Socket Comfort Score (SCS) was developed to assess current socket fit, and has evidence of validity and sensitivity to socket adjustment, repair, or replacement.(2) However, it also exhibits relatively poor test-retest reliability over a 2–3 day recall period, with intraclass correlation coefficients (ICCs) between .63 and .79, depending on administration method.(5) ICCs below .9 typically indicate that an instrument may not be well-suited to assessment of longitudinal changes in individual patients.(6)

One potential reason the SCS exhibited poor reliability in the prior study may be due to typical variation in residual limb volume and socket fit within and between days.(7,8) Daily variations in limb volume and corresponding changes in socket fit may limit clinicians’ ability to accurately assess the range of comfort experienced by prosthesis users with a single measure of current socket comfort. Researchers have previously developed items to assess least, most, and usual pain to supplement single-item measures of current pain intensity for people with chronic pain.(9) Results from this work suggest that compared to current pain, measures of least and usual pain more strongly correlate with true average pain levels.(9) Additional socket comfort items that ask prosthesis users to report best, worst, and average socket comfort over a period of time may also improve assessment of socket comfort and provide meaningful information about within- and between-day variations. The purpose of this study was therefore to evaluate measurement properties of Expanded Socket Comfort Score (ESCS) items that assess best, worst, and average socket comfort relative to the existing SCS item which measures current comfort.

Methods

Data for this methodological test-retest study were collected as part of a larger study on psychometric properties of patient-reported outcome measures administered to lower limb prosthesis users.(5) An institutional review board approved study procedures, and participants were provided an information statement prior to engaging in study procedures.

Participants

Study participants were recruited through a departmental registry of people in the U.S. who consented to be contacted for rehabilitation-related research studies. Individuals with amputation in the registry were invited to participate in the study via mail or email and were screened by phone. Eligibility criteria pertinent to the present study were: 18 years of age or older, lower limb amputation between the hip and ankle, no other amputations (e.g., other leg or arms), use of a lower limb prosthesis to transfer or walk, ability to read, write, and understand English. The minimum sample size of 63 was determined based on methods described by Walter et al.(10) with the following values: α = 0.05, β = 0.20, ρ0 = 0.50, ρ1 = 0.70, and n = 2 assessments. The lower bound was set at ρ0 = 0.50 because reliability lower than 0.50 would indicate that an instrument would not be appropriate for individual- or group-level comparisons. Additional participants were recruited to account for potential attrition between test- and re-test timepoints.

Study Design

Participants were scheduled to take the test and retest surveys 2–3 days apart in order to reduce recall bias (11) while also minimizing the potential for long-term changes in socket comfort due to degrading socket fit.(12) The selected time between administrations is within the 2-day to 2-week test-retest period generally used for psychometric testing of self-reported surveys.(13) Paper surveys were sealed in individual envelopes and mailed to participants in advance of study appointments, which were scheduled as close to the same time of day as possible. Participants were contacted by phone at the time of their appointment and instructed to open each survey, complete all items, and return the survey in a provided envelope. Survey responses were double-entered to minimize data-entry errors, and screened for missing and/or inconsistent data.

Surveys

The SCS is a single-item measure of current prosthetic socket comfort.(4) ESCS items to assess best, worst, and average socket comfort over the past seven days were added by the investigators (Appendix 1). Seven days is commonly used for a recall period to measure constructs that might vary day-to-day.(14) Responses options for all items ranged from 0 (most uncomfortable) to 10 (most comfortable). The surveys also included demographic questions, health questions, and other standardized measures (eg, the Prosthetic Limb Users Survey of Mobility, PLUS-M(15)) to characterize the sample.(5) Prior to administering the retest survey, a researcher asked the participant if they experienced any changes in health or mobility between test and retest appointments, with follow up questions for those who indicated a change. Those who reported changes in health or mobility were not included in the analysis because our aim was to evaluate scores across two timepoints where the construct (eg, socket comfort) was stable.

Analysis

Descriptive statistics were calculated for socket comfort items at each time point and data distributions were assessed for normality. Test-retest reliability was evaluated using the intra-class correlation coefficient (ICC) model 3,1(16) for individual scores using a fixed effect for time (test-retest) and a random effect for individuals (participants). Confidence intervals for ICCs were derived using the F-distribution. Estimates of measurement error and detectable change (standard error of measurement [SEM] and minimum detectable change [MDC]) were derived using established algebraic transformations based on calculated ICCs and z-scores for the 90% confidence interval.

Results

Eighty-four participants enrolled in the study; seventy-eight participants completed both test and retest surveys. Six participants were excluded from the analysis due to reported changes with their socket/prosthesis (n=5) or a painful pinched nerve that may have affected assessment of socket comfort (n=1). Both test and retest surveys were therefore completed by 72 participants (Table 1). Mean scores were similar between test and retest time points for all four socket comfort items. Measurement properties improved (ie, ICC values increased, SEM and MDC values decreased) for best, worst, and average socket comfort items compared to the current socket comfort item (Table 2).

Table 1.

Participant demographic and clinical characteristics for n=72 participants.

Characteristic	Mean (SD)	Range
Age (years)	61.0 (11.9)	27.4–85.3
Time since amputation (years)	20.6 (17.8)	1.3–67.9
Daily prosthesis use (hours)	13.2 (3.8)	1–18
PLUS-M T-Score	52.7 (9.3)	25.2–71.4
Days between test and retest	2.0 (0.1)	1.8–2.5
	n	%
Gender
Female	17	23.6%
Male	55	76.4%
Amputation level
Ankle disarticulation	1	1.4%
Transtibial	47	65.3%
Knee disarticulation	2	2.8%
Transfemoral	22	30.6%
Amputation etiology
Trauma	49	68.1%
Dysvascular	13	18.1%
Infection	8	11.1%
Tumor	1	1.4%
Congenital	1	1.4%

Abbreviations: PLUS-M, Prosthetic Limb Users Survey of Mobility

Table 2.

Baseline and follow-up data, test-retest ICC, 95% CI, SEM, and MDC(90) for the four variants of the SCS (n=72).

	Test		Retest		Psychometric Properties
	Mean (SD)	Range	Mean (SD)	Range	ICC	95% CI	SEM	MDC(90)
SCS Best	7.6 (2.4)	0–10	7.7 (2.3)	0–10	.84	.75, .89	0.97	2.26
SCS Worst	5.8 (3.0)	0–10	5.9 (2.8)	0–10	.88	.82, .92	1.01	2.34
SCS Average	6.9 (2.6)	0–10	6.9 (2.5)	0–10	.85	.77, .90	0.99	2.31
SCS Current	7.1 (2.6)	0–10	7.1 (2.5)	0–10	.77	.66, .85	1.21	2.82

Discussion

The goal of this study was to assess test-retest reliability for ESCS items that measure different aspects of socket comfort. Best, worst, and average socket comfort items demonstrated better measurement stability than the existing SCS (ie, current comfort), but ICCs for all four items (.77-.88) were below the level typically desired for measuring within-individual changes (.9). While relatively low ICCs are typically problematic for clinical applications, this finding may reflect true change in socket comfort over short periods of time just as, limb circumference measurements that differ day-to-day or morning-to-evening can indicate true change in volume rather than an unreliable tape measure. Given the relationship between limb shape, limb volume changes, and corresponding socket fit,(17) perceived socket comfort is likely a variable attribute for many lower limb prosthesis users. Thus, current, best, worst, and average socket comfort items provide unique and complementary information about the variation in socket comfort that may be experienced by patients in their daily life.

While the ESCS provide insight into different levels of socket comfort in the prior week, these items do not provide information about the situations that may be associated with better or worse socket comfort. The Comprehensive Lower Limb Amputee Socket Survey (CLASS) includes items that assess socket comfort during four activities (ie, sitting, standing, walking, and ascending/descending stairs).(3) However, these items do not capture other clinically-meaningful situations that often affect socket comfort, such as time of day or proximity to dialysis. A larger bank of items that could be individualized based on patient characteristics (eg, time since amputation, activity level, etiology) would be useful to guide clinical care. In situations when the socket has not been worn for a period of seven days, such as during a patient’s first socket fitting, the current socket comfort item (ie, the original SCS) will be the only appropriate assessment of socket comfort.

Limitations

The majority (82%) of the study sample had amputations due to non-dysvascular causes, which differs from the prevalence of major lower limb non-dysvascular amputation (19%) in the general population.(18) Additional research is needed to better understand socket comfort variations in people with dysvascular amputation. Participants on average had their amputations 20.6 years prior, thus results may not generalize to people with recent amputation. ESCS items asked respondents to recall extreme and average socket comfort from the prior 7 days, but participants’ scores were not validated against a socket comfort diary to assess their accuracy. Future research to assess the accuracy of ESCS items is needed to understand how well they reflect variations in socket comfort.

Conclusion

Asking prosthesis users about best, worst, and average comfort provides a more holistic assessment of socket comfort than current socket comfort alone. Study results suggest that scores from the three new ESCS items are more stable than the original SCS item, but they are still highly variable. Future research is needed to characterize changes over time and determine whether an item bank further improves overall assessment of socket comfort.

Abbreviations

ESCS

Expanded Socket Comfort Score

SCS

Socket Comfort Score

ICC

intraclass correlation coefficient

SEM

standard error of measurement

MDC

minimum detectable change