The Short Musculoskeletal Function Assessment: a study of the reliability, construct validity and responsiveness in patients sustaining trauma

Abstract

Objective:

To assess test–retest reliability, construct validity and responsiveness of the Dutch Short Musculoskeletal Function Assessment (SMFA-NL) in patients who sustained acute physical trauma.

Design:

A longitudinal cohort study.

Setting:

A level 1 trauma center in The Netherlands.

Subjects:

Patients who required hospital admission after sustaining an acute physical trauma.

Intervention:

Patients completed the SMFA-NL at six weeks, eight weeks and six months post-injury.

Main measure:

The measures used were The Dutch Short Musculoskeletal Function Assessment. Test–retest reliability (between six and eight weeks post-injury) using intraclass correlation coefficients, the smallest detectable change and Bland and Altman plots. Construct validity (six weeks post-injury) and responsiveness (between six weeks and six months post-injury) were evaluated using the hypothesis testing method.

Results:

A total of 248 patients (mean age: 46.5, SD: 13.4) participated, 145 patients completed the retest questionnaires (eight weeks) and 160 patients completed the responsiveness questionnaires (six months). The intraclass correlation coefficients indicated good to excellent reliability on all subscales (0.80 to 0.98). The smallest detectable change was 17.4 for the Upper Extremity Dysfunction subscale, 11.0 for the Lower Extremity Dysfunction subscales, 13.9 for the Problems with Daily Activities subscale and 16.5 for the Mental and Emotional Problems subscale. At group level, the smallest detectable change ranged from 1.48 to 1.96. A total of 86% of the construct validity hypotheses and 79% of the responsiveness hypotheses were confirmed.

Conclusion:

This study showed that the SMFA-NL has good to excellent reliability, sufficient construct validity and is able to detect change in physical function over time.

Introduction

Patient-reported outcome measures have become increasingly important to evaluate functional outcome after trauma. Numerous patient-reported outcome measures have been developed for this purpose, yet most are disease- or body-region-specific, hence not suitable to assess heterogeneous samples such as patients who sustained various kinds of injuries. In 1999, Swiontkowski et al.¹ introduced the Short Musculoskeletal Function Assessment (SMFA), a patient-reported outcome measure that can be used to gauge physical functioning in patients with a broad range of musculoskeletal conditions. The SMFA was designed as an instrument that is not too specific, nor overly general and is considered suitable for heterogeneous samples such as patients with a broad range of traumatic injuries.^2,3

The SMFA originally consisted of two indices: the Function Index (34 items) and the Bother Index (12 items).¹ The Function Index was considered to be a relatively strict measure of functional limitations, while the Bother Index indicated the “amount” of bother due to the functional limitations. The SMFA has been cross-culturally adapted in various languages, including Dutch (SMFA-NL).^4–9

The two indices were originally reported to be valid, reliable and responsive,^10,11 although recently the Function and Bother indices were shown to have insufficient structural validity in Dutch trauma patients using the SMFA-NL.¹² The findings indicated that the two indices are not a valid representation of the latent construct physical functioning. Only a four-subscale configuration consisting of the subscales Upper Extremity Dysfunction, Lower Extremity Dysfunction, Problems with Daily Activities and Mental and Emotional Problems demonstrated sufficient structural validity.¹²

Although the four-subscale structure showed sufficient structural validity in a broad range of trauma patients, additional clinimetric properties (reliability, construct validity and responsiveness) have not yet been evaluated. Evaluation of these properties is required to justify usage in clinical and research settings. The aim of this study was therefore to evaluate test–retest reliability, construct validity and responsiveness of the four subscales of the SMFA-NL in patients with a broad range of traumatic injuries.

Methods

Study design and recruitment of patients

A longitudinal cohort study design was used. Patients were recruited between October 2012 and March 2016 at University Medical Center Groningen, a level-1 trauma center in the Netherlands. The methods employed in this study have been reviewed by the local Institutional Review Board, which waived further need for approval (METc2012.104). Patients consented to participate in this study. The study was conducted in compliance with the principles outlined in the Declaration of Helsinki on ethical principles for medical research involving human subjects.

The inclusion criterion was patients admitted to the hospital due to acute traumatic injuries. Exclusion criteria were age between 18 and 65, patients who could not read or write Dutch, injuries that resulted in severe neurological deficits, pathological fractures and patients with severe psychiatric conditions (such as active psychosis, bipolar disorders, major depressive episodes).

Patients were asked to complete six questionnaires (described below) at six weeks post-injury and to complete the SMFA-NL for a second time after a two-week interval. Patients received the six questionnaires again at six months post-injury. Standard questionnaires were used. Patients received the questionnaires on paper or electronically and non-responders were reminded once.

Outcome measures

Short Musculoskeletal Function Assessment

The SMFA-NL contains 46 items that can be divided into four subscales: Upper Extremity Dysfunction, Lower Extremity Dysfunction, Problems with Daily Activities and Mental and Emotional Problems.^9,12 All items are scored on a 5-point Likert-type scale. The items of each of the SMFA-NL subscales can be summed up and divided by the maximum score to create subscales, each ranging from 0 to 100, where 0 represents best possible function.

Health Utilities Index 3

The Health Utilities Index 3 is a validated 15-item generic health questionnaire that can be used to assess specific Health-Related Quality of Life and specific health domains including Ambulation, Dexterity, Emotion and Pain.¹³ The Health-Related Quality of Life score (Multi Attribute Score) ranges from 0 to 1, with a score of 1 as best. The standard “past one-week” version was used.¹³ The Health Utilities Index 3 has been recommended and shown valid in a wide range of conditions, including patients with acute traumatic injuries.^13–19 The Health Utilities Index 3 is available in several languages, including Dutch.¹³

EuroQoL-5 Dimensions

The EuroQoL-5 Dimensions (EQ-5D) questionnaire is a generic instrument that can be used to assess health status and Health-Related Quality of Life.²⁰ The EQ-5D consists of five items scored on a 3-point scale and from which a single index score can be calculated. The score ranges from 0 (representing death) to 100 (representing optimal health).^21,22 The EQ-5D has been recommended and shown valid for assessing health status and Health-Related Quality of Life in trauma patients.^{14,15,17,23–25} It is available in over 180 languages, including Dutch.²⁶

Disabilities of Arm, Shoulder and Hand

The Disabilities of Arm, Shoulder and Hand is a body region-specific questionnaire that can be used to assess upper extremity dysfunction.²⁷ It consists of 30 items that are scored on a 5-point Likert-type scale, from which a total score can be calculated. The score ranges from 0 to 100, where 0 represents best possible function. The Disabilities of Arm, Shoulder and Hand has been cross-culturally adapted in various languages, including Dutch, and has been validated in patients with upper extremity injuries.^17,28–32

Lower Extremity Functional Scale

The Lower Extremity Functional Scale is a body region-specific questionnaire that can be used to assess lower extremity function.³³ It consists of 20 items scored on a 5-point Likert-type scale. Items are summed to a total score ranging from 0 to 80. A score of 80 represents the best possible function. The Lower Extremity Functional Scale has been cross-culturally adapted in Dutch and shown to be valid for assessing lower extremity function in patients with traumatic injuries of the lower extremity.^34–38

Numeric Pain Rating Scale

The 11-point Numeric Pain Rating Scale is a valid and frequently used unidimensional measurement instrument to assess pain in adults.^39,40 Scores ranged from 0 to 10 in discrete numbers, where 0 indicated no pain at all and 10 represented the worst imaginable pain.

Global Rating of Effect

Global Rating of Effect questions were used to verify whether no clinical change had occurred in the test–retest interval. The Global Rating of Effect questions were specified for all four subscales of the SMFA-NL, with five answer options ranging from “much improved” to “much deteriorated.”

Procedures

Clinimetric properties were assessed in accordance to the COSMIN guidelines.⁴¹ Test–retest reliability⁴² of the SMFA-NL was evaluated using the six and eight weeks post-injury measurements.

Construct validity⁴² was assessed with the six weeks post-injury data. A total of 50 hypotheses were predefined in terms of expected direction and expected magnitude of correlations of the SMFA-NL with the following patient-reported outcome measures and clinical parameters (Table 3 and Supplemental Appendix 1). Outcome measures used were Health Utilities Index 3; EQ-5D; Disabilities of Arm, Shoulder and Hand; Lower Extremity Functional Scale; and Numeric Pain Rating Scale. Clinical parameters were Injury Severity Score, anatomical injury region, surgical complications reported within 30 days of the injury and hospital length of stay. Injury severity scores were obtained from the Dutch Trauma Registry and institutional patient registry.⁴³ Surgical complications were obtained from the institutional complication registry.

Table 3.

Construct validity hypotheses for the SMFA-NL with other instruments and parameters.

	Upper Extremity Dysfunction	Lower Extremity Dysfunction	Problems with Daily Activities	Mental and Emotional Problems
EQ-5D Index	E: moderate (−) O: −0.18	E: high (−) O: −0.71	E: high (−) O: −0.76	E: moderate (−) O: −0.49
HUI3 Multi Attribute Score	E: moderate (−) O: −0.32	E: high (−) O: −0.64	E: high (−) O: −0.73	E: moderate (−) O: −0.57
HUI3 Emotion	E: low O: −0.06	E: low O: −0.14	E: low (−) O: −0.20	E: moderate (−) O: −0.36
HUI3 Pain	E: moderate (−) O: −0.12	E: moderate (−) O: −0.36	E: moderate (−) O: −0.44	E: moderate (−) O: −0.34
HUI3 Ambulation	E: low O: 0.07	E: high (−) O: −0.83	E: high (−) O: −0.66	E: low O: −0.29
HUI3 Dexterity	E: high (−) O: −0.79	E: low O: 0.13	E: moderate (−) O: −0.17	E: low O: 0.14
DASH	E: high (+) O: 0.61	E: low O: 0.46	E: high (+) O: 0.69	E: moderate (+) O: 0.46
LEFS	E: low O: −0.02	E: high (−) O: −0.88	E: high(−) O: −0.83	E: moderate (−) O: −0.40
Numeric Pain Rating Scale	E: moderate (+) O: 0.24	E: moderate (+) O: 0.24	E: moderate (+) O: 0.36	E: moderate (+) O: 0.43
ISS	E: low O: −0.07	E: low O: 0.15	E: low O: 0.17	E: low O: 0.20
Hospital length of stay	E: low O: 0.12	E: moderate (+) O: 0.32	E: moderate (+) O: 0.35	E: low O: 0.24

E: expected direction and magnitude of predefined correlations of the SMFA-NL subscales with other instruments and parameters; high: r ⩾ 0.6; moderate: 0.3 ⩽ r < 0.6; low: r < 0.3; (+) or (−): expected direction of correlation; O: observed correlation; HUI3: Health Utilities Index Mark 3; DASH: Disabilities of the Arm Shoulder and Hand; LEFS: Lower Extremity Functional Scale; ISS: Injury Severity Score; SMFA-NL: Short Musculoskeletal Function Assessment.

Expected low correlations not assigned a direction since it was hypothesized that the correlation coefficient would be close to zero. Confirmed hypotheses are shown in bold.

Responsiveness⁴² was assessed using the six weeks and six months post-injury data. Hypotheses were predefined for the expected correlation between changes in scores on the SMFA-NL and changes in scores on the Health Utilities Index 3; EQ-5D; Disabilities of Arm, Shoulder and Hand; Lower Extremity Functional Scale; and Numeric Pain Rating Scale questionnaires (Table 4). Additional hypotheses were predefined for discriminative capacity between groups of patients based on anatomical injury region or whether a surgical complication was reported within six months post-injury (Supplemental Appendix 2).

Table 4.

Responsiveness hypotheses for the SMFA-NL with other instruments.

	Upper Extremity Dysfunction	Lower Extremity Dysfunction	Problems with Daily Activities	Mental and Emotional Problems
EQ-5D Index	E: moderate (−) O: −0.18	E: high (−) O: −0.61	E: high (−) O: −0.62	E: moderate (−) O: −0.34
HUI3 Multi Attribute score	E: moderate (−) O: −0.25	E: moderate (−) O: −0.43	E: high (−) O: −0.47	E: moderate (−) O: −0.27
HUI3 Emotion	E: low O: 0.01	E: low O: −0.03	E: low O: −0.11	E: high (−) O: −0.13
HUI3 Pain	E: low O: −0.11	E: low O: 0.17	E: moderate (−) O: −0.31	E: moderate (−) O: −0.20
HUI3 Ambulation	E: low O: −0.04	E: high (−) O: −0.67	E: high (−) O: −0.44	E: low O: −0.09
HUI3 Dexterity	E: high (−) O: −0.68	E: low O: 0.03	E: moderate (−) O: −0.11	E: low O: 0.05
DASH	E: high (+) O: 0.37	E: low O: 0.57	E: high (+) O: 0.71	E: moderate (+) O: 0.37
LEFS	E: low O: −0.11	E: high (−) O: −0.65	E: high (−) O: −0.64	E: moderate (−) P: −0.26
Numeric Pain Rating Scale	E: low O: 0.12	E: low O: 0.21	E: moderate (+) O: 0.37	E: moderate (+) O: 0.28

E: expected direction and magnitude of predefined correlations of the SMFA-NL subscales with other instruments and parameters; high: r ⩾ 0.5; moderate: 0.25 ⩽ r < 0.5; low: r < 0.25; (+) or (−): expected direction of correlation; HUI3: Health Utilities Index Mark 3; DASH: Disabilities of the Arm Shoulder and Hand; LEFS: Lower Extremity Functional Scale; ISS: Injury Severity Score; SMFA-NL: Short Musculoskeletal Function Assessment; O: observed correlation; EQ-5D: EuroQoL-5 Dimensions.

Expected low correlations were not assigned a direction since it was hypothesized that the correlation coefficient would be close to zero. Confirmed hypotheses are shown in bold.

Hypotheses were formulated as follows: The correlation of change in … score with change in … score is expected to be …. For example, the correlation of change in EQ-5D index score with change in SMFA-NL Problems with Daily Activities score is expected to be high.

Statistical analysis

To assess clinimetric properties, a sample size of at least 50 patients is considered minimal and 100 patients preferable.⁴⁴ Anticipating a 40% loss to follow-up and 10%–15% of all patients missing one or more items in any of the returned questionnaires, we aimed to include at least 200 patients.

Test–retest reliability was evaluated using the intraclass correlation coefficient (ICC 2,1) for absolute agreement and was based on a two-way random effects model.⁴⁵ Only patients who scored the Global Rating of Effect question of the specific subscale as “not changed” were included in the test–retest analysis. Intraclass correlation coefficients ⩾0.70 were considered an indication of good reliability and values ⩾0.90 an indication of excellent reliability.⁴⁴

Measurement error was evaluated with the standard error of measurement for absolute agreement, smallest detectable change and limits of agreement in Bland and Altman plots.⁴⁵ The smallest detectable change was calculated at the individual and group level.⁴⁶ In the Bland and Altman plots, the difference in scores between the test–retest measurements were plotted against the mean score of the test–retest measurements.⁴⁷ Limits of agreement were calculated as mean test–retest difference ± 1.96 SD_difference. One-sample t-tests were used for all subscales to determine whether the difference between the test and retest measurement was different from zero. A significant difference was considered evidence of systematic bias.⁴⁷ Univariable linear regression analyses were used to investigate proportional bias: the effect of the mean test–retest scores on the test–retest difference scores. Regression coefficients that were statistically different from zero were considered to be an indication of proportional bias.⁴⁷

Construct validity was considered sufficient when at least 75% of the predefined hypotheses were confirmed.⁴⁴ The predefined hypotheses were tested using Pearson correlation coefficients for continuous variables, and mean differences were calculated to assess differences between specific groups of patients. Confirmation or rejection of the hypotheses was based on the magnitude of the correlation coefficient or mean difference, rather than P-values.⁴¹ A correlation coefficient <0.3 was considered low, between 0.3 and 0.59 moderate, and ⩾0.6 high.

The data of the six weeks post-injury measurement was assessed for floor and ceiling effects. Floor or ceiling effects occur when patients score the absolute maximum or minimum score on a measurement instrument. When ⩾15% of the measurements were either the minimum or maximum score, they were regarded as a floor or ceiling effect, respectively.⁴⁸ Patients without upper or lower extremity injuries may be expected to report the best possible score on the Upper or Lower Extremity Dysfunction subscales, respectively. Hence, floor and ceiling effects on the Upper and Lower extremity subscales were analyzed in patients who had an upper or lower extremity injury, respectively. The entire study sample was used to analyze floor and ceiling effects for the Problems with Daily Activities and Mental and Emotional Problems subscales.

A total of 42 predefined hypotheses (Table 4, Supplemental Appendix 2) on responsiveness of the SMFA-NL were tested using Pearson correlation coefficients. For responsiveness, both measurements may carry measurement error; therefore, correlation coefficients <0.25 were considered low, between 0.25 and 0.49 moderate and ⩾0.5 high.⁴⁹ Responsiveness was considered sufficient when at least 75% of the predefined hypotheses were confirmed.⁴⁴

Results

A total of 248 patients completed the questionnaires at six weeks post-injury. The response rate was 64%. The general characteristics of the study sample are shown in Table 1. Most patients were treated surgically and the lower extremity was the most common anatomical region of injury (Table 1). Several patients did not disclose marital status or educational level (Table 1). In total, 145 patients completed both the test and retest questionnaire. A total of 160 patients completed both the six weeks and six months post-injury questionnaires.

Table 1.

General characteristics of the study sample.

General characteristics	N (%)
Gender (n = 248)
Male	148 (60)
Female	100 (40)
Age (n = 248)	46.5 (13.4)a
Marital status (n = 218)
Single	75 (33)
With partner	144 (67)
Educational level (n = 206)
Elementary school	3 (1)
High school	70 (31)
College	70 (31)
Bachelor’s degree or higher	81 (36)
Other	4 (1)
Injuries (n = 678)
Head and neck	40 (6)
Face	30 (4)
Thorax	62 (9)
Abdomen	25 (4)
Spine	98 (14)
Upper extremity	155 (23)
Lower extremity and pelvic bones	214 (32)
Skinb/other	54 (8)
Injury Severity Score (n = 248)
All patients	4 (1–42)c
Major trauma (ISS ⩾ 16)	35 (14)
Treatment (n = 248)
Conservative treatment	43 (17)
Surgeryd	205 (83)
Surgical complication within 30 days (n = 248)	36 (15)

ISS: Injury Severity Score.

^aPresented as mean (SD).

^bSuperficial injuries (abrasion, contusion, lacerations, regardless of anatomical region).

^cPresented as median (range).

^dRequiring surgery for at least one of the injuries.

Clinimetric properties

The intraclass correlation coefficients of the Upper Extremity Dysfunction and Mental and Emotional Problems subscales indicated good reliability (Table 2). The Lower Extremity Dysfunction and Problems with Daily Activities subscales demonstrated excellent reliability (Table 2). The standard error of measurement and smallest detectable change are shown in Table 2. Least measurement error was demonstrated for the Lower Extremity Dysfunction subscale, indicating best precision among the four subscales. The Upper Extremity Dysfunction Subscale demonstrated most measurement error. Bland and Altman plots do not show an upward or downward trend for any of the subscales (Figure 1). The measurements were equally spread above and below the 0 line for all subscales. The limits of agreement were smallest for the Lower Extremity Dysfunction and Problems with Daily Activities subscales and widest for the Upper Extremity Dysfunction subscale. The mean test–retest differences of the subscales were not significantly different from zero for the Upper Extremity Dysfunction, Problems with Daily Activities and Mental and Emotional Problems subscales (Table 2), indicating there was no evidence for systematic bias. Systematic bias was observed for the Lower Extremity Dysfunction subscale (Table 2). None of the regression coefficients were significantly different from zero, indicating there was no evidence of proportional bias (Table 2).

Table 2.

Reliability, measurement error, and systematic bias and proportional bias.

	Subscales of the SMFA-NL
	UED	LED	PDA	MEP
Meantest (SD)	8.9 (19.0)	23.9 (26.8)	53.7 (29.2)	19.8 (13.4)
Meanretest (SD)	8.8 (18.9)	22.4 (26.5)	52.0 (28.8)	19.8 (13.1)
Reliability and measurement error
ICC(2,1)agr	0.89	0.98	0.97	0.80
(95% CI)	(0.84–0.93)	(0.96–0.99)	(0.95–0.98)	(0.69–0.87)
SEMagr	6.28	3.97	5.03	5.95
SDCind	17.4	11.0	13.9	16.5
SDCgr	1.93	1.48	1.95	1.96
Systematic bias
Mean difference	−0.10	−1.55	−1.76	0.00
P-value	0.9	0.04	0.07	1.0
Proportional bias
β	−0.01	−0.06	−0.56	−0.03
P-value	0.9	0.7	0.7	0.8

UED: Upper Extremity Dysfunction; LED: Lower Extremity Dysfunction; PDA: Problems with Daily Activities; MEP: Mental and Emotional Problems; Test: six weeks post-injury; retest: eight weeks post-injury; ICC(2,1)_agr: intraclass correlation coefficient for agreement using a two-way random effects model; SEM_agr: standard error of measurement for agreement; SDC_ind: smallest detectable change at the individual level; SDC_gr: smallest detectable change at the group level; β: standardized regression coefficient; SMFA-NL: Short Musculoskeletal Function Assessment.

Figure 1.

Bland and Altman plots of the test–retest analysis. Bland and Altman plots of the test–retest analysis for all SMFA-NL subscales: Upper Extremity Dysfunction (UED), Lower Extremity Dysfunction (LED), Problems with Daily Activities (PDA) and Mental and Emotional Problems (MEP). Blue line: mean test–retest difference; red dashed lines: limits of agreement. LoA: limit of agreement.

The correlation of the SMFA-NL subscales with other patient-reported outcome measures and clinical parameters is shown in Table 3. In total, 43 of the 50 (86%) pre-specified hypotheses were confirmed. All correlation coefficients that were expected to be high were confirmed as such. All but one of the hypotheses expected to have a low correlation were confirmed. Five out of the six hypotheses on discriminative validity were confirmed (Supplemental Appendix 1). Patients who suffered a surgical complication within 30 days scored 14.5 points higher on the Problems with Daily Activities subscale (Supplemental Appendix 1). Patients with an upper extremity injury scored 20.5 points higher on the Upper Extremity Dysfunction subscale than patients without an upper extremity injury. Patients with a lower extremity injury scored 31.4 points higher on the Lower Extremity Dysfunction subscale than those without a lower extremity injury.

A floor effect was observed for the Upper Extremity Dysfunction subscale: among patients with an upper extremity injury, 23 patients (20%) scored the lowest possible score. In this group, 16 (70%) patients had a fractured clavicle or scapula, or had a small injury to the hand. Other subscales did not show floor or ceiling effects.

The expected Pearson correlation coefficients of changes in scores between the six weeks and six months post-injury measurement on the SMFA-NL and changes in score on other patient-reported outcome measures are shown in Table 4 and Supplemental Appendix 2. Of the 43 predefined hypotheses, 34 (79%) were confirmed. The Upper Extremity Dysfunction subscale showed high correlations with the Health Utilities Index 3 Dexterity subscale and a low correlation with Health Utilities Index 3 Ambulation. The Upper Extremity Dysfunction subscale showed a correlation of 0.37 with the Disabilities of Arm, Shoulder and Hand. The Lower Extremity Dysfunction subscale showed high correlations with the EQ-5D; Health Utilities Index 3 Ambulation subscale; Disabilities of Arm, Shoulder and Hand; and Lower Extremity Functional Scale. The Problems with Daily Activities and Upper and Lower Extremity Dysfunction subscales showed a low correlation with Health Utilities Index 3 Emotion. The change in Problems with Daily Activities score showed high correlations with the EQ-5D; Health Utilities Index 3 Multi Attribute Score; Health Utilities Index 3 Ambulation; Disabilities of Arm, Shoulder and Hand; and Lower Extremity Functional Scale. Hypotheses on the Mental and Emotional Problems subscale were confirmed least. Raw scores and change in scores of the six weeks to six months interval are shown in Table 5.

Table 5.

SMFA-NL scores of the six weeks to six months interval.

	Mean6w (SD)	Mean6m (SD)	Mean diff (SD)
Upper Extremity Dysfunction (n = 159)	13.2 (20.4)	6.2 (11.8)	7.0 (13.8)
Lower Extremity Dysfunction (n = 151)	32.3 (24.7)	17.0 (17.7)	15.3 (19.2)
Problems with Daily Activities (n = 152)	53.1 (24.3)	27.9 (21.1)	25.1 (20.9)
Mental and Emotional Problems (n = 160)	24.2 (14.4)	20.1 (14.3)	4.1 (12.5)

Mean_6w: mean of the six weeks post-injury measurement; Mean_6m: mean of the six months post-injury measurement; mean diff: mean difference of the six weeks and six months post-injury measurements; SMFA-NL: Short Musculoskeletal Function Assessment.

Discussion

This study demonstrated that the SMFA-NL has good to excellent test–retest reliability, sufficient construct validity and responsiveness to assess physical function in patients who sustained trauma. Due to floor effects, the clinical usability of the Upper Extremity Dysfunction subscale may be limited.

To justify the use of the SMFA-NL in clinical practice or in applied research, it is important to establish its clinimetric measurement properties in concordance with the COSMIN criteria.⁴¹ The reliability and validity of the SMFA-NL enable assessment of physical function at a single point in time. The responsiveness of the SMFA-NL allows evaluation of (recovery of) physical function over time. The SMFA-NL can be applied in patients with a broad range of injuries, ranging from wounds to major trauma.

Previous studies have assessed the clinimetric properties of the SMFA-NL in trauma patients, using a slightly modified version where double-barreled items were split.^50,51 Van Son et al.⁵¹ assessed the clinimetric properties in patients with isolated unilateral lower extremity fractures and upper extremity fractures. They reported sufficient reliability, construct validity and responsiveness, yet in that study two different three-subscale structures were used to calculate scores: one set of subscales for patients with upper extremity fractures and another set of subscales for patients with lower extremity fractures. This complicates the scoring, especially in patients who suffered fractures of both the upper and lower extremities. Van Delft-Schreurs et al.⁵⁰ assessed the clinimetric properties in a sample that only contained major trauma patients, one to four years post-injury. In that study, another unique set of three subscales were used, which were concluded to be valid. However, in that study, test–retest reliability was not evaluated and responsiveness was not evaluated as a longitudinal measurement, but was calculated as the difference with a pre-injury baseline-group. In this study, the clinimetric measurement properties of the SMFA-NL have been investigated more extensively and in a broader range of trauma patients. Furthermore, the four-subscale configuration of the SMFA-NL may be easily applied in day-to-day clinical practice, as there is just one set of subscales for all trauma patients.

The choice of the instruments used for the assessment of construct validity responsiveness was based on the constructs that are evaluated with the subscales of the SMFA-NL. The Disabilities of the Arm Shoulder and Hand (DASH) and Lower Extremity Functional Scale (LEFS) are extremity-specific questionnaires that match the extremity-specific subscales of the SMFA-NL. The Health Utilities Index 3 and the EQ-5D are complementary generic instruments that aim to cover the entire spectrum of disease and functional limitations, including constructs such as daily activities and mental and emotional problems. In a guideline aimed at assessing health status after trauma, it has been advised to use both the Health Utilities Index 3 and EQ-5D.¹⁴

The floor effect observed for the Upper Extremity Dysfunction subscale was mainly caused by patients with a relatively mild injury of the upper extremity. This may indicate that the subscale lacks sensitivity to detect some upper extremity functional problems. Alternatively, these patients may have already recovered after six weeks. Floor effects pose largest problems in longitudinal analyses, as patients cannot show any further improvement in score even if they do experience clinical improvement.⁴⁹ Therefore, the use of the Upper Extremity Dysfunction subscale may be limited, especially in patients with a relatively mild injury of the upper extremity. Similar to the findings in this study, floor effects have been reported in the development study of the SMFA and in studies of the SMFA that evaluated clinimetric properties of an upper extremity subscale.^1,9,50,51 The addition of items with a higher difficulty may resolve significant floor effects, yet modification of the SMFA-NL was beyond the scope of this study.

The systematic bias of the Lower Extremity Dysfunction subscale was considered small and may have been caused by subclinical recovery of the lower extremity. Systematic bias of the SMFA had only been investigated in one study. Reininga et al.⁹ reported a small but irrelevant systematic bias for the Bother Index of 2 points. We considered the systematic bias to have had a small influence on the reliability of the Lower Extremity Dysfunction Subscale, as the bias was smaller than the measurement error and may be easily controlled for when needed.⁵²

The smallest detectable change is an important benchmark to interpret changes in scores. It indicates the point from which a change can be considered a true change and not due to measurement error.^44,53 The smallest detectable change values of the SMFA-NL at group level were considered small. However, at the individual level, the smallest detectable change values of the Upper Extremity Dysfunction and Mental and Emotional Problems subscales may limit the ability to assess early clinical changes.

Two studies have reported smallest detectable change values of the SMFA.^9,54 Pinsker et al. reported a smallest detectable change of 9.6 points on the function index. This is a much lower smallest detectable change value compared to our findings; however, that was in patients with clinically stable end-stage ankle arthritis, which is not representative of the sample of this study.⁵⁴ In a sample of Dutch patients with various musculoskeletal disorders, Reininga et al.⁹ reported standard error of measurement values from which smallest detectable change values could be calculated. Despite a slightly higher reliability (intraclass correlation coefficients range: 0.91–0.96) than in this study, the smallest detectable changes of Reininga et al.⁹ were larger than in this study (range: 23.3–31.3 points). Reininga et al.⁹ studied a more heterogeneous sample, which may have led to higher reliability statistics, while not affecting absolute measurement error. In addition, no external criterion was applied to identify patients who had not changed, which may have increased measurement error. Although the measurement error in this study (expressed as smallest detectable change) was smaller than in the study by Reininga et al., the interpretation of a change in score requires an additional benchmark: the minimal important change.⁵³ The minimal important change reflects which change in score is a meaningful change to patients. However, a minimal important change is currently not known for the SMFA.³

The two-week test–retest interval may be considered a limitation of this study, as it carries the risk of recall bias. However, a two-week test–retest interval is generally considered a safe margin to avoid significant recall bias, but short enough to avoid clinical improvement.⁴⁴ Additional Global Rating of Effect questions were used to exclude that patients experienced clinical change, although these questions may not capture subclinical change. Second, although the sample size was considered adequate, due to the longitudinal study design some patients were lost to follow-up for the eight weeks and six months measurements. This may have induced selection bias. Third, the clinical usability of the Upper Extremity Dysfunction subscale may be limited, especially in patients with a relatively mild injury of the upper extremity.

One of the strengths of this study was that this was the first study in which the responsiveness of the SMFA has been evaluated using the COSMIN guidelines, in which hypotheses-testing is recommended.⁴¹ Other studies have reported standardized response means, which is an effect size-based measure that does not relate to the validity of the measured change.^1,4,5,50,51

The SMFA-NL may be used in clinical practice as an overall evaluation of physical function at one moment, or as an instrument to assess change in physical function over time. In research the SMFA-NL may be used whenever the researcher is interested in the functional status or functional recovery of an injured patient, for example t in clinical trials in which conservative and surgical treatment of fractures are compared. To improve interpretability of the SMFA-NL, future research may be dedicated to assess which change in score is important to a patient and which difference in score between groups of patients may be considered relevant.

Clinical messages.

• The Short Musculoskeletal Function Assessment may be used to assess physical functioning at a single moment in patients who sustained trauma.

• The Short Musculoskeletal Function Assessment may be used to measure change in physical function over time in patients who sustained trauma.

• Floor and ceiling effects may limit the usefulness of the Upper Extremity Dysfunction subscale in longitudinal analyses.