The Critical Difference in the DASH Outcome Measure After Essential Upper Extremity Tumor Surgery

Abstract

Background:

The DASH (Disabilities of the Arm, Shoulder and Hand) is a scored questionnaire that is widely used to evaluate the health-related quality-of-life of patients with upper-limb musculoskeletal disorders. However, numerical changes in the measure scores lack clinical significance without meaningful threshold change values of outcome measures that are diagnostically specific. The minimal clinically important difference is useful for the interpretation of scores by defining the smallest change that a patient would perceive. However, the minimal clinically important differences of the scores in orthopaedic oncology patients has not been reported. We aimed to determine the minimal clinically important differences of the measure in orthopaedic oncology patients.

Methods:

Data from our health-related quality-of-life Database from 1999 to 2005 were retrospectively reviewed after institutional review board approval. Seventy-eight patients who underwent surgery and completed two surveys during postoperative follow-up were evaluated. Two different methods were used to estimate the minimal clinically important differences: distribution-based and anchor-based approaches (the latter utilized receiver operating characteristic analysis).

Results:

Using distribution-based methods, the minimal clinically important differences of the DASH questionnaire were 7.4 and 8.3 by half standard deviation and the 90% interval of minimal detectable change, respectively. By anchor-based method (receiver operating characteristic analysis), the minimal clinically important difference was 8.3.

Conclusion:

The minimal clinically important difference values calculated by each method validates that the results for upper extremity oncology patients were similar to those reported in other orthopaedic conditions. These results identify the threshold for meaningful improvements in DASH scores in orthopaedic oncology patients and establish the reference to evaluate health-related quality of life and the outcomes of upper extremity oncology surgery. These data should be further refined for disease- and reconstruction-specific analyses.

Level of Evidence:

Level III, diagnostic study

INTRODUCTION

Due to improved survival of sarcoma patients, there has been increased attention on functional outcomes and health-related quality-of-life (HRQoL) after the treatment of disease. Accordingly, the number of publications on HRQoL after surgery for musculoskeletal tumors has increased.^7,16,17,26 There have been several measures used for patients with sarcoma in the upper extremity, including the Musculoskeletal Tumor Society (MSTS) rating scale^11,18,32 and Toronto Extremity Salvage Score.^1,9,25 Patients have also been evaluated using more generic scales, such as SF-36⁴, or focused scales such as the Patient-Reported Outcomes Measurement Information System (PROMIS) scale^6,13,27, and the Disabilities of the Arm, Shoulder and Hand (DASH) questionnaire.³

The DASH questionnaire is the most widely used region-specific measure of disability and symptoms in people with musculoskeletal disorders of the upper limb.³ Definition of the minimal clinically meaningful change varies with the population studied and is disease- or surgical procedure- specific to form homogeneous population.¹⁹ The longitudinal construct validity of the questionnaore scores has been reported to vary considerably from 15–23 (mean of 19) for “much better” or “much worse” after surgery, and 7–14 (mean of 10) for those reporting the arm to be “somewhat better” or “somewhat worse” in selected surgeries ranging from carpal tunnel release to shoulder arthroscopy.¹⁴

This understanding is imperative to interpret DASH scores to assess HRQoL in patients with upper-limb tumors. However, the minimal clinically important difference (MCID), defined as the smallest change a patient can perceive as being meaningful, of DASH in orthopaedic oncology patients has not to our knowledge been reported to date. This study aimed to determine the MCID of DASH using HRQoL data in patients undergoing surgical treatment for musculoskeletal tumors of the upper extremity.

MATERIALS AND METHODS

This is a retrospective observational study to investigate MCID of the DASH in orthopaedic oncology patients.

Patients and Data Collection

The Musculoskeletal Outcomes Data Evaluation and Management System (MODEMS) instrument, which combines multiple outcomes instruments, including the SF-36 and DASH, was implemented as part of our institutional clinical assessment of all patients. This follows the developed and validated instrument from the American Academy of Orthopaedic Surgeons that was endorsed by the Musculoskeletal Tumor Society after its multispecialty consensus meeting in Tarpon Springs, FL, USA.²⁰

The study was performed at a single institution, which approved the use of human subjects for this investigation (IRB #16–913). All investigations were conducted in conformity with the ethical principles of research.

All patients who underwent surgery and were evaluated for HRQoL using the MODEMS instrument at three months and then every six months after surgery from 1999 to 2005 (n = 960) (Figure 1). At each follow-up survey, information was provided regarding the purposes and methods of the MODEMS instrument.

Figure 1.

Flow diagram of patient inclusion and exclusion criteria. POSNA = Pediatric Orthopaedic Society of North America; HRQoL = health-related quality-of-life; DASH = Disabilities of the Arm, Shoulder and Hand.

All patients underwent surgery, and HRQoL outcomes were evaluated using the DASH questionnaire six months following each surgery. When patients received each follow-up questionnaire, an anchor question with a 5-point Likert scale was administered, which inquired about changes in health condition compared with each patients’ previous visit. The anchor question included the possible answers: “Much better”, “Somewhat better”, “About the same”, “Somewhat worse”, or “Much worse”.

Instrument

The DASH questionnaire³ is a region-specific measure of disability and symptoms in people with any or multiple musculoskeletal disorders of the upper limb and allows comparisons across diagnoses of the upper limb. The DASH questionnaire has 30 items rated on a 5-point Likert scale. Scores range from 0 (reflecting no disability) to 100. The items are composed of 1) the degree of difficulty during the preceding week in performing various physical activities because of problems in a shoulder, arm, or hand (21 items); 2) the severity of each of the symptoms of pain, activity-related pain, tingling, weakness, and stiffness (five items); and 3) the problem’s effect on social activities, work, and sleep, and its psychological impact (four items). In this study, we did not analyze the modules for work and sports/performing arts, as these models are optional for DASH and not included in MODEMS.

Calculation of MCID

The MCID is used in clinical trials to help readers and patients interpret whether the effect size associated with treatment is a sample large enough to justify the risk, pain, cost, or inconvenience of the treatment^22,23. In calculating MCID in this study, two different methodologies were employed: distribution-based and anchor-based approaches.²⁸

The distribution-based approach utilizes the distributional characteristics of the sample. It also determines the observed variation using a standardized metric, such as the standard deviation (SD), effect size, standard error of measurement (SEM), or minimal detectable change (MDC). Calculating MCID using half of SD, SEM, and MDC is widely used because they have been shown to correspond to the MCID across a variety of studies.^5,24 The rationale for using this approach is that it detects the degree to which change between the baseline and different time points exceeds what would be expected from chance alone. The major disadvantage of all methods that employ the distribution-based approach is that they do not, in themselves, provide a good indication of the importance of the observed change.

The anchor-based approach uses an external criterion, or anchor, to determine what patients or their clinicians consider to be an important improvement/deterioration. Anchor-based methods assess which changes in the measurement instrument correspond with a minimal important change defined by the anchor. The advantage of this method is that the concept of “minimal importance” is explicitly defined and incorporated. The major disadvantage of this method is that global assessment scale may not always be valid. For example, 1) it can be susceptible to recall bias, and 2) it may not be appropriate for all dimensions of multidimensional HRQoL scores.

With these issues in mind, Crosby et al. recommended the combined use of anchor-based and distribution-based methods to take advantage of both an external criterion and a measure of variability, and compensate for disadvantages of each method.⁸ Therefore, we combined both distribution-based and anchor-based approaches to determine the MCIDs of the DASH in the postoperative setting following surgical treatment of upper extremity sarcomas.

With the heterogeneity of orthopaedic oncology conditions in mind, we reported MCIDs of DASH according to the severity of the HRQoL to form more homogeneous groups and to determine the MCID for each group separately, as well as those for our overall patient population. The cutoff value of severity of DASH score was based on the median score and defined as 23.

Distribution-Based Methods

MCIDs were calculated using two different statistical characteristics of the distribution of the scores. First, we determined MCIDs by the half SD method²⁴, where minimal change is considered as the half of SD of change scores. Second, the MDC₉₀ method was used, which sets the MDC at a 90% confidence interval, rather than using SEM. MDC₉₀ was chosen because of its use in previous studies regarding DASH, including a report from the developers of the DASH instrument.^3,10,12,29 MDC₉₀ is interpreted as 90% of truly stable patients will demonstrate random variation of less than this magnitude when assessed on multiple occasions.²¹

The SEM and MDC₉₀ were calculated using the following formula, where SD stands for the SD of baseline scores and R stands for the reliability coefficient:

$$\text{SEM}=\text{SD x SQRT }(\mathit{1}-\text{R})$$

$$\text{MD}{\text{C}}_{90}=1.65\text{ x SQRT }\left(2\right)\text{ x SEM}$$

For the reliability coefficient, 0.96 was used for the DASH score in accordance with previous reports.^2,3

Anchor-Based Methods

The anchor-based approach was used to determine cut-off values for MCID based on the answer to the anchor question, which is the standard for assessing the change of patient’s condition. Receiver operating characteristic (ROC) curve analysis to examine changes in DASH score between two surveys was performed to compute a discrete value for the MCID by evaluating a threshold difference in the DASH score that yielded the smallest difference between sensitivity and specificity. The ROC-derived MCID was taken to be the change in scores from the baseline with sensitivity and specificity to distinguish between patients who reported their outcome as “About the same” and those who reported their status as “Somewhat better”. The discriminative ability of the model was assessed using the area under the ROC curve (AUC).

Statistical Analysis

All statistical analyses were conducted using IBM SPSS version 18.0 (IBM SPSS, Armonk, NY, USA). The scores were reported as the mean values ± SD. The AUC provides a measure of accuracy, and AUC was interpreted as defined previously¹⁵: 0.90–1.00, excellent; 0.80–0.90, good; 0.60–0.80, fair; and 0.50–0.60, failed. For the required sample size to calculate MCIDs, the validity of the number of cases included in our retrospective database was confirmed. The COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) checklist was followed, which is the standard to study the psychometric property of HRQoL measures. COSMIN considers a sample size of 50 cases as good³¹. Although our cohort meets the COSMIN requirement, in order to reinforce our statistical strategy, we performed power analysis to determine the required sample size in the following conditions using R version 3.6.2 (R Foundation for Statistical Computing, Vienna, Austria): Type I error (Alpha, significance) = 0.05, Type II error (Beta, 1-power) = 0.10, Area under ROC curve = 0.80, Ratio of sample sized in negative/positive groups = 2, one-sided test. The required number of cases was computed as 31 cases (number of positive and negative cases, 10 and 21, respectively), suggesting our sample size was sufficient for calculating MCID.

RESULTS

Seventy-eight patients who completed at least two surveys after surgery and had no missing data were included in the final analysis. The mean duration between surveys was 8.9 months. The clinical and demographic characteristics of the participants are detailed in Table I.

TABLE I.

Patient demographics

Characteristic	Number of patients (%)
Overall	78 (100)
Age, years; mean [SD a ]	48.4 (range, 18–87) [17.0]
Sex
Male	37 (47.4)
Female	41 (52.6)
Histologic diagnosis
Chondrosarcoma	18 (23.1)
Synovial sarcoma	7 (9.0)
Osteosarcoma	6 (7.7)
Undifferentiated pleomorphic sarcoma	6 (7.7)
Myxofibrosarcoma	5 (6.4)
Liposarcoma	4 (5.1)
Fibrosarcoma	4 (5.1)
Desmoid tumor	4 (5.1)
Ewing’s sarcoma	2 (2.6)
Leiomyosarcoma	2 (2.6)
Skin cancer	2 (2.6)
Benign lesion	1 (1.3)
Other soft tissue sarcoma	5 (6.4)
Cancer metastasis	12 (15.4)
Type of surgery
Wide resection	42 (53.8)
Wide resection + endoprosthetic reconstruction	9 (11.5)
Wide resection + allograft reconstruction	6 (7.7)
Curettage +/− arthroplasty/internal fixation	18 (23.1)
ORIFb	1 (1.3)
Amputation	2 (2.6)

^a-SD = standard deviation;

^b-ORIF = open reduction and internal fixation.

The baseline DASH score, distribution of answers to each anchor question, and change of DASH scores (follow-up score - baseline score) are summarized in Table II. The mean change in DASH score was −6.6, with lower scores signifying clinical improvement. Based on the anchor question, most patients answered either, “About the same” (41.9%) or improved, which included “Much better” or “Somewhat better” (46.0%). The relationship between change of DASH scores and answers to anchor questions is presented using a box-and-whiskers plot (Figure 2), which demonstrates a positive correlation between patient-reported change and change in DASH scores.

TABLE II.

Details of DASH^a score and anchor questions

Characteristic	Number of patients (%)
Baseline survey; mean [SD b ]	24.9 [18.1]
Follow-up survey
Anchor question
“Much better”	19 (25.7)
“Somewhat better”	15 (20.3)
“About the same”	31 (41.9)
“Somewhat worse”	8 (10.8)
“Much worse”	1 (1.4)
Change of the DASH score; mean [SD]	−6.6 [14.8]

^a-DASH-Disabilities of Arm, Shoulder and Hand;

^b-SD- standard deviation.

Figure 2.

Box-and-whiskers plots to demonstrate relationship between change of DASH score and answers to anchor questions. DASH = Disabilities of Arm, Shoulder and Hand.

MCID of DASH Score

Using distribution-based methods, the MCID of DASH was calculated as 7.4 (7.1 for low baseline score and 7.9 for high baseline score) based on the half SD, and 8.3 (3.3 for low baseline score and 6.1 for high baseline score) based on the MDC₉₀, respectively.

Using the anchor-based method, the MCID of DASH that yielded the smallest difference between sensitivity and specificity, which were calculated using ROC analysis, was estimated as 8.3. On ROC curve analysis, the AUC was 0.75 (95% CI, 0.64–0.86; P < 0.001) (Figure 3), which suggested good discriminative ability of the model. When comparing low (DASH score >23, N = 39) and high (DASH score ≤23, N = 39) baseline DASH score, ROC-based MCIDs were 3.6 (AUC, 0.69; 95% CI, 0.53–0.85; P = 0.035) and 8.8 (AUC, 0.82; 95% CI, 0.67–0.97; P = 0.001), respectively. As expected, patients with high baseline DASH score required more change of score to perceive improvement of physical function compared to those with low baseline score.

Figure 3.

The ROC curve plots the true positive rate (sensitivity) against the false-positive rate (1 - specificity) for Disabilities of The Arm, Shoulder and Hand questionnaire scores. ROC = receiver operating characteristic.

DISCUSSION

Meaningful threshold change values of outcome measures are essential for understanding changes in each patient’s functional status in a concise and comprehensible manner. To that end, MCID was developed to calculate benchmarks for interpreting change in patient-reported outcome measures (PROMs).²⁸ For the DASH, MCID has been calculated in various disease conditions in orthopaedic surgery (Table III).^{3,10,12,29,30} Psychometric evaluation (reliability, validity, responsiveness, or MCID) of PROMs should be tailored to the specific patient population in question, such as patients with different backgrounds or disease conditions. Although the DASH is the most widely used and validated HRQoL measure for upper extremity disorder, it still needs to have its disease specific MCID established. The HRQoL profile of orthopaedic oncology patients is different from those with benign orthopaedic diseases. It is widely known that HRQoL domains exhibit differences by cancer type and anatomy involved. Yet, musculoskeletal oncology patients in general, and upper extremity patients in particular, have not been analyzed. It is also reasonable that orthopaedic oncology patients have different HRQoL profiles compared to those of general orthopaedic patients with unaltered life expectancy and more consistent anatomic and functional deficits.^22,23 Therefore, we reported MCIDs of DASH specific to upper extremity orthopaedic oncology patients.

TABLE III.

Comparison of MCID^a and DASH^b among orthopaedic conditions

Report	Author	Number of cases	Subjects	MCID Calculation	MCID
J Hand Ther, 2001	Beaton DE et al.3	200	Patients with upper-limb musculoskeletal disorders	MDCc90	10.7
J Clin Epidemiol, 2004	Schmitt JS et al.29	154	Adults with musculoskeletal upper extremity problems	Anchor	12.6
				MDC90	12.6
Qual Life Res, 2008	Dawson J et al.10	104	Patients undergoing elbow surgery	MDC90	9.3
				Anchor	10.3
J Hand Surg Am, 2013	Sorensen A et al.30	102	Patients undergoing non-operative treatment for tendonitis, arthritis, or nerve compression from the forearm to the hand.	Anchor	10.0
J Orthop Sports Phys Ther, 2013	Franchignoni F et al.12	255	Patients with upper limb musculoskeletal disorders	Anchor	10.8
				MDC90	10.8
Current series	Ogura K et al.	78	Patients undergoing surgery for upper-limb tumor	Anchor	8.3
				MDC90	8.3
				Half SDd	7.4

^a-MCID = minimal clinically important difference;

^b-DASH = Disabilities of the Arm, Shoulder and Hand;

^c-MDC = minimal detectable change;

^d-SD = standard deviation.

The MCID data presented will aid in the interpretation of clinical data regarding different upper extremity procedures. The MCID constitutes a threshold for outcome scores where a patient would consider a given change in score to be meaningful and worthwhile. It is critical to study MCIDs because they can be used for sample size calculation in clinical trials. MCIDs are also used to facilitate the establishment of treatment recommendations. We can investigate the effectiveness of interventions (i.e., physical therapy or drug therapy) by comparing the change of scores in patients with and without interventions by evaluating those that exceed MCID value or not. Therefore, the MCID data will also allow clinicians to better interpret changes in patients’ DASH scores over time.

There is no agreement as to the best approach for calculation the MCID. Crosby et al. recommended combined use of distribution-based and anchor based methods.⁸ Of the several types of distribution-based methods, we used half SD and MDC₉₀-based calculations because they are among the most universally applied distribution-based approaches. For anchor-based methods, we used the global rating of change scale to quantify patient improvement/deterioration over time.

At the time of assessment, patients were asked to independently rate the overall change in their lower extremity condition. While the anchor-based method correlated each patient’s perceived clinical change with changes in HRQoL scores, MCID values derived from the global rating of change scale may suffer from recall bias. In this study, the calculated distribution-based and anchor-based MCID values for the DASH questionnaires were similar (around eight points). Our results were also similar to MCID values previously reported in other orthopaedic conditions, allowing us to conclude that the MCIDs of the DASH calculated in this study in upper extremity orthopaedic oncology patients are reasonable.^{3,10,12,29,30} However, one should be aware that the MCID values can be within a range. Caution should be exercised when interpreting and using published MCID values at the individual level, especially when different anatomic sites and diseases are being studied. Also, there is a clear need for improvement and standardization of MCID methodology in the DASH questionnaire, as well as other PROMs, to allow finer discrimination and evaluation of clinical responsiveness.

The MCID constitutes a threshold for outcome scores where a patient would consider a given change in score to be meaningful. It is critical to study MCIDs because they can be used for sample size calculation in clinical trials. MCIDs are also used to facilitate the establishment of treatment recommendations. We can investigate the effectiveness of interventions (i.e. physical therapy or drug therapy) by comparing the change of scores in patients with and without interventions by evaluating those exceed MCID value or not. Since both anchor-based and distribution-based MCID estimates of the DASH in patients with musculoskeletal tumors of the upper extremity were quite similar, we have confidence in the estimates made, which were about 8 points. This suggests interventions improving DASH by less than that amount are unlikely to be perceived by patients as clinically meaningful. Therefore, those interventions may not justify exposing patients to risk, cost, or inconvenience. When applying novel interventions for orthopaedic oncology patients, including drugs or rehabilitation regimens, considering MCIDs will be vital to evaluate the clinical and potential statistical significance of the change. Also, the MCID-based approach will be helpful to identify subgroups of patients who could benefit from a specific intervention, even if the overall population did not benefit.

Limitations

This study has several limitations. First, we used several methods to calculate the MCID of DASH. Currently, there is no consensus regarding the optimal way to estimate the MCID. Although the half SD method is the most frequently used (approximately 90% of reports on MCID)^5,24, it is affected by the sample studied; it will vary depending on the sample size and distribution of results. The MDC₉₀ based approach may be more effective; however, it requires knowledge of the measure’s reliability.^2,3 Alternatively, the global rating of change scale, which was used as an anchor question, may not always be valid with strong correlation within all the DASH subscales, including physical function, severity of pain or stiffness, or problem associated with social activities, work, and sleep. Due to the heterogeneity of diagnoses and treatments, which are endemic in studies of patients undergoing musculoskeletal tumor surgery, patients with variable levels of function were included in this study increases the ability to generalize their results, but may limit them when finer discrimination is needed. In our sub-analysis, we also found baseline score (severity) affected MCID values of DASH. This heterogeneity reinforces the need for further evaluation of MCID in uniform patient populations based on disese- and reconstruction-specific analyses.

CONCLUSIONS

Since both distribution-based and anchor-based MCID estimates of the DASH in patients with musculoskeletal tumors were so similar, there is confidence in the estimates made, which were about 8 points. This suggests that interventions improving DASH by less than that amount are unlikely to be perceived by patients as clinically important. Therefore, those interventions may not justify exposing patients to risk, cost, or inconvenience. When applying novel interventions for orthopaedic oncology patients such as operations, implants, drugs, or rehabilitation regimens, considering MCIDs will be vital to evaluate. The MCID-based approach will also be helpful to identify subgroups of patients who could benefit from a specific intervention, even if the overall population did not benefit. Our findings provide benchmark values for MCID in musculoskeletal oncology, which serves as a reference for future studies of HRQoL in upper extremity musculoskeletal oncology patients using the DASH questionnaire. Further refinement of the analysis should be disease- and reconstruction-specific.