Minimal Clinically Important Differences in SF-36 Global Score: Current Value In Orthopedic Oncology

Abstract

The SF-36 is widely used to evaluate the health-related quality of life of patients with musculoskeletal tumors. Instead of typical methods, calculating the SF-36 Global Score has recently become an increasingly common reporting approach. However, numerical changes lack clear clinical relevance. The minimal clinically important difference is useful for interpreting changes in functional scores by defining the smallest change patients may perceive as clinically meaningful. The aim of this study is to determine the minimal clinically important difference of the SF-36 Global Score in orthopaedic oncology patients, which has not been reported to date. Three-hundred ten patients who underwent surgery and completed two surveys during postoperative follow-up were reviewed. We used the two most common methods to calculate the SF-36 Global Score: 1) anchor-based methods and receiver operating characteristic analysis based on one-half of the standard deviation of change score and standard error of measurement at baseline and; 2) distribution-based methods. Using anchor-based methods, the minimal clinically important differences of SF-36 Global Scores #1 and #2 were 2.7 (area under the curve [AUC] = 0.85) and 2.5 (AUC = 0.79) for improvement, and −1.5 (AUC = 0.81) and −0.6 (AUC = 0.83) for deterioration, respectively. Using distribution-based methods, the minimal clinically important differences of SF-36 Global Scores #1 and #2 were 4.1 and 4.4 by half standard deviation, and 4.1 and 4.5 by the standard error of measurement, respectively. Our findings provide benchmark values, which can serve as a reference for future studies in musculoskeletal tumor patients using the SF-36 Global Score as a single measure for health-related quality of life.

Trial Registration:

Retrospectively registered.

BACKGROUND

Patient-derived outcome assessments are essential determinants of health-related quality of life (HRQoL) in oncology and help inform treatment decisions for future musculoskeletal oncology patients. There has been a burgeoning of publications regarding HRQoL.¹ More generic instruments, such as SF-36 or Patient Reported Outcomes Measurement Information System (PROMIS), are validated internationally for general orthopaedic conditions.^{2; 3} However, these have not been evaluated critically in musculoskeletal oncology, although they have been used to assess correlations with the six subscales of the Musculoskeletal Tumor Society (MSTS) score.⁴

Statistical significance is typically calculated to study how scores change over time. However, the clinical significance of changes in these scores is not clearly understood. The minimal clinically important difference (MCID), defined as the smallest change that a patient can perceive as being meaningful, has been determined in several orthopaedic conditions, but not musculoskeletal oncology.^5–8

The most common use of SF-36 is based on two component scores (mental component summary [MCS] and physical component summary [PCS]) from eight subscales. However, a single underlying construct of HRQoL in SF-36 was indicated which would correspond to an SF-36 Global Score indicating global health of the patients. Since then, SF-36 Global Scores are being used increasingly to assess global health of the patients in published studies due to simplicity.^9–11 The aim of this study was to determine the MCID for SF-36 Global Scores in orthopaedic oncology patients treated with surgery for a wide range of musculoskeletal tumors, which has not been reported to date.

METHODS

Patients and data collection

This study’s Level of Evidence was Level III, diagnostic study. Each author certifies that the study institution approved the human protocol for this investigation, and all investigations were conducted in conformity with the ethical principles of research. The analysis reported in this study was approved by Memorial Sloan Kettering Cancer Center’s institutional review board protocol number 16–913.

The Musculoskeletal Outcomes Data Evaluation and Management System (MODEMS) instrument, which combines multiple outcomes instruments, including the SF-36, was implemented as part of our institutional clinical assessment for all patients. This was in accordance with the developed and validated instrument from the American Academy of Orthopaedic Surgeons and endorsed in 1994 by the Musculoskeletal Tumor Society after its multispecialty consensus meeting in Tarpon Springs, FL, USA.¹²

All patients who underwent surgery and evaluated for HRQoL using the MODEMS instrument at three months and then every six months after surgery were reviewed from 1999 to 2005. At each follow-up appointment, patients were presented with a survey were asked the following anchor question, “Compared to when you last completed this questionnaire, is your musculoskeletal condition, ‘much better,’ ‘somewhat better,’ ‘about the same,’ ‘somewhat worse,’ or ‘much worse’?”. Of 535 patients enrolled in the MODEMS database, we excluded patients with no follow-up data (patients underwent only a single survey, n = 213), or those with insufficient HRQoL data (missing value on SF-36 PCS or MCS or anchor question, n = 12) (Figure 1).

Figure 1.

Flow diagram showing patient eligibility criteria for analysis.

Instrument

MODEMS was used, which incorporates components from the SF-36 (v1, four-week recall), along with other postsurgical functional and HRQoL parameters. The SF-36 measures of health on eight multi-item dimensions covering functional status, well-being, and overall evaluation of health (i.e. physical functioning [PF], role-physical [RP], bodily pain [BP], general health [GH], vitality [VT], social functioning [SF], role-emotional [RE], and mental health [MH]).¹³ Patients were asked to rate their responses using 2-, 3-, 5-, or 6-point polytomous response options. For each dimension, item scores are calculated, summed, and transformed to a scale ranging from zero (worst health) to 100 (best health). These had a mean score of 50 and standard deviation (SD) of 10, in accordance to guidelines presented by John Ware, SF-36/ SF-12 manuals, and interpretation guides.^{14; 15}

We used the two most common methods for estimating SF-36 Global Score: the arithmetic average for eight subscales (SF-36 Global Score #1) and component summary (SF-36 Global Score #2) using the following formulas, respectively¹⁶:

$$\mathit{\text{SF}}-36\mathit{\text{Global}}\mathit{\text{Score}}\#1=\left(PF+RP+BP+GH+VT+SF+RE+MH\right)/8$$

$$\mathit{\text{SF}}-36\mathit{\text{Global}}\mathit{\text{Score}}\#2=\left(\mathit{\text{PCS}}+\mathit{\text{MCS}}\right)/2$$

MCID Calculation

MCID helps with sample size calculation and interpreting results in clinical trials.¹⁷ Anchor-based and distribution-based methods are the established approaches to calculate MCID. Anchor-based methods also establish independent criteria that patients and doctors believe to be important indicators of improvement or worsening of each patient’s condition. The criteria identify the minimal important variation from the anchor. This method has the advantage of explicitly identifying what constitutes the difference that is of, “minimal importance.” In distinction, the statistical distribution in the population uses a standardized metric to determine the observed variation, which may be the standard error of the mean (SEM), SD, or effect size. Many studies have shown that one half of the SEM or SD corresponds to the MCID.^{18; 19} This strategy identifies the extent of a change from baseline values beyond chance and fails to establish whether the observed change is actually important.

Some recommendations have been proposed regarding the best method to apply for determining robust and reliable MCIDs. For example, many clinicians believe that anchor-based calculations of the MCID, which define, “clinically important” in relation to changes identified as important by the patients themselves, are more relevant to clinical practice than distribution-based methods.^20–22 Although this approach can seem subjective, asking patients about changes in their health status to determine their individual perceptions is critical. However, others criticize the anchor-based method because of the impact of potential response shift effect due to the longitudinal design used which may bias the results.²³

With these issues in mind, Crosby et al. recommended the combined use of anchor-based and distribution-based methods to take advantage of an external criterion, a measure of variability, and compensate for disadvantages of each method.²⁰ Based on the review article on cancer studies related to determination of MCIDs²⁴, >80% of the studies examined used both distribution- and anchor-based methods, suggesting the importance of cross-referencing the MCIDs calculated by two different methods to verify the validity of each value. Therefore, we combined both anchor- and distribution-based methods to determine the MCIDs of the SF-36 Global Score in the postoperative setting following surgical treatment of musculoskeletal tumors.

Anchor-based methods

Anchor-based methods were used to determine cut-off values for MCIDs based on the answer to each anchor question, which is the gold standard for assessing the change of each patient’s condition.²⁵ Receiver operating characteristic (ROC) curve analysis for a change in SF-36 Global Score between two surveys were performed to compute a discrete value for the MCID by evaluating a threshold delta in the SF-36 Global Score that yielded the smallest difference between sensitivity and specificity. The ROC-derived MCIDs were taken to be the change in scores from the baseline with sensitivity and specificity to distinguish between patients who reported their outcome as, “About the same” and those who reported their status as, “Somewhat better” for improvement and, “Somewhat worse” for deterioration. The discriminative ability of the model was assessed using the area under the ROC curve (AUC).

Distribution-based methods

MCIDs were calculated using two different statistical characteristics of the distribution of the scores at the baseline. First, the MCIDs by the half SD method were calculated,¹⁹ where minimal change is considered as the half of SD of change scores. Second, the minimum amount of change potentially detectable using the SEM was estimated. The SEM was calculated using the following formula, where SD is for the baseline scores, and R stands for the reliability coefficient:

$$\mathit{\text{SEM}}=\mathit{\text{SD}}\times \mathit{\text{SQRT}}(1-R)$$

For the reliability coefficient, 0.90 was used for the SF-36 Global Score in accordance with previous reports.^{5; 26}

Statistical analysis

All statistical analyses were conducted using IBM SPSS version 18.0 (IBM SPSS, Armonk, NY, USA). The scores were reported as the mean values ± SD. The AUC was interpreted as defined previously²⁷: 0.90–1.00, excellent; 0.80–0.90, good; 0.60–0.80, fair; and 0.50–0.60, failed.

RESULTS

The mean duration between the surveys was nine months. The clinical and demographic characteristics of the participants are detailed in Table 1.

TABLE 1.

Patient demographics

Characteristic	Number of patients (%)
Overall	310 (100)
Age, years; mean [standard deviation]	54.1 (range, 14–87) [16.7]
Sex
Male	157 (50.6)
Female	153 (49.4)
Histologic diagnosis
Chondrosarcoma	38 (12.3)
Undifferentiated pleomorphic sarcoma	26 (8.4)
Benign lesion	24 (7.7)
Osteosarcoma	23 (7.4)
Chordoma	14 (4.5)
Synovial sarcoma	11 (3.5)
Ewing’s sarcoma	9 (2.9)
Liposarcoma	7 (2.3)
Leiomyosarcoma	7 (2.3)
Fibrosarcoma	6 (1.9)
Myxofibrosarcoma	6 (1.9)
Desmoid tumor	5 (1.6)
Skin cancer	5 (1.6)
Other soft tissue sarcoma	11 (3.5)
Cancer metastasis	118 (38.1)
Type of surgery
Wide resection	129 (41.6)
Wide resection + endoprosthetic reconstruction	73 (23.5)
Wide resection + allograft reconstruction	17 (5.5)
Curettage +/− arthroplasty/internal fixation	36 (11.6)
Open reduction and internal fixation	12 (3.9)
Amputation/rotationplasty	10 (3.2)
Others*	33 (10.6)

^*Spinal decompression, surgery for nonunion, etc.

The baseline SF-36 Global Scores, distribution of answers to each anchor question, and change of SF-36 scores (follow-up score - baseline score) are summarized in Table 2. Based on the anchor question, most patients answered either, “About the same” (39.7%) or improved, noted in “Much better” or, “Somewhat better” responses (40.6%). The relationship between changes of SF-36 Global Scores and answers to anchor questions is presented using a box-and-whiskers plot. This demonstrated a positive correlation between changes in both SF-36 Global Scores #1 and #2 and patient-reported change in disease condition (Figure 2).

TABLE 2.

Details of SF-36 scores and anchor question

Characteristic	Number of patients (%)
Baseline survey
SF-36
Global Score #1; mean [SDa]	44.0 [9.3]
Global Score #2; mean [SD]	44.3 [9.9]
Follow-up survey
Anchor question
”Much better”	55 (17.7)
“Somewhat better”	71 (22.9)
“About the same”	123 (39.7)
“Somewhat worse”	46 (14.8)
“Much worse”	15 (4.8}
Change of the SF-36 scores
Global Score #1; mean [SD]	+1.9 [7.6]
Global Score #2; mean [SD]	+1.2 [6.7]

^a-SD, standard deviation.

Figure 2.

Box-and-whiskers plots demonstrate the relationship between change of SF-36 Global Scores #1 (A) and #2 (B) and also displays answers to each anchor question.

MCID of SF-36 Global Scores

Table 3 shows the MCID of SF-36 Global Scores. Using the anchor-based method, the discrete MCID values that yielded the smallest difference between sensitivity and specificity are shown and suggested good discriminative ability of the model (Figure 3).²⁷ As expected, patients required an increased change in score to perceive improvement than deterioration (Table 3).

TABLE 3.

MCID^a of SF-36 Global Scores

Method of calculation	SF-36 Global Score
	#1	#2
Anchor-based method, ROCb
Improvement	2.7 (AUCc, 0.85)	2.5 (AUC, 0.79)
Deterioration	−1.5 (AUC, 0.81)	−0.6 (AUC, 0.83)
Distribution-based method
Half SDd	4.1	4.4
SEMe	4.1	4.5

^a-MCID, minimal clinically important difference;

^b-receiver operating characteristic;

^c-AUC, area under the curve;

^d-SD, standard deviation;

^e-SEM, standard error of measurement.

Figure 3.

The ROC curves for improvement (A) and deterioration (B) plots the true positive rate (sensitivity) against the false-positive rate (1 - specificity) for SF-36 Global Scores #1 (green line), and #2 (blue line).

DISCUSSION

As survival improves for orthopaedic oncology patients, increased interest has developed in patient-reported outcomes (PROs), particularly for function and HRQoL.^28–30 The SF-36 is a generic measure of HRQoL and is widely used in healthcare to evaluate patient’s self-perceived health.^{31; 32} Interpreting outcome measures presents a challenge, as statistical significance does not always translate into clinical significance. Therefore, meaningful interpretation of these PROs requires an understanding of the MCID required to interpret a change in status. The SF-36 Global score works effectively to characterize outcome for musculoskeletal oncology patients.¹⁶

Anagnostopoulos et al. used a structural equation model analysis to investigate the latent structure of SF-36 in a population-based sample of adults³³. Three orders of latent structure of SF-36 were identified: the first-order factors were eight SF-36 subscales. These displayed three second-order factors as PCS, MCS, and “well-being” based on GH and VT domain scores. They also showed a third-order factor, which would correspond to an SF-36 Global Score, indicating that all SF-36 responses address a single underlying construct of HRQoL³³

In the scoping review by Lins et al.¹⁶, only one of the reports discussed involved a study in oncology.³⁴ Cookson et al. stated that analyses of SF-36 dimensionality denote cross-loadings from the eight domains and also between PCS and MCS.³⁴ These findings would suggest the existence of uni-dimensionality that would support the calculation of a single index of HRQoL. However, they noted that there has not been full validation of it. Our study of the MCID of the SF-36 Global score is an important element that helps to validate its clinical utility. These results justify the use of the SF-36 Global Score as a simple single measure of HRQoL. Furthermore, current trends in data reporting support the use of the SF-36 Global Score as a simple, single measure of HRQoL. Other instruments recommend the calculation of a single index of quality of life, such as the Pediatric Quality of Life Inventory, EuroQol-5D, and the 15-D.¹⁶ Given this trend of using the SF-36 Global Score as a simple, single measure of HRQoL, there is a compelling need for guiding metrics to interpret the clinical significance of changes of SF-36 Global Score. These are essential for the SF-36 Global Score to be used in meaningful contexts in clinical care.

Based on the user’s manual for the SF-36 health survey, a distributional approach applied to a representative United States general population sample produced the following estimates of MCID values for each SF-36 score: 2 points for RP, 4 points for RE, 3 points for other 6 subscales, and 3 points for PCS and MCS.³⁵ Based on these estimates, the estimated MCIDs of SF-36 for each of the calculated total scores (#1, the average of MCIDs for each of the eight subscales, and #2, average of MCIDs for the PCS and MCS) would be estimated to be three points each.³⁵ However, there is no prior data available regarding the MCID of SF-36 Global Score. The MCID data for improvement presented in the current study seems to be valid based on the estimates of MCID values provided in the SF-36 health survey user’s manual. The lower MCID values for deterioration suggested that the SF-36 is more responsive to deterioration than to improvement, a phenomenon also reported for other HRQoL measures.^36–38

Although the SF-36 was not originally designed to examine economic evaluation, SF-6D was developed for use in economic evaluation studies, which is a single index ranging from 0.0 (worst health state) to 1.0 (best health state). It can be used in the assessment of the quality adjusted life years and the cost-effectiveness of various healthcare interventions, including EQ-5D.^{39; 40} The SF-6D focuses on seven of the eight subscales covered by the SF-36: PF, SF, BP, MH, VT, and RP/RE. Only the GH domain is not included. The SF-6D is widely validated in several studies, including MCID ranging from 0.026–0.041.^41–44 However, it is designed for economic evaluation studies and not for measuring health-related quality of life.

Several limitations of this study should be noted. First, the global rating of change scale, which we used as an anchor question, may not always be valid for every subscale in SF-36. The use of multiple anchor questions would be ideal, considering SF-36 contains multiple dimensions of HRQoL. However, no previous literature has validated the MCID of SF-36 using such sophisticated methodology. We also confirmed moderate to strong correlation between anchor question and change in the scores to verify the anchor we used is valid for both Global Scores (Spearman’s correlation coefficient, 0.54 and 0.63 for Global Scores #1 and #2, respectively). Second, MCID values derived from the global rating of change scale using anchor-based approaches may suffer from intrinsic weakness, such as subjective retrospective judgment of change. The validity of the scale used for the global assessment may or may not be accurate; it is also vulnerable to recall bias, and not equally appropriate for each dimension of HRQoL. Third, due to the heterogeneity of diagnoses together with the diverse range of treatments (a common problem in studies of musculoskeletal tumor surgery), patients with variable levels of function were included in this study. Widely-varied operations can be evaluated by the same instruments. However, the outcomes are different.⁴⁵ This heterogeneity reinforces the need for further evaluation of MCID in a more uniform patient population. Finally, the MCIDs presented cannot necessarily be applied to patients with other disease conditions; the MCID can be impacted by population characteristics, such as disease type and severity.⁴⁶ Physicians who use these metrics need to carefully interpret them, especially when evaluating patients with non-orthopaedic oncologic disease conditions.

The MCID values were calculated for the SF-36 Global Scores based on both anchor- and distribution-based approaches. The consistency of our results across different approaches suggests that a change in score of 2.5 to 4.5 for improvement and −0.6 to −1.5 for deterioration is meaningful for patients with musculoskeletal tumors. This suggested SF-36 is more responsive to deterioration than to improvement, a phenomenon also reported for other measures. Our findings provide benchmark values for MCID of SF-36 Global Score, which can serve as a reference for future studies of HRQoL in musculoskeletal tumor patients using the SF-36 Global Score as a single measure for HRQoL.

Availability of data and materials

The data supporting the findings of this study are available upon request, but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of the study institution.