Reporting the Pooled Last Follow-up Musculoskeletal Tumor Society Score Underestimates Functional Improvements Seen Over Time

Abstract

Introduction:

The Musculoskeletal Tumor Society (MSTS) score is the most commonly used functional outcome tool in Orthopaedic Oncology. However, there are no published recommendations on how to collect, analyze, and report these data. To address this knowledge gap, we performed (1) a systematic review to analyze the methodology of reporting MSTS scores and (2) an analysis of two different reporting methods (pooled last follow-up versus longitudinal time based).

Methods:

The mean MSTS score and length of follow-up were calculated using data provided in articles included in the systematic review. Paired t-tests were used to compare mean MSTS scores calculated at standardized postoperative intervals versus mean MSTS scores calculated from last follow-up time points using prospectively collected data from a single institutional database.

Results:

More than 300 articles met inclusion criteria. Most authors reported a single pooled score calculated from MSTS scores obtained at the patients' last follow-up visits (91.6%, n = 356). When comparing this method (pooled reporting) versus longitudinal time-based reporting, the pooled MSTS scores were only comparable to the MSTS scores at the 6-month postoperative visit. The pooled last follow-up scores underestimated MSTS scores obtained after 6 months (P < 0.0001 at 1 year and P = 0.0047 at 3 years). Improvements were seen in the MSTS domains after the 6-month visit (average time to stability was 1.8 years).

Conclusion:

The most common historical method of reporting MSTS scores, the pooled last follow-up method, underestimates functional improvements observed during long-term follow-up. Longitudinal time-based reporting provides more accurate representation of the functional outcomes over time.

The effect of surgery on patients with cancer can be difficult to evaluate. Historically, surgical and oncologic outcomes, such as local recurrence rates, overall, and disease-specific survival, were indicators of success.^1-5 However, in 1987, Dr. William F Enneking developed a tool to measure functional outcomes and impairment following surgical resection of appendicular tumors.⁶ The original Musculoskeletal Tumor Society (MSTS) score was a collaborative initiative between the MSTS and the International Symposium on Limb Salvage.^2,6 The 1987 version (MSTS87) was a physician/provider scored outcome measure that had seven domains reported using one of four descriptive terms: “Excellent, Good, Fair, and Poor.” The final rating was based on an algorithm of the number of items with each descriptor. In addition, a numerical score was calculated by assigning values to each of the seven items using 5 = excellent, 3 = good, 1 = fair, and 0 = poor.⁶ In 1993, Enneking et al² published an updated MSTS score that was designed as a hybrid tool based on inputs from both the provider and the patient. In the new format, the patient rated their emotional acceptance on a scale from 0 to 5, with a higher score reflective of greater acceptance.² The provider rated the patient's pain and physical function on the five-point scale, with a higher score indicating no pain or use of pain medication, as well as restored function. For patients with lower extremity (LE) procedures, the use of supports, walking ability, and gait were assigned a score of 0 to 5. Both a numerical score and percentage were calculated to facilitate valid comparisons of the functional outcome score. The MSTS87 scores could be converted to numerical values and reported as a percentage.² The updated hybrid version of the MSTS score (MSTS93) has been widely accepted as the functional outcome tool of choice because of the reported low interobserver variability.⁴ To date, there are no published recommendations for how clinicians should collect, analyze, and report these data.^3,5 We aim to evaluate how the MSTS scores are historically collected and reported in the orthopaedic oncology literature. Furthermore, we aim to analyze the effect of reporting MSTS scores at multiple time points versus as a single time point.

Methods

To understand the historical trends in collecting and reporting MSTS scores in the literature, a systematic review was conducted and reported using the Preferred Reporting Items for Systematic Review and Meta-Analysis guidelines. Four reviewers (A.K., G.M., J.B., and T.N.) independently reviewed the literature. Disagreements were resolved by discussion. English language articles published from 1987 to 2016 in the OVID, Medline, Embase, CINAHL Plus, PsychINFO, Database Health and Psychosocial instruments, Cochrane library, or PubMed databases were screened. The search strategy identified articles related to the MSTS functional outcome scores. After removal of duplicates, references were saved to an EndNote library. Distiller SR software (Evidence Partners) was used to screen the EndNote library. Articles were excluded at level 1 through a title and/or abstract review if there was no orthopaedic surgery on a human, the full-text article was not available, or if the abstract did not mention the use of the MSTS score. Articles or abstracts were excluded at a level-2 review based on the following criteria: description of surgical or treatment methodology only, review article or editorial comment, case series less than four patients, molecular, genetic, or laboratory study only, or use of the Enneking criteria but not the Enneking MSTS score. The remaining articles were evaluated for the methodology of collecting and reporting MSTS functional outcome scores (Figure 1). Articles with tables of MSTS scores were used to calculate the mean MSTS score, the range, and mean length of follow-up for patients reported in each article.

Figure 1.

Flow chart showing the Preferred Reporting Items for Systemic Review and Meta-Analysis details of the number of articles identified, included, and excluded.

To analyze the effect of reporting scores at multiple postoperative time points, a retrospective review of our longitudinal orthopaedic oncology database of patients treated at our institution between January 2012 and April 15, 2022, was approved by our institutional review board. Patients with impending or completed pathologic long bone fractures in the LE were identified and included if they had surgical stabilization of a long bone by either open reduction and internal fixation or insertion of an intramedullary nail. Patients without postoperative MSTS93 scores were excluded. Demographic data, including patient age, sex, race, ethnicity, cancer type, anatomic site, and surgical details, were collected for each patient. MSTS93 scores were collected at standardized intervals: preoperatively, when possible, and postoperatively at 2 weeks, 6 weeks, 3 months, 6 months, and annually thereafter (Figure 2). The pain, function, use of supports, walking ability, and gait domains were completed by the provider at every visit. The emotional acceptance MSTS domain was completed by the patient outside of the presence of the surgical team. The mean MSTS93 scores, the range, and mean length of follow-up were calculated for each patient in our cohort. The pooled score was calculated from the MSTS93 score that was recorded at the patient's last (most recent) follow-up clinic visit (Figure 3). A comparison of the pooled score with scores at each postoperative interval was done using paired Student t-test. Significance was set at P < 0.05. A domain analysis of patients with long-term follow-up was done. Time to stability was calculated for each domain by identifying the earliest postoperative visit where scores remained unchanged thereafter.

Figure 2.

Graphical representation showing how MSTS93 scores were collected at standardized postoperative intervals. MSTS = Musculoskeletal Tumor Society

Figure 3.

Graphical representation showing the pooled MSTS93 score calculated from values obtained at the patient's last follow-up visit. MSTS = Musculoskeletal Tumor Society

Results

To understand the historical trends in collecting and reporting MSTS scores in the literature, more than 3,000 articles and abstracts were assessed for eligibility. Following a two-level review, 368 met inclusion criteria (Figure 1). Full-text articles were available for 96.7% (n = 356) of literature, and most were retrospective studies (88.6%, n = 326). Functional outcome scores were most often reported using the MSTS93 score (89.9%, n = 331); however, 10.6% (n = 39) of articles reported data using the MSTS87 format. Eight articles reported values in both the 1987 and 1993 formats after the MSTS93 revised version was published. The MSTS93 score was most frequently reported as a percentage of the mean (68.8%, n = 253) and/or range of scores (69.3%, n = 255). In 51.9% (n = 191) of articles, the raw mean was reported. Only 47.6% of authors reported a median MSTS value (n = 175). Forty percent of articles included a table of MSTS scores, which allowed the standard deviation and confidence interval to be calculated in 55.7% and 42.7% of articles, respectively. Ten percent of the publications (n = 37) did not report numerical MSTS scores, instead they reported the descriptive response term. Of those articles that reported a descriptive term, a definition of score ranges that represent each term was not provided.

Only 6.5% of articles indicated that patients themselves answered the emotional acceptance item. Most authors (91.0%, n = 334) had clinical staff complete the entire MSTS assessment on behalf of the patient. The MSTS was used as a longitudinal assessment in 12.2% of the articles. Therefore, we were unable to evaluate the effect of reporting MSTS93 scores at multiple time points using this historical data. The most common historical method of reporting the MSTS score was as a single value, at the patient's last follow-up visit (91.6%). However, the follow-up time frame varied considerably from patient to patient, often within each article. Of the articles that report a single pooled MSTS93 value, the time of last follow-up ranged from 1 week to 39 years, with a mean value of 10.4 ± 6.9 years (n = 295). In four articles, the last follow-up score that was reported was calculated by the clinician from a chart review after the patient died.

To evaluate the effect of reporting MSTS93 scores at multiple postoperative time points, we did a retrospective review of our orthopaedic oncology surgical database. Initially, a total of 234 cases with an impending or completed long bone LE pathologic fracture between 2014 and 2022 were screened for eligibility. Eight patients were excluded because only preoperative data were available for review. One patient was excluded for open reduction and internal fixation of the patella. A total of 171 surgical procedures were performed on 162 patients. The MSTS93 scores were collected over 529 visits and were included in this analysis. The study population was 57.4% (n = 93) female, 72.2% (n = 117) White, 14.2% (n = 23) African American, and 2.5% (n = 4) Asian, with a mean age of 56.8 ± 17.5 years at the time of surgery. The most common diagnoses were renal cell carcinoma (19.1%, n = 31) followed by multiple myeloma (17.3%, n = 28) (Figure 4). Stabilization with an intramedullary nail was the most common procedure performed (71.3%, n = 122), and the femur was the most common location (80.7%, n = 138) followed by tibia (14.6%, n = 25). The mean MSTS93 score improved from the preoperative score by the 6-month postoperative visit (P = 0.036821) and continued to improve until four postoperative years (P = 0.029245 at 4 years). The mean pooled LE MSTS93 score approximates the LE MSTS93 score at 6 months and underestimates most time points beyond that visit (P < 0.0001 at 1 year and P = 0.0047 at 3 years) (Figure 5). Even in our institutional database, the range of last follow-up varies considerably between patients with a mean last follow-up day of 508.95 ± 632.6 days.

Figure 4.

Graph showing the primary diagnoses for patients treated for impending or pathologic long bone fractures.

Figure 5.

Graph showing the mean MSTS 93 lower extremity score at standardized preoperative (1) and postoperative intervals at 2 weeks (2), 6 weeks (3), 3 months (4), 6 months (5), 1 year (6), 2 years (7), 3 years (8), 4 years (9), 5 years (10), 6 years (11), 7 years (12), and pooled last follow-up (14). MSTS = Musculoskeletal Tumor Society

A domain analysis was done on all patients with more than 6 months of postoperative data (53.1%, n = 86). Seventy-six patients (46.9%) did not have scores beyond the 6-month postoperative visit due to death from disease or complications of treatment. An analysis of the long-term follow-up patients identified change in all domains except emotional acceptance after the 6-month postoperative visit (Table 1). Our data show that less than half of patients changed their perceived emotional acceptance score after 6 months. However, all the domains related to pain, physical function, use of walking aids, and gait continued to change for most patients beyond the 6-month visit. Most scores did reach stability around the second annual postoperative visit (Table 1).

Table 1.

Domain Analysis of the Musculoskeletal Tumor Society Lower Extremity Scores of Patients With Follow-up Longer Than 6 Months

	Pain	Function	Emotional Acceptance	Use of Walking Aids	Walk	Gait
% change	69	66	45	57	64	66
Average time to stability (yr)	1.95	1.8	1.6	1.7	1.8	1.8

Discussion

A patient's postoperative function is difficult to accurately measure and report. Although objective clinical outcomes such as range of motion, muscle group strength, or timed up and go are easy to quantify, they do not encompass all the factors associated with physical function. The MSTS93 score was developed for musculoskeletal tumor surgeons to measure and compare patient outcomes after resection of appendicular tumors. Pain, physical function, and emotional acceptance are graded on a five-point scale, with a higher score indicative of greater function and less pain.² In 1993, the authors of the original questionnaire (MSTS87) updated it to include inputs from both the care team and the patient. To date, there are no published guidelines on how the MSTS score should be administered or reported in the literature, and our review found variability in the utilization, collection, and reporting of data generated from the MSTS87 and MSTS93 questionnaires. For example, our review found 39 articles that used the MSTS87 scoring system and eight articles that reported values in both the 1987 and 1993 formats after the MSTS93 revised version was published. We found that many publications that include MSTS93 scores did not obtain direct patient input. Our data suggest that only 6.5% of publications included emotional acceptance scores directly from the patient. It is unclear how close the clinician's estimation of a patient's emotional acceptance was and if this markedly affected the reported scores. We do know that as time progresses from surgery, the oncology surgeon overestimates the patient's satisfaction. This has been confirmed by Janssen et al⁴ who compared patient-reported and clinician-reported MSTS scores in metastatic bone disease and found that providers overestimate the score by an average of eight points.

Reporting outcomes data with the hybrid MSTS93 tool emphasizes the importance of the patient's perspective of their function when assessing outcomes. The importance of good psychological outcomes for oncologic patients has been well documented in the literature.^7-10 Xu et al¹⁰ studied patient emotional acceptance following scapulectomy for malignant shoulder tumors, and they emphasize the importance of obtaining not only good oncologic outcomes but also high emotional acceptance even in patients with metastatic disease. Understanding this relationship is key to interpreting and using published data to counsel patients. In addition, our review indicates that most publications report a single postoperative score, usually at the patient's final follow-up visit. However, we found that the range of last follow-up varied from 1 week to 39 years with a mean value of 10.4 ± 6.9 years. The usefulness of the MSTS93 last follow-up scores is difficult to determine because the time reported as last follow-up varied greatly from months to years in many publications.¹¹ In statistics, the term pooling is used to describe the practice of gathering small sets of data that are assumed to have the same value of a characteristic (eg, a mean) and using the combined larger set (the “pool”) to obtain a more precise estimate of that characteristic. To evaluate the effect of reporting pooled last follow-up data, we retrospectively reviewed the LE MSTS93 scores of patients undergoing stabilization for impending or pathologic long bone fractures and compared them with values obtained at standardized postoperative time points. All patients were being treated for metastatic disease and were a mean age of 59 years old at the time of surgery. The MSTS93 LE scores for patients in this metastatic disease cohort continuously improved from preoperative to 3 years postoperative (P < 0.0001 at 1 year and P = 0.0047 at 3 years). Our domain analysis lost-to-follow-up rate was low, with more than half of the cohort available for analysis beyond the 6-month postoperative visit despite their metastatic disease diagnosis. We found that the emotional acceptance domain stabilizes early with more than 50% of the cohort reporting unchanged scores after the 6-month postoperative visit. However, the pain, physical function, use of walking aids, and gait-related domains continue to change after the 6-month visit. We found the time to stabilization was closer to the 2-year postoperative visit. A recent article by Westlake et al¹² noted similar findings. In their study, 66% of patients had optimized their MSTS93 score by three postoperative months. The longest time to return of optimal function was noted to be 2.8 years after tumor surgery.¹² In their study, patients with cancer, procedures involving bone, and patients treated with chemotherapy and/or radiation had statistically significant longer return to optimal function.¹² The MSTS score has been criticized in the literature as being of limited value, especially in high functioning patients that return to preoperative function early in their recovery period.¹³ This ceiling effect has been frequently cited as the need for updated functional assessment tools.¹³ However, our data suggest that the current practice of reporting pooled follow-up data masks important functional improvements seen over time. Reporting at standardized time points captures and accurately reports these functional improvements and should be used to counsel patients during their recovery period rather than pooled aggregate data.

This study has some strengths and limitations. To our knowledge, this is the first review of its kind to evaluate the variation in the methodology and reporting of the MSTS score in the literature. In addition, strong adherence to the Preferred Reporting Items for Systematic Review and Meta-Analysis guidelines and the use of four independent reviewers are strengths of this publication. One weakness is related to the search parameters. Although we conducted a robust review of the musculoskeletal literature, the database search was done using terms for studies about the MSTS score and functional outcomes. These indexed terms may not comprise all published literature, and some references may not have been identified and included. In addition, only English language articles were included, and these criteria may have excluded articles that are relevant. Next, the small sample size and heterogenous patient population may make confounders like chemotherapy or radiation therapy difficult to exclude. However, this is reflective of the challenges patients face during their recovery and likely increases the applicability of our findings.

More research is needed to understand the patient, physician, and system-related factors that make obtaining reliable data difficult. Redundant surveys and survey fatigue have often been cited as a patient-related barrier. Our group has recently validated a concise 14-item functional assessment tool derived from the 50-item Toronto Extremity Salvage Score for the LE and the Lower Extremity Functional Scale.¹⁴ Using item response theory modeling, a new questionnaire was created addressing redundancy while preserving the sensitivity and specificity when compared with the Toronto Extremity Salvage Score for the LE and Lower Extremity Functional Scale.¹⁴ Similar studies are needed to validate concise functional outcome measures in orthopaedic oncology patient populations that minimize the effect of other patient-related and physician-related barriers.

Conclusion

This systematic review highlights the current flaws in obtaining and reporting the MSTS93 score in the orthopaedic oncology literature. Reporting the MSTS93 score at a single time point underestimates the functional improvements patients make during their recovery. In our institutional cohort, the pooled last follow-up MSTS93 scores approximate scores obtained at the 6-month follow-up visit and underestimate the MSTS93 scores obtained at long-term follow-up. We recommend reporting MSTS93 scores at standardized postoperative intervals rather than a single score.