Use of scoring systems for assessing and reporting the outcome results from shoulder surgery and arthroplasty

Abstract

To investigate shoulder scoring systems used in Europe and North America and how outcomes might be classified after shoulder joint replacement. All research papers published in four major journals in 2012 and 2013 were reviewed for the shoulder scoring systems used in their published papers. A method of identifying how outcomes after shoulder arthroplasty might be used to categorize patients into fair, good, very good and excellent outcomes was explored using the outcome evaluations from patients treated in our own unit. A total of 174 research articles that were published in the four journals used some form of shoulder scoring system. The outcome from shoulder arthroplasty in our unit has been evaluated using the constant score (CS) and the oxford shoulder score and these scores have been used to evaluate individual patient outcomes. CSs of < 30 = unsatisfactory; 30-39 = fair; 40-59 = good; 60-69 = very good; and 70 and over = excellent. The most popular shoulder scoring systems in North America were Simple Shoulder Test and American shoulder and elbow surgeons standard shoulder assessment form score and in Europe CS, Oxford Shoulder Score and DASH score.

Core tip: We have identified the most commonly used shoulder scoring systems used when results of surgery are published. The constant score (CS) can be used to categorize the outcomes after shoulder arthroplasty into unsatisfactory; fair; good; very good; and excellent. This be carried out using both the original CS and the Adjusted (for age and sex) CS. For the majority of orthopaedic surgeons the reporting of outcomes in this way is clearer than providing the mean and standard deviation of one of the commonly used shoulder scoring systems.

INTRODUCTION

Functional shoulder disabilities are frequently reported by patients with a wide variety of conditions, such as arthritis, rotator cuff disease, breast carcinoma, trauma and radiation therapy[1]. Upper limb disabilities may include pain, stiffness, decreased range of movement, lymphoedema and reduced activity tolerance. There are a myriad of scoring systems used in everyday orthopaedic practise and research, and a number of scoring systems have been developed and established to assess the function and limitations of the shoulder. None of these are ideal methods for correlating both the physician’s and the patient’s perspective[2] as some are more inclined towards the opinion of the physician while others are more centred on the patient’s self-assessment. A review of four popular journals (see below) showed that in their 2012 volumes 16 different scoring systems were commonly used to evaluate shoulder conditions. This can make it difficult to compare outcomes between studies and interventions, and makes it more problematic for practising orthopaedic surgeons to decide which of their treatment options is providing the best outcomes if the scoring systems used are different. This review aims to evaluate the evidence behind the scoring systems, and will provide some guidance as to which to rely on.

Many shoulder scoring systems are used inappropriately without being tested for their sensitivity, reproducibility and validity[3]. In order to apply any scoring system, their complete validation must be documented. Over the last 5 years the use of patient reported outcome measures (PROMs) have been promoted as they are considered to place a greater emphasis on the quality of life perceived by the patient[4]. There is therefore a need to develop questionnaires or shoulder scoring outcome measures which can address specific conditions or abnormalities of the shoulder so that they can fulfil both the patient’s and clinician’s perspective[5]. Above all, shoulder scoring systems need to be evaluated and modified for specific condition to produce a valid, sensitive and reproductive outcome.

COMMONLY USED SCORING SYSTEMS

Constant score

The constant score (CS)[6] is one of the most commonly used shoulder scoring systems[7], and is considered the gold standard in Europe[8]. It is comprised of 4 parts - Pain reported by the patient (15 points); Activities of Daily Living reported by the patient (20 points); Range of Movement - assessed by the examiner (40 points) and strength - assessed by the examiner (25 points), with the better functioning shoulders having a greater number of points up to a maximum of 100 points. It has a long history and is easy to apply, hence it is used very extensively[7,9]. Because of its long term use since it was formally published in 1987, it is likely to remain popular so that treatment outcomes previously reported in the literature can be compared with more modern surgical or non-surgical treatments. Authors have criticised the CS, suggesting its level of standardisation is poor[10,11]. It does not evaluate shoulder instability and this is a weakness[12], as is the non-standardised strength evaluation[13] although Bankes et al[14] have described the fixed spring balance modification (fixed to a desk or to the floor) as a low-cost technique, which gave similar and equally reproducible values for power when compared with an electronic measuring device[15]. However, it does reliably detect improvements in shoulder function after intervention[16]. Some concerns have been expressed about the different results reported for males and females and the reduction of scores with increasing age[15] but these have been addressed by the use of the modified CS, which corrects for both age and gender[17].

Oxford shoulder score

The oxford shoulder score (OSS)[18] is a joint specific scoring system that is patient assessed, and scored out of 48. It is an outcome measure based on the patient’s perspective of their outcome and using standard questions where 4 of the 12 questions are related to pain. The OSS is a reliable, easy to use outcome measure which is sensitive to surgical outcome and clinical changes[19,20]. It has good correlation with clinical findings. Again the OSS does not assess shoulder instability[21] and, as a consequence the oxford instability score (OSIS) has been developed subsequently[22] (see below). Olley et al[20] concluded that the OSS can be used for audit purposes, however, they have suggested that larger prospective studies should be carried out to identify whether OSS varies with age, gender, and size of rotator cuff tear or not. Frich et al[23] studied the Danish version of the OSS, reporting that the OSS psychometric properties are valid and reliable. They also reported that the OSS is appropriate to evaluate individuals suffering from degenerative or post-traumatic shoulder diseases.

OSIS

The OSIS[22] is based on a 5-option response (Likert scale) for each item, with each response scored from 0 to 4, with 4 being the best outcome. All questions are laid out similarly with response categories denoting least (or no) symptoms being to the left of the page (scoring 4) and those representing greatest severity lying on the right hand side (scoring 0). The overall OSIS score is reached by simply summing the scores received for individual questions. This results in a continuous score ranging from 0 (most severe symptoms) to 48 (least symptoms).

Disability of the arm, shoulder and hand

The disability of the arm, shoulder and hand (DASH) score is a 30-item patient-reported tool to assess symptoms and physical disability in the arm. The DASH parameters are symptoms, physical, social and psychological functions. The score evaluates pain, physical disability and sleep disturbance. The pain score and physical disability assessment make a large part of this outcome measure. The DASH score has been shown to assess improvement after surgery [Lewis (2012)], and multiple authors have found it to be a valid and reliable score [Slobogean et al[24] (2010) Huisstede et al[25] (2009) Bilberg et al[26] (2012)]. As with the OSS, the DASH score has also been used cross-culturally and has provided similar results (Jianmongkol et al[27] 2012). However, as the name suggests, it is not a shoulder specific scoring system, and being a patient self-reported scoring system, DASH may fall victim to patient bias. In addition the DASH score results are inverted with the higher scores (maximum = 100) representing a greater disability and the lower scores occurring in a good functioning arm.

Simple shoulder test

The simple shoulder test (SST) was developed by Rick Matsen, initially in San Antonio and later in Seattle at the University of Washington Shoulder and Elbow Service. It has been reported as simple, valid, highly reliable and free practical patient self-assessment tool. The SST is a questionnaire with 12 questions designed for “yes/no” answers. It is validated for pre and post-operative shoulder function, and, is popular in North America[28]. However, it has also been validated in a number of other countries[29], including Brazil[30], Holland[16] and Italy[31] and is considered to be user friendly[16]. Drawbacks associated with the SST are perhaps its generosity (high scores when significant disability is present), and the different effects of age and type of injury or disease on the scores.

American shoulder and elbow surgeons standard shoulder assessment form

The American shoulder and elbow surgeons standard shoulder assessment form (ASES)[32] is easy to apply and consists of an assessment of the patients activities of daily living and a patient self-evaluation. It can be applied to all shoulder patients regardless of diagnosis. Some authors report it has good reliability, high constructive validity and high responsiveness[32]. However, Bafus et al[33] have reported that the ASES is not a valid and reliable scoring system for shoulder pathology as there are questions like “do usual sport” and “throw ball overhand” which are not easy for some patients to answer as they do neither. Although ASES is a highly accepted shoulder scoring system, it does contain several shortcomings in its construction.

Western ontario shoulder instability index

The Western ontario shoulder instability index (WOSI)[34] is a specific instability score designed to address the lack of validity of other scores in assessing shoulder instability symptoms. It is a self-assessment shoulder scoring tool that is disease-specific and also assesses the quality of life of patients with symptomatic shoulder instability. It is highly accepted by patients and surgeons because of the perceived importance of the items questioned, and has been found to be valid and reliable[35,36]. The disadvantages of the WOSI are that it has 21 questions each scored using 100 mm visual analogue scales, and its research usability is moderate as it is specific to instability conditions.

Japanese orthopaedic association shoulder score

The Japanese orthopaedic association (JOA) shoulder score is extensively used throughout Japan but it is not commonly reported outside that country. It is a much more complicated scoring system. For each of the 36 questions, patients are asked to self-interpret their symptoms using a scoring system divided into five levels (0 to 4) in which the larger values mean a better shoulder. The grades are: I have no difficulties (= 4); I have minor difficulties (= 3); I have some difficulties but I can manage on my own (= 2); I have major difficulties and require help from someone (= 1); and I cannot do it at all (= 0). These are very similar to the answers to the OSS questions, but 32 questions are used by the JOA while only 12 are used for the OSS. However the complex part is the final calculation which involves transferring scores from one domain to another, and as a result, it has not found popularity outside Japan.

Short form-36, short form-12, EQ-5D and short form-6D for general health

The short form (SF)-36 and the shorter SF-12 have become the most widely used measures of general health in clinical studies throughout the world. The SF-36 currently generates eight dimension scores and two summary scores for physical and mental health. Whilst such scores provide an excellent means for judging the effectiveness of health care interventions, they have only a limited application in economic evaluation because they are not based on preferences. The SF-6D[37] provides a means for using the SF-36 and SF-12 in economic evaluation by estimating a preference-based single index measure for health from these data using general population values. The SF-6D allows the analyst to obtain quality adjusted life years from the SF-36 for use in cost utility analysis. The EQ-5D is a similar, five-dimension, questionnaire and is now becoming popular in evaluating cost utility analysis and changes in general health after surgical operations in the United Kingdom and Europe.

REVIEW OF THE MOST COMMONLY USED SHOULDER OUTCOME SCORES IN EUROPE AND AMERICA

A review was carried out of all the articles in the 2012 and 2013 volumes of four major MEDLINE/PubMed referenced journals: The Journal of Bone and Joint Surgery (Am); The Journal of Bone and Joint Surgery (Br - now called the Bone and Joint Journal); The Journal of Shoulder and Elbow Surgery; and the American Journal of Sports Medicine. That review is summarised in Figures 1 and 2 and Table 1. The CS and the ASES are those most frequently used. This may be because of their long history, and the tradition of combining both objective and subjective assessments of patients. Both have also been extensively validated. However, there is evidence that patient self-reported scoring is not only easier and cheaper, but may also be a more accurate method of assessing orthopaedic outcomes[38-40]. Other popular scoring systems include the DASH, the SST and the university of California-Los Angeles shoulder scale (UCLA). The remainder are used far less commonly, sometimes for specialised situations (i.e., the Kerlan-Jobe Orthopaedic Clinic - KJOC overhead athlete score). Different shoulder scoring systems appear to be used in different countries. In Europe the CS, DASH and the OSS appear to be the most common, while in North America, the ASES, SST, and UCLA are used to assess the shoulder most often. All of these scoring systems either test different things or features in different ways to assess shoulder function. However, one would presume some kind of inter-correlation might be found between the scoring systems used in Europe and those being performed in United States as all of them intend to evaluate the shoulder function.

Figure 1.

Number of articles that used different shoulder scoring systems during 2012 and 2013.

Figure 2.

Most popular shoulder scoring systems during 2012 and 2013. ASES: American shoulder and elbow surgeons standard shoulder assessment form; DASH: Disabilities of the arm, shoulder and hand; UCLA: University of California-Los Angeles shoulder scale.

Table 1.

Shoulder scoring systems used in 2012 and 2013

Shoulder scoring system	No. of times used	No. of times used	No. of times used in
	in articles in 2012	in articles in 2013	articles in 2012 and 2013
CS	47	44	91
American shoulder and elbow surgeons evaluation form	41	32	73
The disability of the arm, shoulder and hand score	22	13	35
SST	16	13	29
University of California/Los Angeles shoulder score	15	13	28
Western ontario osteoarthritis score	6	0	6
The OSS	4	6	10
SF-12 general health	3	7	10
Western ontario rotator cuff score	3	5	8
SF-36 general health	1	8	9
Penn shoulder score	1	2	3
Shoulder pain and disability index	1	4	5
Western ontario shoulder instability score	1	2	3
Rowe shoulder instability score	1	2	3
Single assessment numeric evaluation score	1	7	8
Kerlan-Jobe orthopaedic clinic overhead athlete score	1	2	3
Hospital for special surgery shoulder score	0	0	0
OSIS	0	2	2

CS: Constant score; SST: Simple shoulder test; OSS: Oxford shoulder score; SF-12: Short form-12; OSIS: Oxford instability score.

EQUIVALENCE OF SHOULDER SCORING SYSTEMS

A number of papers have evaluated whether different scoring systems are comparable. The CS is a combination of both a clinically-based and patient assessment outcome measure for patients with shoulder problems, while most other scores focus on more patient-based outcome measures. One would therefore suspect that the CS cannot be interchanged or merged with the other scoring systems which are based on self-assessment by the patients. In contrast, OSS, DASH, SST and WOSI scoring systems can be compared and interchanged to some extent. However there are confounding factors which will influence these scores. All the patient-based assessment outcome measures may be affected by personal or patient response biases. Scott et al[41] have compared the CS and OSS, and found that using the OSS gave a higher proportionate overall score for the shoulder patient than the CS and that the agreement between the scores was poor. However, other studies have revealed that OSS is more stable than CS in terms of outcome measures[18,42]. Skutek et al[43] have also shown only moderate correlation between the CS and SST. Hirschmann et al[13] suggested that CS is poor at measuring shoulder strength and should be re-evaluated for its sensitivity and reliability. Lewis[44] reviewed patients with subacromial impingement, and measured CS, DASH and the EuroQol (EQ-5D) quality of life measurement. He found that CS and DASH were both significantly improved in the intervention exercise group, but CS improved more than DASH, suggesting poor agreement between the two systems.

As the OSS and DASH are patient self-assessment measurement outcomes they could be expected to be interrelated to each other but may sometimes be used combined. Studies have reported that OSS is better to assess the surgical outcome in patients with rheumatoid arthritis as compared to the DASH and SF-36[45].

In order to check the reliability and validity of DASH and SF-6D, Slobogean et al[24] studied patients with treated proximal humeral fractures. They reported that DASH and SF-6D questionnaires were adequate in assessing psychometric properties, and suggested that these scoring systems are appropriate to assess the outcome in patients treated for proximal humeral fractures. Interestingly, van de Walter et al[46] reviewed five scoring systems-CS, -OSS, -DASH, -UCLA and a Subjective Shoulder Value following proximal humeral fractures and concluded that currently available shoulder scoring systems may not offer significant value for assessing functional improvement after such fractures.

Godfrey et al[29] conducted a study on 1077 patients suffering from shoulder instability to assess the reliability, validity, and responsiveness of the SST. They found that the SST is significantly correlated with the ASES score and both responded to change in shoulder function. However, they reported that the results were different for different age groups and different types of shoulder injury. The study conducted by van Kampen et al[16] also revealed that the SST is highly correlated with the DASH, OSS and SF-36 in terms of physical subscales; however, it did not correlate well with the CS.

WHEN TO USE SCORING SYSTEMS

Many surgical and non-surgical shoulder interventions take months to gain full benefit after shoulder treatment, so any follow up has to be measured over months and years to evaluate longer term outcomes. Our preference is to score all our patients preoperatively, then at 6 mo and 1 year if the patient continues to be followed up. For shoulder joint replacement patient’s scores are then repeated at 3, 5 and 10 years unless the patients are scored more regularly as part of a research protocol.

USING THE CONSTANT SCORE TO EVALUATE THE OUTCOME FOLLOWING SHOULDER ARTHROPLASTY

It has been standard practice in our unit for 20 years to use the CS to evaluate the outcome from shoulder operations and, in particular shoulder arthroplasty. Previous studies, reported in 2007[15] have highlighted the changes in the CS in normal people with increasing age and between the sexes. Table 2 shows the estimated normal CSs for age and sex.

Table 2.

Estimated normal constant scores for age and sex[15]

Age (yr)	Men	Women
50-59	95 ± 2	88 ± 2
60-69	92 ± 2	85 ± 2
70-79	89 ± 2	82 ± 2
80-89	86 ± 2	79 ± 2

As a practical example from our unit, in 2013 we evaluated the outcomes from all our primary anatomic and inverse (or reverse) shoulder replacements using a new Vaios shoulder replacement design[47] and the results are shown in Table 3 for anatomic shoulder arthroplasty and Table 4 for reverse shoulder arthroplasty. However these means and standard deviations are difficult to understand for most orthopaedic surgeons but they become more meaningful if they are converted into different grades of improvement.

Table 3.

Constant and oxford shoulder scores for the vaios anatomic shoulder replacements

Outcome measure	No. of	Oxford score	Pain	ADL	ROM	Strength	Total CS	Constant score adjusted
	shoulders	(/48)	(/15)	(/20)	(/40)	(/25)	(/100)	for age and sex (%)
Pre-op mean (SD)	49	16.8 (7.8)	4.8 (3.2)	6.7 (3.7)	10.1 (5.1)	1.0 (2.2)	21.9 (9.7)	29.2 (12.4)
Post-op 1 yr mean (SD)	49	33.0 (12.8)	11.3 (4.1)	13.3 (4.7)	20.0 (11.3)	4.0 (4.8)	44.6 (17.6)	59.6 (24.0)
Post-op 2 yr Mean (SD)	30	36.5 (12.3)	12.0 (3.9)	14.0 (6.0)	20.5 (10.7)	6.7 (6.0)	47.2 (19.4)	62.1 (22.8)
Post-op 3 yr mean (SD)	8	38.4 (10.6)	10.8 (2.9)	11.8 (4.0)	16.4 (6.2)	6.4 (3.6)	45.4 (10.4)	63.6 (21.0)

ADL: Activities of daily living; CS: Constant score.

Table 4.

Constant and oxford shoulder scores for the vaios inverse shoulder replacements

Outcome measure	No. of shoulders	Oxford score	Pain	ADL	ROM	Strength	Total CS	Constant score adjusted
		(/48)	(/15)	(/20)	(/40)	(/25)	(/100)	for age and sex (%)
Pre-op mean (SD)	63	18.9 (9.0)	6.6 (4.0)	8.0 (4.2)	11.2 (8.1)	0.8 (2.1)	25.9 (13.1)	37.0 (18.2)
Post-op 1 yr mean (SD)	63	35.4 (11.7)	12.3 (3.7)	13.5 (5.4)	22.8 (10.5)	4.7 (5.3)	49.4 (18.1)	69.0 (25.5)
Post-op 2 yr mean (SD)	28	34.4 (13.8)	12.4 (3.7)	13.7 (5.5)	23.9 (11.5)	5.6 (4.3)	49.8 (19.0)	69.7 (27.7)
Post-op 3 yr mean (SD)	6	33.0 (10.6)	11.8 (4.7)	16.3 (3.9)	24.0 (9.8)	7.8 (0.5)	59.8 (17.0)	81.6 (23.5)

ADL: Activities of daily living; CS: Constant score.

We have therefore developed a meaningful grading system for assessing the outcomes following arthroplasty using either the CS as shown in Table 5 or the age and sex adjusted CS as shown in Table 6. These results are more meaningful than means and standard deviations and allow us to appreciate that 60% of patients achieve a good, very good or excellent result after anatomic Total Shoulder Replacement while 76% achieve a good, very good or excellent result after Inverse or Reverse shoulder replacement using the age and sex adjusted CS.

Table 5.

Categories of outcome after shoulder arthroplasty using the constant score

Category	Total CS (/100)	Age and sex adjusted CS (%)
Average shoulder function before arthroplasty	< 30	< 40
Post-op unsatisfactory	< 30	< 40
Post-op fair outcome	30-39	40-49
Post-op good outcome	40-59	50-69
Post-op very good outcome	60-69	70-79
Post-op excellent outcome (i.e., a virtually normal shoulder)	≥ 70	≥ 80

CS: Constant score.

Table 6.

Stratified outcome for the vaios primary total shoulder replacements using the adjusted constant score (adjusted for age and sex)

Post-op outcome	Adjusted CS (%)	No. of anatomic TSRs n = 46	% anatomic TSRs using adjusted CS	No. of inverse TSRs n = 58	% inverse TSRs using adjusted CS
Unsatisfactory outcome	< 40	10	22	9	16
Fair outcome	40-49	9	20	5	9
Good outcome	50-69	9	20	14	24
Very good outcome	70-79	3	7	5	9
Excellent outcome	≥ 80	15	33	25	43
Totals		46	100	58	100

CS: Constant score.

CONCLUSION

It is difficult to choose the best scoring system as a “best choice of outcome measurement tool” for patients with shoulder problems. There remains a need to develop a comprehensive outcome measurement tool that can adequately deal with both the clinician’s and the patient’s perspective. None of the scoring systems ideally fulfil this criterion. For instance, the clinically-based outcome measuring tool may not satisfy the patient, and the patient self-assessment tools may not provide the information that the clinician believes is important. Patient-based assessment tools may fall a victim to bias as the patient may under or over report symptoms: under-reporting if they wish to please the surgeon or over-reporting for secondary (or compensation related) gain. Similarly, physician-based tools may not truly reflect what the patient feels about their outcome. Therefore it makes sense to combine scoring systems when collecting data for outcome measures, and we would recommend using multiple scores, including clinically - based and patient self-assessment tools. We currently use the CS and OSS in our unit for the majority of our patients but the SST has also been proven to be a valuable outcome assessment in North America. These assessment do need to be carried out at the appropriate time and our policy in Nottingham is to always carry out a pre-operative assessment and then to carry out the first post-operative evaluation at 6 mo after surgery when the patient’s condition is reaching a plateau. The United Kingdom health service has become much more focused on using PROMs and many are moving towards using the OSS as their preferred PROM for general shoulder assessment.