Clinimetrics corner: the Global Rating of Change Score (GRoC) poorly correlates with functional measures and is not temporally stable

Abstract

The Global Rating of Change Score (GRoC) is a frequently used outcome measure that is used independently to measure improvements in a patient’s condition or as an anchor for other outcomes measures. The tool has been criticized for recall bias, biases in administration, and for poor reliability over time. Our findings, captured from a sample of patients with shoulder impingement, suggest that all these concerns are of merit. Our results show poor correlation of the GRoC with functional measures after 2 and 3 weeks and decay of the associative stability of the GRoC from week to week.

Background

The Global Rating of Change Score (GRoC) is a frequently used outcome measure that has been used to independently score self-perceived improvement in a patient and has been used as an anchor method to determine minimal clinically important change scores. The GRoC is a single-item, recall-based questionnaire of well-being that is based on progress (or lack of progress) since an initial treatment encounter.¹ Patients are routinely asked to make global ratings on changes in regards to their level of shoulder well-being since the previous week’s treatment on a 15-point self-report scale (from −7 to 7), although other scale values have been used. The GRoC has been the outcome of choice for patients with shoulder pain in previous studies.^2,3 The tool is easy to administer, easy to use, and is considered a bastion for outcomes measures.

The GRoC is a retrospective judgment tool and there are weaknesses that have been identified with these forms of outcomes measures.^4,5 First, recall bias may limit the reliability of the tool over time.⁵ Second, retrospective tools are prone to bias, specifically those associated with social responsibility or mode of administrative influences that are consciously or subconsciously supplied by the clinician.^6–8 Last, the tools allow the patient to score their perceived recovery using any construct they choose,⁹ which is a strength and a weakness. Thus, elements such as pain or factors that are not considered important by the treating clinician may be the factors selected by the patient when determining recovery.

Recently, within a clinical context, we have noticed the lack of temporal stability of individual GRoC measures for patients who were enrolled in our shoulder study. Patient GRoC findings fluctuated widely from week to week, and in many cases, the actual values presented during the weeks before the final week (which was taken as the final GRoC score for the analysis) differed dramatically from the later values. In addition, clinically, there did not seem to be a longitudinal correlation between GRoC scores and independent measures of function, measures that have historically served to determine who truly has benefited from a specific intervention. This finding prompted the investigation of the temporal stability of the GRoC and whether the weekly GRoC values were related to weekly American Shoulder and Elbow Surgeon’s Scale (ASES) change scores and whether or not individuals indicated the ability to achieve the three Patient Specific Functional Scale measures they identified as challenges before the initiation of rehabilitation. We hypothesized that the GRoC individual weekly measures did not correlate with the functional measures and that a weekly finding of a GRoC does not have consistent associative stability with functional measures over time.

Methods

Data for this secondary analysis were obtained from a prospective, longitudinal trial involving a standardized treatment program. Details of the study are outlined elsewhere.¹⁰ The original study was designed to identify associative variables for prognosis of patients who received physical therapy for shoulder impingement syndrome. All patients in the study exhibited some or all of the following criteria: (1) manifestation by gradual onset and characterization of a dull ache at the anteroloateral aspect of the shoulder often with radiation to the deltoid insertion; (2) patient complaints with overhead activity; and (3) the condition is worsened with resisted abduction, resisted external rotation, overhead positioning of the arm, or direct pressure against the shoulder such as lying on it. All subjects were diagnosed with impingement syndrome by an attending physician and the diagnosis was confirmed by the treating physical therapist.

Outcomes data germane to this study in addition to the GRoC included the ASES shoulder questionnaire¹¹ and the Patient Specific Functional Scale (PFPS).¹² The ASES was developed by the Research Committee of the American Shoulder and Elbow Surgeons and was adopted by the membership as a standardized form for measurement of musculoskeletal function.¹¹ The questionnaire, which includes two theoretical subscales, pain and function/disability,¹¹ has been shown to possess reliability,^13,14 construct validity with the SF-36, ^14,15 and responsiveness when moderate to large effect sizes are present during treatment.¹⁴ In addition, a minimally clinically important difference has been demonstrated as a change of 6.4 ASES points.^14,15 The PFPS¹² uses three questions that are designed to assess difficulties in functional activities that are specific to that person. The original PFPS allows users to rate the level of difficulty of each activity from 0 (unable to perform) to 10 (able to perform at pre-injury level). The same three activities are summed to provide a total score of 0–30. We modified the PFPS within our study by asking individuals to pick an activity they could not do and then at each week asking whether or not they were currently able to do the task. All outcomes including the GRoC were collected weekly by a third party, non-clinician.

Statistical analysis

We created regression coefficients for each time period by analyzing the association of biweekly measures for the GRoC, with the assumption that the two are related to one another and that progression (improvement) is a linear phenomenon. Biweekly regression coefficients were plotted over time and the coefficient and 95% confidence interval measures were reported.

Median values and low and high range values for each GRoC weekly score were calculated as well as mean and confidence interval measures. In addition, we plotted the correlations between weekly GRoC measures and independent weekly ASES and PFPS measures (all weekly), over time.

Results

Figure 1 outlines the temporal stability of the GRoC after assessment of the influence of the prior week’s GRoC scores with the subsequent week. The seven timeframes (8 different weeks) show an inconsistent pattern of stability and that the prior week’s score do not always drive the follow-up week. In addition, the very wide confidence intervals suggest a high degree of variability in the findings as well. The weekly GRoC median values ranged from a low of 0 to a high of 3. Mean values were also dispersed significantly from week to week with very wide confidence intervals (Table 1).

Figure 1.

Linear association of weekly GRoC measures (stability) over time.

Table 1. Median and mean weekly GRoC measures.

	Median (range)	Mean (95% confidence interval)
Week 1	2 (−4–7)	2.4 (−1.83–2.96)
Week 2	2 (−4–7)	2.3 (−1.57–3.02)
Week 3	3 (−4–7)	2.4 (−1.63–3.16)
Week 4	3 (−4–7)	1.9 (−0.90–2.89)
Week 5	0 (0–6)	1.5 (−0.40–2.59)
Week 6	2 (−3–6)	2.3 (−0.57–4.02)
Week 7	2 (−1–6)	2.6 (−0.93–4.2)
Week 8	0 (−2–0)	0.7 (−0.61–2.01)

Correlational findings with the functional measures of ASES and PFPS yielded significant correlations in the first 2 weeks and only two instances of association after that point (Table 2). The correlations were statistically significant but relatively low.

Table 2. GRoC correlations with weekly ASES and weekly PFPS measures.

	Comparison of weekly GRoC to weekly ASES change score	Comparison of weekly GRoC to obtained of defined function 1	Comparison of weekly GRoC to obtained of defined function 2	Comparison of weekly GRoC to obtained of defined function 3
Week 1	0.31**	0.26*	0.35**	0.34**
Week 2	0.29*	0.38**	−0.43**	−0.24*
Week 3	0.27*	0.15	−0.00	−0.20
Week 4	0.23	0.15	−0.19	−0.25
Week 5	0.13	0.36	−0.17	−0.34
Week 6	0.31	0.40	−0.79**	−0.26
Week 7	−0.05	−0.20	−0.34	−0.64
Week 8	0.9	−0.50	0.50	1.0**

Note: *Significance at 0.05.

**Significance at 0.01.

Discussion

Our findings support our hypothesis that the GRoC is not temporally stable in that a particular finding 1 week ago is not associative to functional results the following week. This suggests that patient’s report of global change in their condition may fluctuate widely from one week to another and that GRoC outcomes are not linear and progressive, but instead fluctuate widely over multi-week period. This wide degree of variability could affect the overall outcome at the end point of a trial and could influence a minimally clinical important change score if used as an anchor point in a study. This deserves further investigation to whether this phenomenon is unique to our study or if it is consistent, yet unexplored in others.

Our most notable finding is that the GRoC, as a whole, does not correlate with the ASES or PFPS measures after 2 weeks. This is in contrast to others who have found a notable correlation, albeit at 4 weeks only.¹⁶ We found a small correlation at 1 and 2 weeks for all three measures and for week 3 with the ASES, in addition to two rather specious findings of correlation after that point. This suggests two possible considerations. First, the GRoC may measure a construct outside of domain of function. Since it is a multidimensional questionnaire, one could argue that the construct is specific to each patient and that this finding has limited impact. However, what may be more compelling possibility is that the correlation with function declines significantly after week 2 for the PFPS and after week 3 for the ASES. This suggests that concerns associated with recall decline or reliability of the GRoC may be warranted. As a reminder, the ASES and PFPS are both measures that are designed to capture findings at the present point of time, whereas the GRoC requests that individuals remember back before their therapy was initiated.

We witnessed a decline in median GroC values over time, specifically after the third and fourth weeks of care. This may mean that the chances for identifying a self-perceived improvement reduces over time and that studies that examine GRoC-related outlines are less inclined to see positive outcomes with long term versus short term. Since a number of studies have used the GRoC for short-term analyses only, it is unlikely that this phenomenon has been observed during care in these studies. If our study limited data collection to less than 4 weeks, we too would have not identified this finding.

Conclusions and Key Points

Further work is needed to determine the true value of the GRoC as an outcome measure and in turn as an anchor measure. Several key points were identified in this study:

1. There is fluctuant temporal stability of the GRoC from week to week.

2. There is poor correlation between the GRoC and functional measures.

3. The GRoC is only correlated to functional measures up to 3 weeks.