The minimal clinically important difference of the Michigan Hand Outcomes Questionnaire

Abstract

Purpose

To determine the change in score required in various domains of the Michigan Hand Outcomes Questionnaire (MHQ) to indicate meaningful patient improvement, or the minimal clinically important difference (MCID), for three common hand conditions: rheumatoid arthritis (RA), carpal tunnel syndrome (CTS) and distal radius fracture (DRF).

Methods

The MHQ was administered to patients at two time-points. Patient satisfaction was defined as a satisfaction score equal to or greater than 80% of the standard deviation of that patient sample. The minimal change in score in specific MHQ domains that corresponded with patient satisfaction was determined using receiver operating characteristic (ROC) curves.

Results

For CTS patients, MCIDs of 23, 13 and 8 were identified for the Pain, Function and Work domains, respectively. For RA patients, Pain and Function were also identified as having discriminative ability, with MCIDs of 11 and 13, respectively. An MCID of 3 was identified for the Activities of Daily Living domain. For DRF patients, no MHQ domains showed discriminative ability due to the ceiling effect at the 3 month assessment period.

Conclusions

Individual domains of the MHQ can be used to discriminate between patients who are satisfied and those who are not after either carpal tunnel release or silicone arthroplasty of the metacarpophalangeal joints for RA. Pain and Function are the domains of the MHQ that are best able to discriminate between patients who are satisfied and those who are not. The identical Function MCID for both RA patients and CTS patients, despite markedly different pre-operative values, indicates that a standard amount of functional change may indicate patient satisfaction. High post-operative satisfaction, even only 3 months after surgery, prevented any domains from showing discriminative ability for the DRF patients.

Introduction

A common concern amongst survey researchers is the amount of change necessary in questionnaire score to indicate a clinically important difference. For objective measures, like range of motion, improvement is easy to quantify. But if a patient’s MHQ Pain score changes by 10 points, is this difference clinically important? To get at this, the effect size and Standardized Response Mean (SRM) of outcomes questionnaires are generally reported as part of studies’ results. But, these measures rely on the amount of change based on the standard deviation and are not applicable to all conditions. For some patients, those with severely debilitated hands due to scleroderma, for instance, a small improvement in the ability to perform everyday activities may be clinically important. The effect size may be small, but the change in function may be clinically important to the patient, nonetheless. On the other hand, even a large change may not be satisfactory to a patient after carpal tunnel surgery if he or she expects total elimination of numbness and pain in the hand. The growing interest in measuring patient-rated satisfaction means that the minimal clinically important difference (MCID) of questionnaires like the MHQ needs to be defined for a number of hand surgery conditions. This will allow physicians to understand how much change will be deemed important to patients afflicted with these hand and upper extremity conditions. The MCID compares the change found on an outcomes instrument with an agreed upon “gold standard” of change, often a diagnostic test, but now commonly patient-rated measures.¹ The simplest way to conceptualize the MCID is as the point at which the average patient stops “judging themselves as ‘about the same’ and starts judging themselves as ‘a little bit better’.”²

Because patient satisfaction has increasingly been emphasized as a gauge of treatment success,³ we will use the Satisfaction domain in the MHQ to assess change in scores in other domains of the MHQ including Function, Activities of Daily Living (ADLs), Work, Pain and Aesthetics. The specific aim of this project is to define the MCID of each domain of the MHQ by using three prospective cohorts of patients with common conditions (rheumatoid arthritis, carpal tunnel syndrome and distal radius fracture) to which the MHQ has been demonstrated to have great utility in defining outcomes of surgical treatment.^4–6

Methods

Three prospective cohort studies with ongoing data collection were used to measure the MCID for the MHQ. The conditions of interest are: rheumatoid arthritis (RA) of the hand treated with silicone metacarpophalangeal joint arthroplasty (SMPA), carpal tunnel syndrome (CTS) treated with limited-incision technique carpal tunnel release and treatment of distal radius fracture (DRF) using the Volar Locking Plating System (VLPS).^{4, 7, 8} Data obtained from these prospective cohorts studies have not previously been used to measure the MCID for these respective conditions.

Outcomes assessment

The Michigan Hand Outcomes Questionnaire (MHQ) was first published in 1998.^{9, 10} It is a hand-specific outcomes instrument comprised of 37 questions over six domains: (1) overall hand function, (2) activities of daily living (ADL), (3) pain, (4) work performance, (5) aesthetics and (6) patient satisfaction.

Assessment methods

Patients undergoing Silicone metacarpophalangeal joint arthroplasty (SMPA) for rheumatoid arthritis (RA) or limited-incision technique carpal tunnel release for carpal tunnel syndrome (CTS) were approached in the University of Michigan Plastic Surgery Clinic by a research associate during a pre-operative visit and invited to participate in the study. Patients returning to the clinic for 3-month visits following internal fixation of a distal radius fracture (DRF) using the Volar Locking Plating System (VLPS) were similarly approached. The various aspects of each study were explained and informed consent was obtained. All three studies were approved by the University of Michigan Medical School Institutional Review Board. The broad inclusion criteria were age of 18 years or over and the ability to read and write in English; we excluded patients with serious mental illness or dementia. The inclusion and exclusion criteria specific to each diagnostic group have been detailed elsewhere.^{4, 5, 11} The MHQ was administered to patients in the CTS and RA groups before and after surgery. For this investigation, we used MHQ scores obtained 12 months after surgery for patients in the RA study. The second MHQ was administered to patients in the CTS study an average of 9 months after surgery. Patients in the DRF group completed the MHQ 3 and 12 months after fracture fixation.

Identification of the MCID

The first step toward the identification of the MCID is the designation of a “gold standard” to which all other values will be compared. We have chosen to use the Satisfaction domain of the MHQ as the standard of patient improvement. Change scores quantifying the amount of improvement for each domain were calculated for each participant by subtracting the score of the second administration of the MHQ from the score from the first (The MHQ is scored from 0–100, with 100 being the “best” score). The construction of Receiver Operating Characteristics (ROC) curves requires that the “gold standard” be a dichotomous variable. Because Satisfaction, like all MHQ domains, is scored as a continuous variable, we needed to convert it to a dichotomous variable. Using Satisfaction change scores, we determined a cut-off point that would divide satisfied patients from those who were not. This is a common practice when determining the MCID for questionnaires.^{1, 12, 13} Deciding what value along this range represented the break between patients who were not satisfied and those who were is the most challenging aspect of determining the MCID. We used Cohen’s effect size definition to guide us in this process – selecting a cut-off point of 0.8 of the standard deviation of each studies’ change scores, corresponding to a large effect size.¹⁴ Patients with Satisfaction change scores at or above this point would be considered “satisfied” whereas those with change scores below this point would be considered “not satisfied.” We could then convert Satisfaction into a dichotomous variable.

With Satisfaction as the “gold standard,” we constructed ROC curves for each domain in each study. ROC curves compare a dichotomous variable with a continuous variable to determine the ability of the continuous variable to discriminate between positive and negative cases of the dichotomous variable. They are constructed by plotting sensitivity (true positive) vs. 1-specificity (false negative).^{15, 16} For our purposes, the ROC curve illustrates how accurately each domain can discriminate between patients who are satisfied and those who are not.¹⁵ Discriminative ability is determined by measuring the area under the ROC curve. The area under the curve (AUC) can range from 0.5 (no discriminative ability) to 1.0 (perfect ability to discriminate between satisfied and unsatisfied patients).¹⁵ The MCID is the point closest to the upper left corner of the curve, where sensitivity is closest to 1.0 and 1-specificity is closest to 0.^{1, 12, 17} An AUC > 0.75 is considered to provide clinically useful discriminative ability.¹⁸ Data were analyzed using SAS 9.1 (SAS Institute, Cary, NC) and SPSS (SPSS Inc, Chicago, IL) statistical analysis software.

Results

Forty RA patients, 53 CTS patients and 51 DRF patients completed the MHQ at two time points. A summary of patient demographic data is shown Table 1.

Table 1.

Patient Demographics

Study	Rheumatoid Arthritis	Carpal Tunnel Syndrome	Distal Radius Fracture
Total patient count	40	53	51
Gender
Male (%)	7 (18%)	12 (23%)	19 (37%)
Female (%)	33 (83%)	41 (77%)	32 (63%)
Mean age (range)	61 (44–73)	54 (27–90)	50 (19–83)
Laterally of procedure
Right (%)	24 (60%)	14 (26%)	22 (43%)
Left (%)	16 (40%)	8 (15%)	29 (57%)
Bilateral (%)	--	31 (58%)	--

Swanson metacarpophalangeal joint arthroplasty for rheumatoid arthritis

For RA patients, mean Satisfaction score showed a significant increase from 28 points (out of 100) before surgery to 61 points one year after surgery (p<0.0001). (Table 2) The mean change was 33 points, with a standard deviation of 24 points. With 0.8 of the standard deviation serving as a cut-off point for patient satisfaction, 30 patients were satisfied and 10 patients were not satisfied.

Table 2.

Michigan Hand Outcomes Questionnaire Satisfaction domain scores

Study	Mean score at first observation (range)	Mean score at last observation (range)	Mean change (range)
Carpal Tunnel Syndrome	31 (0 – 92)	67 (0 – 100)	36 (−44 – 96)
Distal Radius Fracture	78 (25 – 100)	89 (42 – 100)	10 (−25 – 50)
Rheumatoid Arthritis	28 (0 – 92)	61 (0 – 100)	33 (−8 – 79)

Three MHQ domains showed discriminative ability related to patient satisfaction: Function, Activities of Daily Living (ADLs) and Pain. An MCID of 13 points was identified for the Function domain, with an AUC of 0.89. For ADLs, an MCID of 11 points was identified, with an associated AUC of 0.77. Finally, for Pain we identified an MCID of 3 points, with an AUC of 0.80. (Figure 1, Table 3)

Figure 1.

ROC curve - Swanson metacarpophalangeal arthroplasty for rheumatoid arthritis

Arrows indicate the MCIDs, the points closest to the upper left quadrant of the graph, where sensitivity is closest to 1 and 1-specificity is closest to 0.

Table 3.

Michigan Hand Outcomes Questionnaire domains showing discriminative ability

Study	Domain	AUC	Sensitivity	1-Specificity	MCID
Carpal Tunnel Syndrome	Work	0.87	0.74	0.27	8
	Pain	0.86	0.81	0.22	23
	Function	0.82	0.77	0.27	13
Rheumatoid Arthritis	Function	0.89	0.90	0.22	13
	Pain	0.80	0.69	0.30	3
	ADLs	0.77	0.73	0.20	11

Limited-incision technique carpal tunnel release

Mean satisfaction scores for CTS patients improved significantly from 31 points to 67 points (p<0.0001). The mean increase was 36 points, with a standard deviation of 33 points. Using 0.8 of the standard deviation as the cut-off point for patient satisfaction, we are able to establish that 31 patients were satisfied with the procedure and 22 patients were not satisfied.

Three MHQ domains showed discriminative ability related to patient satisfaction: Function, Work and Pain. An MCID of 13 points was identified for the MHQ Function domain with an AUC of 0.82. For the Work domain, an MCID of 8 points was identified, with an AUC of 0.87. An AUC of 0.86 was found for the Pain domain, indicating an MCID of 23 points. (Figure 2, Table 3)

Figure 2.

ROC curve – Limited-incision technique carpal tunnel release

Arrows indicate the MCIDs, the points closest to the upper left quadrant of the graph, where sensitivity is closest to 1 and 1-specificity is closest to 0.

Internal fixation of distal radius fracture using the Volar Locking Plating System

Mean satisfaction scores also improved significantly for DRF patients, from 78 points to 89 points (p<0.0001), with a mean increase of 11 points. (Table 2) This increase is noticeably lower than that shown by CTS and RA patients. This is most likely due to the high first observation value shown by DRF patients. Eight of the 51 patients reported that they were completely satisfied (Satisfaction score of 100 points) at the first observation. Seven of these patients remained completely satisfied on the last observation, making their Satisfaction change score 0 points, contributing to the low mean change score. The standard deviation of the change in satisfaction was 15 points. We used 0.8 of this figure to separate satisfied patients from unsatisfied patients. We also included the 7 patients who reported complete satisfaction at both time-points in the group of satisfied patients. Overall, 38 patients were satisfied after treatment and 13 patients were not satisfied.

No domains of the MHQ demonstrated discriminative ability in regards to patient satisfaction.

Discussion

This publication demonstrates the process of identifying the MCID of a patient-rated outcomes questionnaire. Our study indicates that a different strategy is necessary when calculating the MCID for various patient populations. For instance, DRF fixation with VLPS allows patients to reengage in everyday activities much sooner than previous methods of DRF fixation.⁸ Because of this, patient-reported satisfaction was relatively high only 3 months after fixation. In fact, scores on all domains were higher than 75, demonstrating a self-reported lack of impairment in any outcomes domain. These high initial scores created a “ceiling effect” that prevented the 12-month post-surgical scores from showing high improvement. The result of this was that no MHQ domains were identified as having discriminative ability.

Pain and Function were found to be the domains of the MHQ that are best able to discriminate satisfied patients from unsatisfied patients. Both of these domains showed this property in the RA and CTS patient samples. For CTS, the MCID for Pain was 23 points. Very different results were found for Pain in RA patients; the MCID was quite small, only 3 points. One can speculate that the large MCID indicates satisfied CTS patients experience a much greater level of pain relief than do CTS patients who are not satisfied, whereas for RA patients, only a small amount of pain relief separates satisfied RA patients from those who are not satisfied. This may be because RA patients suffer from chronic pain; therefore, a small change in the pain scores is sufficient to discriminate between patients who are satisfied and those who are not. But CTS patients experience wax and waning acute pain, and they expect a much greater improvement in their pain symptoms after surgery. These findings for CTS corroborate a recent study by Özyürekoğlu et al. who identified an MCID of 1.04 points on the Symptom Severity Scale (SSS) of the CTS disease-specific questionnaire.¹ The main components of the SSS portion of the CTS are the Pain and the Sensory domain. The CTS is scored on a scale of 0–5.0, making a 1.04 point MCID analogous to our MHQ Pain domain MCID of 23 points. The MCID for Function was 13 points for both CTS and RA patient groups, which established a consistent MCID for these two conditions.

In addition to Pain and Function, one additional domain showed discriminative ability in each patient population. For RA patients this domain was ADLs, with an MCID of 11 points. RA patients report low ability to perform ADLs before SMPA; the mean pre-operative score was only 38 points. An increase of 11 points, therefore, is an increase of nearly 30%. This large change in the ADL scores suggests that a large effect is necessary before a patient is satisfied with the everyday activity performance. In patients who have undergone carpal tunnel release, the Work domain showed discriminative ability. The MCID was a rather low 8 points. Carpal tunnel syndrome often affects patients’ ability to work (mean pre-operative score: 50 points). A low-value MCID, such as this, may indicate that satisfied patients may be content with even a small increase in work ability.

One of the obvious limitations of calculating the MCID with this method is the difficulty when evaluating traumatic conditions, like DRF. Because DRF patients go from a completely immobilized state to near full recovery of their previous state of presumably little to no impairment in a relatively short period of time, the 3-month post-surgical scores we obtained were already quite high. This created a “ceiling effect” in which the high baseline score prevents substantial improvement, resulting in low Satisfaction change scores. Scores in each of the other measured domains were equally high at the initial 3 month assessment period. This lack of change provided little basis for the calculation of the MCID. Another limitation is the lack of firm standard for calculating patient satisfaction. We chose to calculate a satisfaction change score from the score on the Satisfaction domain of the MHQ at two time-points. This method is commonly used^{1, 12, 13} and given its ability to account for baseline differences¹⁹ we felt it was the best method available. However, the calculation of a change score necessitates the assignment of a cut-off point for satisfaction. The sheer number of options for this point can make this method potentially unreliable. Other methods include asking patient if they are satisfied. While these methods require no calculation to separate satisfied and dissatisfied patients, the criterion for satisfaction can vary from patient-to-patient, the score relies on patients’ memory of past health-states and there is no way to standardize the responses.^{17, 20}

The presentation of effect size as an indication of patient improvement may no longer be sufficient. Viewing the concept of patient satisfaction in the context of MCID can provide a better picture of the patient experience. This can provide physicians with information to guide patients’ expectations and to help with patient decision-making. Furthermore, MCID can be used to evaluate diverse patient groups, as we have, comparing three very different procedures and patient populations. Finally, this information provides the opportunity to compare upper extremity questionnaires to determine the amount of change needed to produce similarly satisfied patients. Future studies can be undertaken to establish the MHQ MCID for other hand conditions, such as arthroplasty procedures.