CORR Insights®: Test-retest Reliability and Construct Validity of the Satisfaction with Treatment Result Questionnaire in Patients with Hand and Wrist Conditions: A Prospective Study

Where Are We Now?

In healthcare, value-based medicine relies on patient-reported outcome measures (PROMs), which generally include a selection of measured domains such as activity and participation, pain, esthetics, and satisfaction. Of those, investigators and clinicians alike have had a love-hate relationship with one particular domain: satisfaction.

Satisfaction is hard to define and difficult to measure [3]. While some PROMs, such as the Michigan Hand Questionnaire (MHQ), include a satisfaction domain, most do not. The MHQ’s satisfaction domain measures satisfaction with the condition of the hand (function, motion, sensation), not with treatment outcome. The latter implies a comparison with baseline. As such, there are no validated PROMs that are dedicated to satisfaction with treatment even though many investigators have included a variety of nonvalidated satisfaction PROMs, a practice that should be avoided [3].

Satisfaction with treatment is multidimensional. Surgeons and investigators should be cognizant of the distinction between satisfaction with treatment outcomes and satisfaction of care delivery and environment [5]. Satisfaction with treatment outcomes is determined by the starting point, adequate understanding of the problem by the patient and provider, treatment goals, choices made, implementation, compliance, understanding of the spectrum of possible results, and comprehension of associated risks. Satisfaction with delivery is determined by cost (from the patient perspective), convenience, and comfort (such as scheduling, travel time, time off, waiting, interaction with staff and providers), perhaps among other things. Of note, each patient values each variable differently. Some can be paradoxical, such as satisfaction with pain relief accompanied by dissatisfaction with motion following joint fusion, and satisfaction with motion but dissatisfaction with cost following enzymatic digestion for Dupuytren contractures. In addition, confounding variables not captured by PROMs (such as depression, anxiety, and social determinants of health) have an effect, yet are outside the scope of the problem being addressed. As such, the level of satisfaction does not necessarily reflect the quality of clinical care, which creates a conflict, as PROMs and satisfaction are intended to determine the value of care provided. These complex issues led Clinical Orthopaedics and Related Research® to be skeptical of studies that investigate satisfaction with treatment outcomes [5].

The current study by De Ridder and colleagues [1] sought to determine reliability and construct validity of the Satisfaction with Treatment Result Questionnaire (STRQ) following the treatment of a variety of hand and wrist conditions. The questionnaire included two questions: “How satisfied are you with your treatment result thus far?” and “If you would be in the same circumstances, would you be willing to undergo this treatment again?” They concluded that the STRQ had good-to-excellent construct validity and very high test-retest reliability in patients treated for hand and wrist conditions. They also noted a moderate association between the two [1]. Their findings suggest that asking about satisfaction with treatment outcomes and willingness to have the surgery again are worthwhile including in our routine patient evaluations. They may even be of value with long-term follow-up.

Where Do We Need To Go?

Several points remain to be investigated in order to arrive at a usable patient satisfaction assessment tool. While the two questions in the study by De Ridder et al. [1] are fairly clear, do these two and other similar questions truly assess satisfaction with treatment outcomes in a comprehensive fashion? Content validity, as opposed to face validity, is an assessment of the degree to which a measure’s content reflects the construct it seeks to measure [6]. Determining content validity is a key first step toward developing a PROM. I suspect that any “Satisfaction with Treatment Outcomes PROM” will need to be multidimensional as mentioned earlier, as each element of a treatment’s result may warrant its own domain. These domains include, but may not be limited to, strength, motion, activity and participation, pain, esthetics, and sensation. Besides, the two STRQ questions were assessed individually, so they really are two single-question PROMs rather than a single, two-question PROM, as there was no total STRQ score that was analyzed.

There has been a recent interest in the Single Assessment Numeric Evaluation (SANE) in orthopaedics, and a recent article validated the SANE in hand surgery [2]. Validation of the SANE was done by benchmarking it against established multidomain PROMs. We currently do not have a validated multidomain satisfaction PROM, so the adequacy of using SANE to measure satisfaction with treatment outcomes still needs to be determined. Asking just a few questions may be adequate for an overview, but may also be an oversimplification that blurs the differences between distinct yet very important satisfaction with treatment outcome domains. As such, the results of a satisfaction with treatment outcome SANE may prove effective at assessing patients’ sentiments regarding treatment, but I am suspicious they will prove to be enough to guide improvements in what we do.

How Do We Get There?

Developing a PROM starts with item development and reduction. Items are collected from stakeholders, which include the patients themselves, as well as a variety of healthcare providers such as hand surgeons, hand therapists, caregivers, and administrators [3]. These items are considered by a panel of qualitative research experts who review the items’ wording and remove redundancies (item reduction). A pilot study is then conducted to determine ceiling and floor effects as well as missed responses of various items to determine whether such items are adequate and relevant [6]. If we use the STRQ as a starting point, we might combine the two questions and, instead of item reduction, consider item addition to reflect other aspects of patient satisfaction with outcomes. Only after this process is completed will we be able to find out whether one, two, or more questions are needed to assess satisfaction with treatment outcomes.

Does satisfaction with treatment outcomes warrant a standalone PROM? This question is tricky. A standalone satisfaction PROM can be validated and applied across a variety of fields beyond hand surgery or orthopaedics. The two questions in the STQR are not specific to hand or orthopaedic surgery and, as such, are generic PROM questions. A more detailed, multidimensional satisfaction with treatment outcomes PROM would delve into the details of treatment outcomes and thus become more specific to certain fields. For example, knee patients probably have a completely separate set of items than those of patients with hand injuries.

Finally, what do we do with satisfaction scores? Interpretability is the degree to which one can assign qualitative clinical meaning to a PROM score or change in score. Valid and reliable PROMs are useless without an established clinical value. Interpretability is addressed by determining the minimum clinically important difference [4].

In conclusion, satisfaction remains to be an elusive construct to measure, but progress is being made. I believe we are getting better at identifying and understanding the factors that drive patient satisfaction and encourage further investment in this area of PROMs.