Developing a core outcome set for hand fractures and joint injuries in adults

Abstract

Aims

The aim of this study was to develop a core outcome set of what to measure in all future clinical research on hand fractures and joint injuries in adults.

Methods

Phase 1 consisted of steps to identify potential outcome domains through systematic review of published studies, and exploration of the patient perspective through qualitative research, consisting of 25 semi-structured interviews and five focus groups. Phase 2 involved key stakeholder groups (patients, hand surgeons, and hand therapists) prioritizing the outcome domains via a three-round international Delphi survey, with a final consensus meeting to agree the final core outcome set.

Results

The systematic review of 160 studies identified 74 outcome domains based on the World Health Organization International Classification of Functioning, Disability, and Health. Overall, 35 domains were generated through thematic analysis of the patient interviews and focus groups. The domains from these elements were synthesised to develop 37 outcome domains as the basis of the Delphi survey, with a further four generated from participant suggestions in Round 1. The Delphi survey identified 20 outcome domains as ‘very important’ for the core outcome set. At the consensus meeting, 27 participants from key stakeholder groups selected seven outcomes for the core outcome set: pain/discomfort with activity, pain/discomfort with rest, fine hand use/dexterity, self-hygiene/personal care, return to usual work/job, range of motion, and patient satisfaction with outcome/result.

Conclusion

This set of core outcome domains is recommended as a minimum to be reported in all clinical research on hand fractures and joint injuries in adults. While this establishes what to measure, future work will focus on determining how best to measure these outcomes. By adopting this patient-centred core outcome set, consistency and comparability of studies will be improved, aiding meta-analysis and strengthening the evidence base for management of these common and impactful injuries.

Cite this article: Bone Jt Open 2023;4(2):87–95.

Introduction

Hand fractures and joint injuries are common,^1-3 with significant impact on patients, healthcare resources, and the wider economy through lost productivity.^4,5 A priority setting partnership has determined the priorities for future research on hand and wrist conditions, including trauma, as agreed by both patients and clinicians.⁶ A recognized lack of consensus on the key patient-centred outcomes to be collected hinders the usefulness of such research and makes it challenging to interpret the available evidence as results cannot be meaningfully compared. Several reviews of the management of hand fractures and joint injuries highlight inadequate outcome assessment and large variation in reported outcomes.^7-9 Such inconsistencies lead to research waste, as study outcomes cannot be meaningfully combined in meta-analyses or systematic reviews.¹⁰

A solution to the issue of heterogeneous outcome selection is the concept of a core outcome set (COS). This is an agreed minimum set of outcomes which should be assessed in clinical research on a given health condition, thereby improving consistency and comparability.¹¹

The aim of the SO-HANDI study (Standardized Outcomes for HAND Fractures and joint injuries in adults) was to establish such a core set of outcome domains to be used in future clinical research on all interventions for hand fractures and joint injuries in adults.

Methods

The SO-HANDI study was based on standards recommended for the development of core outcome sets as per COS-STAD (Core Outcome Set STAndards for Development) and guidelines by the Core Outcome Measures in Effectiveness Trials (COMET) Initiative.^11-13 We followed the recommended standards for reporting our COS development as per COS-STAR (Core Outcome Set STAndards for Reporting).¹⁴ There were two phases to the SO-HANDI study (Figure 1). In Phase 1, we identified outcome domains that key stakeholders might find relevant and important. Phase 2 prioritized the outcome domains to determine a final COS of ‘what’ to measure. The study was registered in the COMET database.¹⁵

Fig. 1.

Flowchart of the key steps and phases of the SO-HANDI study.

Stakeholder participants and eligibility

Patients, hand surgeons, and hand therapists were identified as the key stakeholders. Adult patients with hand injuries were identified through fracture clinics at a UK trauma centre (Queen's Medical Centre, Nottingham). These injuries included fractures (of the phalanges, metacarpals, carpal bones, or distal radius or ulna), or injuries to any of the joints between these bones. We excluded complex hand injuries (i.e. ‘mangled hand’, amputations requiring replantation), primary nerve injuries, burns, and open tendinous injuries.

Surgeons and hand therapists who manage patients with the included injuries were eligible, with the requirement that they work at an independent practitioner level (i.e. consultant or equivalent) and have a subspecialty interest in injuries within the scope of the COS. Clinician participants were identified through established national and international clinical and research networks and professional societies. This involved email correspondence sent to the Secretariat of each of the member societies of the International Federation of Societies for Surgery of the Hand and the International Federation of Societies for Hand Therapy. We also publicized the study through the British Society for Surgery of the Hand (BSSH) newsletter, the British Association of Hand Therapists (BAHT) e-bulletin, the Centre for Evidence Based Hand Surgery Hand Surgery Evidence Updates, and via an announcement and brief presentation to the audience of the weekly Derby Pulvertaft webinar. Many of these clinicians have a role as clinical academics, with both a clinical and researcher perspective.

Phase 1: outcome identification

A systematic review of published clinical research over a five-year period from 2014 to 2019 was conducted,¹⁶ as per the protocol registered on PROSPERO (CRD42019126299), with outcomes extracted verbatim. In brief, we extracted unique outcomes from recently published clinical studies on management of hand fractures and joint injuries. These were categorized into outcome domains as per the World Health Organization International Classification of Functioning, Disability and Health (WHO ICF) framework.¹⁷ Patient-reported outcome measures were analyzed by component subscales and, where appropriate, linked to multiple WHO ICF domains.¹⁸

The patient perspective on outcome domains of relevance to their injuries was explored through extensive qualitative interviews and focus groups with thematic analysis to identify the outcome domains of relevance to this stakeholder group. This process continued until the point of data saturation in terms of generation of novel themes. Further details are provided in the Supplementary Material, and the online protocol on Figshare.¹⁹

These two streams were synthesized by the research group to generate a longlist of outcome domains and descriptors. A table summarizing the linking step is provided (Supplementary Material table i). The longlist of domains and descriptors was presented to stakeholder representatives, including patients, members of the BSSH Research Committee, and members of the BAHT Clinical Evidence Committee. The goal was to ensure a clear and layperson-friendly list of outcome domains and descriptors to minimize ambiguity and facilitate the prioritization work of Phase 2.

Phase 2: outcome prioritization

The longlist of outcome domains was prioritized through a two-step international consensus process conducted in English, with further details provided in the online protocol on Figshare.²⁰ For both steps, pre-specified consensus thresholds were selected by the research group and included in the protocol.

We attempted to recruit clinician participants from all member nations of the international hand surgery and therapy societies via the various secretariats. In addition, we contacted the corresponding authors of all studies included in our earlier systematic review of treatment outcomes to invite them to participate, and encouraged them to spread the word to any other clinicians in the field that they felt might be interested in participating.¹⁶

The first step consisted of patients, surgeons and therapists completing a three-round online Delphi survey using the COMET Initiative DelphiManager software.²¹ Participants rated each outcome for importance to be measured in all future clinical research on hand fractures and joint injuries, using a nine-point Likert scale with three distinct categories specified (1 to 3 = ‘less important/not important’; 4 to 6 = ‘important but not very important’; and 7 to 9 = ‘very important’), or selecting ‘unable to score’. At the end of Round 1, participants were invited to enter free text suggestions for additional outcomes to be incorporated into subsequent rounds. Suggested outcomes were reviewed by four members of the research group, with agreement reached on whether they warranted the generation of additional Delphi outcome items. In subsequent rounds of scoring, ratings of each of the three stakeholder groups were presented in summary through bar charts revealing the separate distribution of scores, and each participant was reminded of the previous rating they gave each outcome in the previous round. Consensus status was determined at the end of Round 3 using pre-specified criteria (Table I). All ‘consensus important’ outcomes were automatically entered for detailed consideration at the stakeholder consensus meeting detailed below, while ‘no consensus’ ones were briefly considered by discussion and a vote to ‘salvage’ for further consideration.

Table I.

Pre-defined percentage threshold consensus criteria for Delphi study.

Consensus status	Required ratings thresholds
Consensus important	ALL stakeholder groups have: ≥ 70% rating 7 to 9 AND ≤ 15% rating 1 to 3
Consensus not important	ALL stakeholder groups have: ≥ 70% rating 1 to 3 AND ≤ 15% rating 7 to 9
No consensus	All other scenarios

The second step consisted of an international online consensus meeting, conducted using Teams (Microsoft, USA) and the Poll Everywhere platform for online voting.²² The panel comprised key stakeholder groups as previously described, with the addition of a health economist and trial manager. When recruiting for the meeting, the patient stakeholder group purposely had greater weighting than any other to preserve the patient voice at this stage of the development process.

The meeting focused on discussion of the ‘consensus important’ outcomes, with a goal of determining those most important and necessary for the final COS. Early in the meeting panellists had an opportunity to decide whether any 'no consensus' outcomes from the Delphi should be added for further consideration as potential outcomes to include in the COS – to do so, an outcome required a ≥ 80% ‘yes’ vote.

The Chair (JJK) was a senior academic with experience in COS development. An adapted nominal group technique was used. Through discussion in three small groups with a stratified mix of the key stakeholder groups, participants were tasked with categorizing ‘consensus important’ outcomes from the Delphi (plus any of the others which were ‘salvaged’ by the initial phase of the meeting). Outcomes were categorized as

• ‘Essential’ (3 points);

• ‘Important but not essential’ (1 point), or;

• ‘Not needed’ (0 points)

Points based on the three small group categorizations were aggregated, and shown to meeting participants. Any outcomes scoring a total of 0 or 1 were eliminated. After whole group discussion of the reasons for prioritization category given to the remaining outcome domains, participants voted anonymously (yes/no) on each of the remaining items. For an outcome to be included in the COS at this stage, it required a ≥ 80% ‘yes’ vote.

Patient and public involvement

Patient input was sought during consideration of the methods, review of study documents, and design of the survey, including the wording of outcomes, descriptors, and examples. Typically, the patient voice is not well-represented in outcome selection for clinical research,¹¹ and yet as the end-user of the evidence base that is developed through clinical research, it should be a priority. Patients brought expertise of their lived experience to the development process, with the key eligibility requirement being that they had sustained an injury within the scope of the COS.

Ethics approval

Ethical approval was granted by the South Central Berkshire B Research Ethics Committee (19/SC/0549) (qualitative study) and by the London - Harrow Research Ethics Committee (20/PR/0178) (Delphi and consensus meeting). All were adopted by the National Institute for Health Research (NIHR) Clinical Research Network Portfolio (Portfolio numbers 43855 and 47694).

Results

A summary of key results at each stage of the study is presented in Figure 2. Through the systematic review, 639 unique outcomes were identified across 160 studies. These ranged across 74 WHO ICF outcome domains.¹⁶ The qualitative study developed 35 outcome domains across six broad themes. The two sets of domains were synthesized by the research group to form a longlist of 37 outcome domains, with linking of outcome domains as shown in Supplementary Material table i. These domains were presented in Round 1 of the Delphi survey along with descriptors and examples for clarification (Supplementary Material table ii).

Fig. 2.

Summary of key results from the various steps of the SO-HANDI study.

Delphi survey

A total of 160 patients, hand therapists, and surgeons registered for the Delphi survey, of whom 152 (57 patients, 40 therapists, 55 surgeons) completed Round 1. Nearly 95% (144/152), including 54 patients, 35 therapists, and 55 surgeons, completed the full study (Figure 2). All patients completing the entire survey were from the UK, as were 54% (49/90) of the hand therapists and surgeons. The remaining 41 therapists and surgeons were from 21 countries. Further details are provided in Supplementary Material table iii.

A summary of overall rating category percentages across the three survey rounds for each outcome and stakeholder group is presented in Supplementary Material tables iv-vi. Four additional outcomes were added as a result of participant suggestions at the end of Round 1: ‘pain/discomfort during activity’, ‘pain/discomfort during rest’, ‘patient satisfaction with outcome/result’ and ‘speed of movement’. The first three were more specific descriptors of existing outcomes presented at the start of the Delphi rather than novel domains. However, as they were suggested by participants, the research group considered that their narrower scope may offer an important qualifier to the outcome from the participant perspective; hence it was decided to add these to the Delphi in Round 2. Other suggestions amounted to overlap with or duplication of existing outcomes, or were not consistent with the concept of ‘what’ to measure but rather ‘how’ to measure. Further details are provided in Supplementary Material Table vii.

Consensus was reached for 20 outcomes being ‘very important’ for potential inclusion in all future clinical research on hand fractures and joint injuries (Supplementary Material table viii). No consensus was reached for the remaining 21 outcomes.

Given the very low attrition rate (Figure 2), it was deemed unnecessary to complete an attrition analysis between completers and non-completers.

Consensus meeting

A total of 27 participants attended the virtual consensus meeting (12 patients, seven surgeons, six hand therapists, one health economist, and one trial manager). All except one patient, one surgeon, the health economist, and the trial manager had taken part in the Delphi survey. The meeting participants were predominantly from the UK, but four were from four other countries (USA, Canada, South Africa, and Sweden). All were sent a pre-meeting information pack in preparation for the meeting. This provided an outline of the concept of core outcome sets, a summary of the study and the Delphi results, and the plan for the meeting.

Although ten of the 21 outcomes which had reached ‘no consensus’ in the Delphi were nominated for further consideration at the meeting, none passed the vote for salvage. The remainder of the meeting therefore involved solely the 20 outcomes which had reached consensus for inclusion via the Delphi survey. Further discussion and prioritization of the 20 outcomes resulted in aggregate points as shown in Figure 3. Participants highlighted the value of ‘patient satisfaction with outcome/result’ and it became clear that in the small groups this outcome was assigned a low priority mainly because of concern about difficulty in being able to measure it. During whole group discussion, participants were therefore reminded that the goal of this meeting was to focus of ‘what’ to measure, putting aside ‘how’ to measure an outcome when reaching any decisions here. Discussion on other outcomes continued until participants felt all points had been raised. Participants were then invited to a final ‘yes/no’ vote on whether each of the 20 outcomes should be in the final core outcome set. Seven outcomes met the pre-specified ≥ 80% ‘yes’ threshold: fine hand use/dexterity; pain/discomfort during activity; pain/discomfort during rest; return to usual work/job; self-hygiene/personal care; range of motion (ROM); and patient satisfaction with outcome/result. They are shown together with their descriptors in Table II.

Fig. 3.

Aggregate of small group categorizations on importance of outcomes for inclusion in core outcome set.

Table II.

Core outcome set for hand fractures and joint injuries in adults.

Core outcome domain	Descriptor
Fine hand use/dexterity	Being able to do fine motor tasks or precise activities with the hand/wrist e.g. writing, drawing, picking up coins from a table, using a key
Pain/discomfort during activity	Discomfort or pain in the hand or wrist specifically during activities (NOT at rest) e.g. ache, shooting pain, sharp pain, throbbing, discomfort/pain due to not being able to tolerate hot or cold sensation
Pain/discomfort during rest	Discomfort or pain in the hand or wrist specifically during rest (i.e. with the hand/wrist not moving, so NOT during activities) e.g. ache, shooting pain, sharp pain, throbbing, discomfort / pain due to not being able to tolerate hot or cold sensation
Return to usual work/job	Being able to return to the work or job that one was doing prior to their hand/wrist injury (NOT including the financial impact of any lost income)
Self-hygiene/personal care	Being able to do the usual tasks involved in maintaining one’s own hygiene and self-care e.g. washing oneself, toileting, washing hands, washing the face, brushing teeth, shaving, looking after one’s hair, applying make-up
Range of motion	How much movement one has through the joints of the hand or wrist, whether active (i.e. moving it with the muscles of the injured side) or passive (e.g. if someone else were to try to move it for the patient) Includes stiffness in the joints or how much one can bend or straighten the thumb, fingers or wrist
Patient satisfaction with outcome/result	Satisfaction with the overall result from the patient’s perspective (NOT with treatment or recovery process, but the end result only)

Discussion

Through a systematic process, the SO-HANDI study identified a consensus among key stakeholders for the core outcome domains to measure in all clinical research of hand fractures and joint injuries. The process centred on patients and sought an international perspective from clinicians. The COS informs on what outcome domains are to be captured when planning future research. It is also of potential value to clinicians not involved directly in research in this field, as the seven COS domains could be considered in future consultations with patients who have sustained these injuries as a baseline of the typical outcomes felt to be of core importance. Discussions would naturally be tailored on a patient-by-patient basis, but some discussion of these seven domains might help to set appropriate expectations in these key areas.

This is the first patient-centred COS to focus on hand fractures and joint injuries. Previous work has attempted to establish the most important domains for distal radius fractures specifically,²³ or for hand and wrist conditions very broadly.^24,25 All have taken different approaches, typically involving only minimal representation of the patient perspective on the importance of outcomes.

The International Consortium for Health Outcomes Measurement (ICHOM) standard set for hand and wrist conditions is designed specifically for use in clinical settings rather than research.²⁵ While the standard set recommends a number of outcome domains which overlap with those of the SO-HANDI COS, there are important distinctions between the two. The SO-HANDI COS specifically defines the aspects of hand function to be measured, that pain/discomfort should be reported both for the rest state and during activity, and that ROM should be measured for all injuries within the scope of the COS. The ICHOM standard set appeared to only directly involve patients’ input at the final stage to ratify outcome domains already selected by health professionals, rather than as an integral stakeholder group in creating a longlist of domains, iterative Delphi rounds and selection of core domains.²⁵

A key strength of the SO-HANDI COS is that the rigorous and reproducible methodological approach is based on core outcome set consensus guidelines including standards for development and reporting,^13,14 thus providing clarity and credibility about the process. The SO-HANDI COS has had the benefit of an extensive patient qualitative study of 35 patients to generate the outcomes domain longlist, as well as patient input alongside that of clinicians when developing the outcome domain descriptors. By involving patients throughout, the COS aims to ensure that the patient voice is heard – this is further reinforced by patients comprising the largest stakeholder group in the consensus meeting (and therefore carrying the highest voting power). The Delphi attrition rate was very low, counter to the typical issue arising with this methodology,²⁶ and compares favourably to other recent orthopaedic and musculoskeletal COS development Delphi surveys.^27,28

On revealing the core outcome set at the meeting, some participants voiced a concern about the fact that the outcome ‘emotional/mood impact to self’ did not reach consensus for consideration. On further discussion, while both patients and clinicians recognized the importance of this outcome domain as a holistic aspect of patients’ health, there appeared to be general agreement that for most injuries within the scope of the COS the impact on emotion/mood was secondary to impact on one or more of the seven outcomes selected for the final COS. It should be noted though, that use of a COS when designing studies does not preclude any supplementary outcome being measured in an individual study with specific focus.

There are some limitations. Despite the process reaching an international range of surgeons and hand therapists, patient participants were recruited from a single centre in the UK. The SO-HANDI study systematic review was not limited by geographical location, but did still have an English language restriction. Furthermore, the Delphi survey and consensus meeting were in English only, a logical step given that the qualitative work was also based on outcome domains derived from English-speaking participants. Despite our recruitment efforts, participation was disproportionately from higher income countries. This issue of under-representation of participants from low and middle income countries in COS development studies has been highlighted previously.²⁹ There are numerous reported barriers in conducting research in lower income countries, including a lack of time, lack of funding, competing priorities, and regulatory obstacles.³⁰

A COS is the minimum set of outcomes agreed to be of critical importance to always be measured in future clinical research. Nevertheless, outcome selection is not restricted to only the COS in any individual study and researchers may measure additional outcomes to these core ones, as deemed appropriate. To that end, we also present the outcomes which were deemed ‘very important’ through the Delphi survey, as well as those that reached a threshold of ‘very important’ in one or two stakeholder groups (Supplementary Material table ix). This may be used to inform researchers considering supplementing their outcome selection beyond the core outcome set.

Finally, the COS defines ‘what’ should be measured, but not ‘how to measure’ these domains. One reason to separate the steps of ‘what to measure’ and ‘how to measure’ is that as advancements are made, an outcome domain that at one time can be seen as impossible to reliably measure subsequently has an appropriate measurement instrument developed. By not restricting our ‘what’ domains on the basis of ‘how’ they can be measured at this stage, we reduce the chance that a domain that stakeholders have agreed is important is disregarded for a measurement issue that could well change in future. Of course, an important step to implement the COS is to proceed to define ‘how to measure’ the outcomes. Further work on outcome measurement for hand conditions, and in particular trauma, is needed to establish the optimal ‘how’ and ‘when’ to measure these core domains. In the interim, researchers should aim to select measurement tools that are patient-centred in their development, well-received by end-users, and with the best clinimetric evidence for the COS outcome domains.³¹

Take home message

- Clinical research on hand fractures and joint injuries has issues of research waste and heterogeneity in outcome selection. This paper provides clinicians with an insight into a potential solution to address these problems.

- Of more direct clinical relevance, we recommend that the seven core outcome set domains are considered as potential areas of discussion in future consultations with patients who have sustained these injuries.