International multiphase mixed methods study protocol to develop a cross-cultural patient-reported outcome and experience measure for hand conditions (HAND-Q)

Abstract

Introduction

patient-reported outcome measures (PROMs) are instruments used to measure outcomes and experiences of healthcare from the patient perspective. The specific methodology used to develop a PROM should be communicated to establish the quality of the instrument. This mixed methods protocol describes the development of a cross-cultural, internationally applicable PROM for hand conditions, the HAND-Q.

Methods and analysis

The multiphase approach used for this study has been previously used with the development of other PROMs by our team (eg, BODY-Q, BREAST-Q, CLEFT-Q, FACE-Q). In Phase I, we establish what important concepts matter to patients with hand conditions. A conceptual framework is developed from a systematic review of existing PROMs in the field and an extensive international qualitative study. Interpretive description is the qualitative approach used. Item generation is based on the qualitative data. The preliminary scales will be created for each theme identified in the conceptual framework. These scales will be refined by cognitive debriefing interviews with participants and expert input. Phase II involves a large international sample of patients with varied hand conditions completing the field-test version of the HAND-Q. The scales will be refined using the modern psychometric approach of Rasch Measurement Theory. Analysis will result in a shortened set of clinically meaningful and scientifically robust HAND-Q scales.

Ethics and dissemination

This study is coordinated at Flinders University (Adelaide, Australia) where it has ethics board approval for phase I and phase II. Findings will be published in peer-reviewed journals and presented at local, national and international conferences.

Strengths and limitations of this study.

• The inclusion of a diverse international sample of participants in the field-test ensures that the HAND-Q will serve the global community.

• Qualitative interviews have only been performed with English speaking patients from Australia, the USA and Canada.

• Including the full spectrum of hand conditions and injuries with minimal participant exclusion criteria will ensure that the HAND-Q is broadly applicable to any form of congenital, traumatic or degenerative hand condition.

• Independently functioning scales will allow tailoring of scales to the patient, study or clinical setting, which will reduce patient and administrative burden.

• Use of a modern psychometric approach will produce HAND-Q scales that are both scientifically sound and clinically meaningful.

Introduction

Patient-reported outcome (PRO) data are increasingly collected and used in the assessment of treatment effectiveness and healthcare funding.^{1 2} It is important that PRO data are captured with appropriately robust instruments that satisfy best practice guidelines.^{3 4} The methodology used to develop and validate a patient-reported outcome measure (PROM) can be complex; however, it is important for researchers and clinicians to understand the approach used in order to judge the appropriateness of an instrument for their specific application.^{5 6}

There are a wide variety of conditions and injuries that affect the hand, and each has the potential to have an enormous impact on an individual’s daily function, their livelihood and their quality of life. Conditions include chronic degenerative bony diseases, such as osteoarthritis and rheumatoid arthritis, chronic progressive soft tissue disorders, such as Dupuytren’s contracture, and acute injuries of the hand. Hand injuries occur commonly within the workplace and in the home environment and can range from a simple skin laceration to complex tendon and bone damage. This heterogeneity of conditions and causes can make the measurement of treatment outcomes extremely challenging and can limit the clinical applicability of condition-specific PROMs.

There is growing awareness of the economic and environmental impacts of medical care, particularly in the surgical sphere where access to the operating theatre is competitive and expensive.^{1 7 8} These forces have led to an increasing amount of hand surgery performed outside of operating theatres with a fully awake patient and the use of only local anaesthetic.⁹ This wide-awake approach minimises the economic burden of hand surgery.^{10 11} It is important to be able to measure patient satisfaction with their experience of hand surgery anaesthesia using an appropriately designed PROM to allow for comparisons to be made of the different anaesthetics approaches to hand surgery. The HAND-Q will explore experience of care concepts that have not been addressed by existing PROMs.

PROMs developed using a modern psychometric approach have the potential to function consistently regardless of the population that is being measured. This is possible because the psychometric properties of the PROM are that of the instrument itself, not a reflection of the population that is being tested. This approach to measurement allows the same PROM to be used in different populations with results comparable on the same metric. Legacy instruments developed using classical test theory methodology cannot be used in this manner. A further benefit of the modern psychometric approach is that measurement is given in interval format, which permits accurate and meaningful measurement of change, which is particularly important in the surgical field.

The vast majority of PROMs are developed in a single language and usually within a single country. As hand conditions are prevalent around the world, this approach means that there is limited content validation within different cultures and economic environments. Many PROMs used in hand surgery have been developed in English and then translated into multiple other languages. Patient outcome of Surgery-Hand/Arm¹² and the Patient Evaluation Measure¹³ were both developed in the UK. The Disabilities of the Arm, Shoulder and Hand¹⁴ and the Michigan Hand Questionnaire¹⁵ were developed in North America and subsequently translated into many languages.^16–19 Although guidelines exist to ensure appropriate cross-cultural adaption of PROMs,^{20 21} a more rigorous approach is to involve people from different countries of varied economic status in the development and validation of a new instrument, rather than performing translation retrospectively.⁵ An instrument developed in a cross-cultural manner could encourage and facilitate multinational research in the field of hand surgery.

The process to develop a cross-cultural PROM that is both scientifically sound and clinically relevant is a complex undertaking. The following protocol details the methodology that will be used for the international study to develop the HAND-Q.

Methods and analysis

Overview of patient-reported outcome measure development

We use a modern psychometric approach and engage patients and experts in all phases of the development to create clinically meaningful and scientifically robust PROM scales. The development of the HAND-Q follows the international best practice guidelines as determined by the Scientific Advisory Committee of the Medical Outcomes Trust,⁴ the USA Food and Drug Administration³ and the International Society for Pharmacoeconomics and Outcomes Research (ISPOR).^{22 23} We have previously published⁵ the three-phase mixed methods approach that we employ to develop a PROM, a summary is reproduced in figure 1. This approach establishes a PROM that satisfies the minimum requirements of reliability and validity as set forth by the International Society for Quality of Life Research²⁴ and the Consensus-based Standards for the Selection of Health Status Measurement Instruments (COSMIN).^{6 25} The application of this three-phase approach to the development of the HAND-Q will be expanded upon in this paper.

Figure 1.

Flow diagram illustrating the multiphase mixed methods approach to the development of the HAND-Q. QUAN, quantitative study component; QUAL, qualitative study component. Image reproduced from Wong Riff et al.⁵

Phase I: What should we measure?

The objective of phase I is to develop preliminary scales founded on excellent content validity. A systematic literature review was performed to identify the currently available PROMs relevant to hand conditions and hand surgery.²⁶ This comprehensive literature search was carried out according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines with a total of 3039 citations retrieved. A total of 24 instruments were reviewed; 10 regional upper limb, six hand/wrist specific and eight hand condition-specific instruments. Review of the developmental methodology of those existing instruments revealed areas of significant weakness.²⁶ The review also found profound content limitations with no hand-specific instrument found that covers the full spectrum of outcomes relevant to hand surgery. Based on this review it was determined that there was need for a comprehensive PROM for hand surgery patients and we proceeded with this study. Concepts from other PROMs were used to form a preliminary interview guide.

Conceptual framework

The starting point in any phase I PROM development project is the development of a conceptual framework,⁴ which includes a description of the concepts of interest and the relationship between these concepts within the population that the instrument is designed for. Based on the findings of the systematic review, concepts from existing instruments are mapped along with other concepts of interest to create a preliminary conceptual framework.

Qualitative study

The preliminary conceptual framework will be used as a basis for an interview guide to establish which of the concepts are meaningful to patients. We use an approach from applied health services research known as Interpretive Description.^{27 28} This approach acknowledges that there is pre-existing theoretical and clinical knowledge informing the study. In addition, this approach aims to produce knowledge relevant to the clinical context with the proviso that the individual’s understanding of a concept is of the greatest importance, regardless of the clinical or theoretical explanation.²⁹

Participants, setting and recruitment

Phase 1 qualitative study will recruit participants from centres in Adelaide, Australia and Saint John, New Brunswick, Canada. Recruitment will be by direct referral from clinicians or administrative staff. Participants will be eligible for inclusion if they had experienced surgery on either or both of their hands over the preceding12 months, with a minimum of 4 weeks since their surgery. Exclusion criteria for this phase of the study will be the inability to speak English or cognitive delay that prevents participation in a semistructured interview.

Sampling

Participants will be purposively sampled to include a heterogeneous population with respect to age, gender, hand condition, setting of surgery (hospital operating theatre vs private rooms), funding for surgery (public vs private) and whether surgery was performed with general anaesthesia, sedation or local anaesthetic. Sampling will continue until saturation is achieved, ie, no further important concepts are identified in three consecutive interviews.³⁰

Data collection

Written consent will be obtained from all participants before commencing with individual, semistructured interviews. Participant age, gender, hand condition and date of surgery will be collected. Interviews will be audio recorded and transcribed verbatim with all identifiers removed. The interview guide (box 1) used is based on concepts identified from the literature review²⁶; this guide includes a list of open-ended questions to encourage discussion. The interviewer will probe for new concepts during the interviews. An iterative approach will be used, whereby data collection and analysis will take place concurrently in order to ask about newly identified relevant concepts.

Box 1. Interview guide for qualitative interviews to be performed in phase 1.

Experience of care

1. What treatments have you had for your condition?

2. What was good or bad about the treatment?

3. If the participant has had surgery:

   1. What was your experience of the anaesthetic used? Probe: general anaesthetic, block, local

   2. Would you have considered having treatment under local anaesthetic? Probe: why, why not

4. Who do you see when you come to the hospital clinic? Probe: receptionist, nurse, doctor, occupational therapist

5. What are the people like who care for you? Probe: friendly, made you feel comfortable, easy to talk to, listened to you

6. What kind of verbal and written information did they give you? Probe: gave enough information, let you ask questions, answered your questions, provided information about recovery

Physical function

1. Does your condition create any functional problems? Probe: work, personal care, hobbies

2. What specific things do you have difficulty with due to your hand problem? Probe: getting dressed, cooking, typing, sport

3. Do you experience any symptoms related to your functional problem? Probe: pain, discomfort, embarrassment, mood disturbance

Psychological well-being

1. How does your hand problem make you feel? Probe: frustrated, angry, upset, worried, stressed

2. How does your hand problem make you feel about yourself? Probe: self esteem, body image, confidence, self-conscious, different from others

Appearance

1. How would you describe the appearance of your hand/s? Probe: from close up, from far away, symmetry, texture, attractiveness

2. How has your hand appearance changed since your treatment? Probe: scarring, descriptive detail

3. What do you like or dislike about your hand appearance?

4. Is there anything about your hand appearance that you would like to change? Probe: for details

5. Do you ever hide your hands? How do you do this?

6. How important is the appearance of your hands to you?

Other

1. Is there anything I have not asked you that you think it is important for me to know?

2. Would you like to receive a copy of the transcript from today’s discussion?

3. Would you be interested in participating in Cognitive Interview?

Data analysis

A line-by-line approach will then used to code the data, with a process of constant comparison used to identify common concepts of interest.³¹ A codebook will be developed that outlines the codes for evolving domains and themes. The information from these interviews will be used to shape and reconfigure the conceptual framework to represent outcomes and experiences important to participants.

Rigour

Several strategies have been put in place to ensure rigour of this qualitative study. A single interviewer will perform all of the qualitative interviews. One team member will perform all of the coding of the transcripts, which will then corroborated by a second team member. The iterative process allows for member checking to confirm if concepts are deemed to be valuable to subsequent participants.³² The members of the study team will discuss data analysis and this peer debriefing will provide consistency.³²

Item generation

The evolved conceptual framework will be used to identify which scales should be generated. A comprehensive list of potential items will be generated from the coding process. Each of the domains identified will have multiple themes; each important theme will be developed into a preliminary scale. This approach will ensure that the suite of scales that comprise the HAND-Q will cover all of the important concepts of interest identified by patients with hand conditions. The scales are constructed from the items identified in the coding process; participants’ language is kept intact as much as possible.

Refining the preliminary scales

Incorporating feedback from both patients and experts in hand conditions will enhance the quality of the preliminary HAND-Q scales. Participants who have previously participated in qualitative interviews as well as new participants will be invited to participate in cognitive debriefing interviews. Input will also be sought from a multidisciplinary international pool of experts who will be invited to provide feedback via an online survey using Research Electronic Data Capture (REDCap).

Cognitive debriefing interviews

After the preliminary scales are constructed, they will be shown to participants with hand conditions in the form of semistructured cognitive interviews. These interviews will be conducted by telephone, audio recorded, transcribed and analysed. The purpose of these interviews is to ensure that participants have understood the instructions, items and the response options. The participant will be asked to go through each of the scales using the ‘think aloud’ technique, while the interviewer notes the specific content that causes problems.^{33 34} Feedback will be sought on the relevance of the items to the individual. If an item is unclear to a participant, they will be asked for suggestions on how we can improve item wording. Further probing will be used to enquire about any new content that may have been overlooked. Cognitive interviews will be performed in three rounds to provide the opportunity to make changes between rounds. The first round will include participants from the qualitative interviews from both the Australian and Canadian cohorts. Subsequent rounds will also include new participants from the USA to ensure content validity for patients in that country.

Expert clinician input

It is useful to seek feedback from clinicians who specialise in the management of hand conditions to ensure that the scales capture all clinically relevant concepts. Experts in the management of patients with hand conditions from the fields of plastic and reconstructive surgery, orthopaedic surgery, hand therapy and physiotherapy will be invited to review the HAND-Q in the form of an online survey. We will administer this survey using REDCap, which is a secure web-based data collection system.³⁵ Researchers and academics in the field will also be invited to participate. An international sample of experts will be included to ensure that we engage with professionals working in diverse healthcare systems with varied treatment strategies. Experts will be asked to provide feedback on all aspects of each scale (instructions, response options, items) and to nominate missing items. Feedback from experts will be used to refine the scales. Expert input will be obtained before the final round of participant cognitive interviews in order to show any changes made from expert input to participants.

Translation

To facilitate an international field test, cultural and linguistic validation of the HAND-Q into multiple languages will take place. The field-test scales will be translated into the required field-test languages according to the guidelines set forth by the ISPOR²⁰ and the WHO²¹ recommendations for linguistic validations. In summary, this approach requires the forward translation (English to target language) to be performed by two independent bilingual individuals, with back translation (target language to English) to be performed by a third individual. Any discrepancies are resolved at each step and the resultant version is then shown to a small group of patients to ensure that the translation is valid and ready for use.

Phase II: How should we measure the concepts identified in phase I?

A large heterogeneous sample of people aged 18 years and older with any type of hand condition will be recruited. Analysis of the field-test data allows the scales to be refined to include the subset of items that are the most effective in measuring the concepts of interest. We will follow the modern psychometric method of Rasch Measurement Theory (RMT).³⁶ This approach dictates that in order to achieve accurate measurement the data must comply with the Rasch measurement model.³⁷ Performing RMT analysis will make it is possible to identify poorly functioning items and to only retain the best subset of items that together map out a clinical hierarchy for each scale. The details of the psychometric parameters used in RMT analysis are described in detail elsewhere.³⁸ Scales refined using RMT analysis can provide measurement that is suitable for application at an individual patient level, unlike instruments developed using classical test theory, which are designed for analysis of groups of individuals. RMT produces interval level measurement where the intervals between adjacent scores are equal throughout the breadth of the possible scores. Subsequently measuring change over time (such as comparing preoperative and postoperative scores which is a common PROM application in the surgical field) with RMT designed scales is mathematically sound.³⁹ Another benefit of scales derived using the RMT approach is that the scales function consistently, independent of the population that is being studied, which allows comparison of scores between different populations.

International field-test and Rasch Measurement Theory analysis

The aim of the international field-test study is to collect scale responses from a large cohort of patients with hand conditions to allow the refinement of the scales using RMT and examine the psychometric properties of the resulting scales.

Study participants

The international field test will includes participants from multiple English and non-English speaking countries. Field-test sites will be included based on their interest in participating and the feasibility of recruiting an adequate sample size over the anticipated timeframe. We aim to include sites in five countries and with 200 participants recruited per country (n=1000). Participants will include anyone aged 18 years or older with a hand condition that is able to comprehend the study and legally provide consent. Participants will be recruited in clinic settings and asked to complete the scales while waiting for their appointment.

Data collection

Demographic information will be collected to allow for subcohort analysis. Participants will be asked to complete the HAND-Q scales either using a paper booklet or electronic device (smartphone or tablet) using a REDCap survey.

Data analysis

If participants complete the scales using a paper booklet, members of the research team will enter these data into the REDCap database. Completed data files will be downloaded into IBM SPSS Statistics V.25. The SPSS file will be formatted suitably and imported into RUMM2030 for RMT analysis. Each of the scales will be analysed separately to determine its psychometric properties according to the following criteria. The thresholds for item response options must be appropriately ordered. This means that a response of ‘1’ sits lower on a continuum than ‘2’ etc. The hierarchy of items on the scale is then determined, from the ‘easiest’ to endorse to the ‘hardest’ to endorse. Several item fit statistics are used to determine if the scale works adequately¹; log residuals, which reflect the item-person interaction,² χ² statistic, which reflect the interaction between the item and the concept being measured and³ item characteristic curves which demonstrate graphically the relationship between groups of responders and the trait of interest.⁴⁰ Each item will be assessed using these parameters and excluded from the scales if they do not perform well. Lastly the targeting of the scale within the patient population is reviewed to ensure that the scale is able to measure the full breadth of the construct within the population of interest.

The next requirement in the assessment of the scales is to ensure that the items display internal consistency; that is that they are appropriately inter-related. The first step is testing for unidimensionality; which is checking if all of the items that contribute to the scale are measuring the same underlying construct. The scale is then evaluated using the Person Separation Index, which is a similar measure to Cronbach alpha in classical test theory. This process of analysis is iterative; scales that are not functioning optimally are improved by excluding items with poor performance. This process is completed when each of the scales show the aforementioned statistics within the acceptable range.

Differential item functioning

As RMT produces a scale that behaves in the same manner regardless of the population for which it is used, differences between subgroups of the population can be identified. When one subpopulation answers an item differently to another subpopulation, that item is said to display differential item functioning (DIF). Inclusion of multiple countries in the international field test will allow for any differences based on country of origin to be considered. The software RUMM2030 can identify DIF and any items that display DIF can be excluded during the item reduction process, or other adjustments made to account for any differences.

Item reduction

The location of each item on the difficulty spectrum of each scale helps to determine if there is any redundant items; that is, two or more items that measure the same point on the difficulty spectrum. Items will be reduced if they are identified as poorly functioning by the previously mentioned statistics or if they are redundant. Items will be reduced until the optimal number of items remains. This approach will be determined by a combination of the distribution of item locations as well as the clinical requirement for the degree of accuracy. Once this process is complete each of the scales is finalised. A scoring table is then produced for each scale using the RMT analysis; the cumulative score of a scale is more complex than the simple addition of individual items.

Construct validity

Following the finalisation of scales the logits will be used to transform scale scores for each participant from 0 (worst) to 100 (best). Basic relationships between scores and demographic characteristics can then be calculated using independent samples t-tests and analysis of variance (ANOVA) or the equivalent nonparametric tests as appropriate in SPSS.

There are various components of construct validity, such as structural validity, hypothesis testing and cross-cultural validity. RMT analysis of unidimensionality addresses the structural validity. Analysis for DIF addresses the cross-cultural validity of the scales. A priori hypothesis testing of known groups is a technique to assess whether the responses to the scales correlate or differ between groups in a way that would be expected.⁴¹ In the HAND-Q, we intend to test the following hypotheses using ANOVA in SPSS:

1. Those patients with hand conditions that affect the appearance of the hand (such as rheumatoid arthritis and Dupuytren’s contracture) will be more dissatisfied with their hand appearance than those with carpal tunnel syndrome (which rarely has a significant affect on hand appearance).

2. Those patients requiring further intervention for their hand condition would have lower quality of life scores than those who do not require further intervention.

3. Those patients who describe their hand condition as ‘severe’ will have lower quality of life scores than those who describe their conditions as ‘mild’.

Phase III: How does the instrument work?

In developing the HAND-Q we will comply with the COSMIN checklist.⁴² Many of the required components will be addressed in preceding phases. The third and final phase includes further psychometric testing to establish the reliability, validity and responsiveness of the HAND-Q. All testing within this phase will use the final versions of the HAND-Q scales.

Patient and public involvement

Our patient-oriented approach engages patients and healthcare providers in all stages of our research as experts and research team members. Their input is fundamental to the design of the study and development of content for HAND-Q scales. All participants in the initial qualitative interviews will be invited to continue to collaborate in our study by taking part in cognitive interviews where they can provide feedback on our findings and help to refine the final set of scales. We will disseminate updates on the HAND-Q development to the patient community and healthcare providers via local health newsletters with links and information about presentations and publications.

Ethics

Ethics board approval will be obtained prior to the study commencing at a collaborating site. Participants will provide written or oral consent prior to participating. As participants will be asked to discuss their experiences, they could potentially experience some distress. Participants will be advised that should this occur that the interviewer can put them in touch with an appropriate skilled clinician. Participants will be assured that any personal information will be kept confidential. In the qualitative phase all identifying information will be removed during the process of transcription. Any personal information required for follow-up of individual participants will be kept secure and confidential following institution rules for data storage.

Dissemination

To ensure wide uptake of the HAND-Q, once developed it will be made available free of charge for non-commercial use. Our team will actively promote the HAND-Q at local, national and international conferences. We will publish findings for the HAND-Q in journals known to be valued and read by our target audiences. This study has no intention to directly compare outcomes from different centres. Any published work or public presentations resulting from this study will not identify specific centres. Collaborations with multiple sites internationally is hoped to increase the application of the HAND-Q in the future. The individual scored HAND-Q data from phase II and phase III will be returned to the providing sites for their own use.