SUMMARY
Objective:
To develop sets of core and optional recommended domains for describing and evaluating Osteoarthritis Management Programs (OAMPs), with a focus on hip and knee Osteoarthritis (OA).
Design:
We conducted a 3-round modified Delphi survey involving an international group of researchers, health professionals, health administrators and people with OA. In Round 1, participants ranked the importance of 75 outcome and descriptive domains in five categories: patient impacts, implementation outcomes, and characteristics of the OAMP and its participants and clinicians. Domains ranked as “important” or “essential” by ≥80% of participants were retained, and participants could suggest additional domains. In Round 2, participants rated their level of agreement that each domain was essential for evaluating OAMPs: 0 = strongly disagree to 10 = strongly agree. A domain was retained if ≥80% rated it ≥6. In Round 3, participants rated remaining domains using same scale as in Round 2; a domain was recommended as “core” if ≥80% of participants rated it ≥9 and as “optional” if ≥80% rated it ≥7.
Results:
A total of 178 individuals from 26 countries participated; 85 completed all survey rounds. Only one domain, “ability to participate in daily activities”, met criteria for a core domain; 25 domains met criteria for an optional recommendation: 8 Patient Impacts, 5 Implementation Outcomes, 5 Participant Characteristics, 3 OAMP Characteristics and 4 Clinician Characteristics.
Conclusion:
The ability of patients with OA to participate in daily activities should be evaluated in all OAMPs. Teams evaluating OAMPs should consider including domains from the optional recommended set, with representation from all five categories and based on stakeholder priorities in their local context.
Keywords: Osteoarthritis, Implementation, Outcomes, Delphi
Introduction
Osteoarthritis (OA) is a serious disease that places a substantial burden on individuals and health systems1. High quality, guideline-concordant care is essential for mitigating the many negative impacts of OA. However, numerous studies have identified gaps in quality of care for people with OA, particularly underuse of recommended behavioral and rehabilitation therapies such as patient education, exercise, physical activity and weight management2–6. To address this problem, there have been efforts internationally to develop and implement Osteoarthritis Management Programs (OAMPs) that deliver comprehensive and multi-faceted care for hip and knee OA7,8. OAMPs have been operationally defined as models of evidence-based care that are implemented in real-world settings and include the following four components: 1) personalized OA care (tailored to the needs of people with OA), 2) a package of care with longitudinal reassessment and progression, 3) two or more components of the core non-surgical, non-pharmacological interventions (education, exercise and weight loss) and 4) optional evidence-based adjunctive treatments as required (e.g., assistive devices, psychological support)7. OAMPs have been implemented internationally and vary considerably in their target populations, health care settings, intervention content, and delivery processes8,9. Considering the variability and rapid emergence of new OAMPs, it is important to develop standards for describing and evaluating these programs. This will facilitate comparison of the effectiveness of different OAMPs and inform decisions about replication in different health care settings.
In 2018, the Joint Effort Initiative (JEI), comprised of researchers and clinicians and endorsed by Osteoarthritis Research Society International (OARSI), was established to develop standards for the delivery and evaluation of OAMPs. The JEI Outcome Measures Working Group, a group of international, interdisciplinary experts in OA research and clinical care, was specifically mobilized to develop a recommended set of outcome domains for OAMPs7. There have been prior efforts to develop sets of recommended outcome measures for OA10–12. However, they have all been developed for use in traditional clinical trials, which typically allow a more intensive set of measures than is feasible in within real-world clinical settings. In addition, prior outcome measurement standards focus on patient-level outcomes and rarely consider impact at the health system level or metrics important for implementation10–12. Further, a prior review highlighted the considerable variability in the domains included in the evaluation of various OAMPs8. Harmonization in the evaluation of OAMPs would be helpful for comparing these programs and building a strong evidence base, particularly as OAMPs are increasingly being implemented in real-world settings9,13–26. The JEI Outcome Measures Working Group therefore sought to develop recommended sets of core and optional domains, specifically focused on evaluating OAMPs and describing the programs, participants and providers when they are implemented in real-world clinical or community settings. This goal was achieved through then initial development of a comprehensive list of potential domains, followed by a 3-round international Delphi survey to achieve consensus.
Methods
Overview
The JEI Outcome Measures Working Group, which led the Delphi process, included 25 members from Australia, Denmark, UK, Netherlands, New Zealand, Norway, Sweden, USA including physiotherapists (JE, JQ, ATT, KB, RSH, STS, AA, MH, JHA, ME, CP, SS, NO), exercise scientists (EH, JB, KA), biostatisticians (EL, BC), rheumatologists (GL, SPY, AW), orthopedic surgeons (KK, PJ), a general practitioner (KM) and a sociologist (CS). The Working Group met to determine the scope of measures to be included, Delphi survey design, participants eligibility criteria and survey administration processes, all of which are detailed below. These processes were informed by prior efforts of the JEI to establish a core capability framework for health professionals to optimize care for people with OA27. While we did not specifically address or directly follow the OMERACT guidelines during our process, we did follow very similar processes to those outlined in the OMERACT Core Domain Set Development Workbook including assembly of the working group, developing a workplan, generating candidate domains and prioritizing domains. Study procedures were reviewed by the University of North Carolina at Chapel Hill under IRB # 21–0291 and deemed exempt from federal human subject research regulations. The survey was administered over three rounds between April 19, 2021, and April 13, 2022.
Initial domain selection
The Working Group determined that the Delphi survey would focus on general domains (e.g., pain, function) important to evaluate in the context of OAMPs, rather than specific measurement tools. As there are multiple validated tools available for health outcome measurement in musculoskeletal care, we opted for teams to choose the specific measure for any given domain that would be most relevant and feasible for their context. The Working Group identified 5 broad categories that were considered important for describing and evaluating OAMPs: Patient Impacts, Implementation Outcomes, Participant Characteristics, OAMP Characteristics (including delivery) and Clinician Characteristics. Within each of these 5 categories, we then developed an initial set of domains. The Working Group sought to be inclusive and identified potential domains based previous sets of OA outcome measures10–12, outcomes reported in prior OAMP evaluations8,9, sets of quality indicators and implementation outcomes28,29, and the expertise of Working Group members. We focused on hip and knee OA due to the high prevalence and impact of these conditions on pain and disability. There were 75 initial domains included in the Round 1 Delphi survey, ranging from 4 to 26 domains within each category.
Involvement of people with OA in survey development
Once the first draft of the Round 1 survey was compiled, the Working Group solicited input from people with OA. Two separate virtual discussions were held with 7 individuals (5 Americans, 2 Australians) identified by Working Group members as having an interest in contributing to the project. These individuals made recommendations related to additional domains for inclusion, simplifying survey languages, and providing definitions and explanations for some of the included domains. After these changes were incorporated into the survey, these individuals were given an opportunity to provide further feedback on the Round 1 survey via email before it was finalized.
Delphi participant inclusion criteria
Drawing from the JEI work on core capabilities for health professionals delivering OA care27, and considering the key groups that are typically involved in OAMP delivery, participation, implementation and decision-making, the Working Group identified 5 categories of experts to recruit to complete the Delphi survey. These groups, along with inclusion criteria for each, were as follows.
Academic researchers
Meet at least one of the following criteria 1) First or last author on at least two peer reviewed papers per year on original human research in OA in the past 5 years OR 2) Invited to give a plenary or keynote presentation on clinical OA research at an international conference in the past 5 years OR 3) Delivered an intervention or been responsible for collecting outcome measures on patients, in the context of original human OA research or a clinical OAMP, for at least 3 years AND co-author on at least 2 peer reviewed publications reporting original data from clinical human OA study(ies) in the past 3 years.
Healthcare professionals who treat patients with OA
Currently registered to practice as a healthcare provider in their home country and have managed at least one patient with OA per week over the past 6 months.
Clinician researchers
Currently registered to practice as a healthcare provider in their home country and have managed at least one patient with OA per week over the past 6 months AND meet at least one of the following criteria: 1) Have been first or last author on at least two peer reviewed papers per year on original human research in OA in the past 5 years OR 2) Invited to give a plenary or keynote presentation on clinical OA research at an international conference in the past 5 years OR 3) Delivered an intervention or been responsible for collecting outcome measures on patients, in the context of original human OA research or a clinical OAMP, for at least 3 years AND co-author on at least 2 peer reviewed publications reporting original data from clinical human OA study(ies) in the past 3 years.
People with OA and consumer/patient advocacy representatives
Have a clinician diagnosis of OA OR work for a consumer/patient OA advocacy organization.
Health administrators and policymakers
Have decision-making capacity regarding aspects of care management programs that are implemented in a clinical setting or health system.
Potential participants were excluded if they were not able to understand English well enough to respond to the survey or if they did not have sufficient computer or internet access to complete the survey.
Recruitment & enrollment
Participants were recruited via email and social media. A list of potential survey participants was prepared by the Working Group, drawing from their academic, research and clinical networks, as well as from authorship of published OA research. Contact email addresses were obtained from publicly available sources, such as publications and university or clinical practice websites. In addition, OARSI emailed a participation invitation to its member database. IRB-reviewed social media messages were posted by JEI Working Group and Steering Committee members, OARSI, the University of North Carolina Thurston Arthritis Research Center, and the Osteoarthritis Action Alliance via Facebook, Twitter, Instagram, and LinkedIn. OARSI also included information about the survey in their Weekly Briefing email. Members of the Working Group were eligible to complete the survey.
Email invitations and social media posts included a link to the Round 1 Delphi survey. Participants were first asked to read a description of the study and review the research information sheet. If they agreed to participate, they proceeded to the survey screening questions. In Round 1 of the survey, participants were invited to suggest other potentially qualified participants. Participants were also asked to provide their own email address so they could be contacted for future Delphi rounds. Participants were only asked to complete Rounds 2 and 3 if they completed the prior Round. Links for the surveys in Round 2 and Round 3 were sent directly to participants via email.
We aimed for a sample size of at least 20 participants in each of the 5 groups described above to complete the final round of the survey. Since there is no accepted method for determining sample size in Delphi surveys30, our sample size goal was based on reasonable representation from each group. Based on previous work27, we expected a response rate of approximately 70–75% for Rounds 2 and 3. To account for non-response, we aimed to enroll at least 36 participants in each of the five groups for Round 1.
Survey administration
The survey was administered using REDCap. Three Delphi rounds were conducted to reach consensus on the recommended domains. We planned for each round to be open for approximately 2 weeks but extended this time to meet sample size goals. Descriptions of each survey round are as follows.
Round 1
Participants were asked to rate each domain as “not important,” “somewhat important,” “important,” or “essential” for evaluating OAMPs (in real-world clinical settings), focused on hip and knee OA. An individual domain was retained for Round 2 if at least 80% of participants indicated that it was “important” or “essential”. Participants were also invited to suggest other domains that were not included in the initial survey. These suggested domains were included in Round 2 unless the majority opinion of the Working Group was to exclude them based on redundancy/duplication with other domains already in the survey or not aligning with the principles used to determine initial domains for the survey (e.g., relevance to OA and feasibility of assessment in the context of evaluating an OAMP).
Round 2
Participants were asked to rate their level of agreement/disagreement that each domain was essential for evaluating OAMPs (0 = strongly disagree to 10 = strongly agree). Panelists were provided with summary results from Round 1 for each domain that was included in that round. Individual domains were retained for Round 3 if at least 80% of participants rated the domain at least a 6 on the numeric rating scale.
Round 3
Participants were asked to reconsider and rate the domains retained from Round 2, using the same 0–10 numeric rating scale. Panelists were provided with summary results from Round 2 for each domain. Domains were considered “core” measures if ≥80% of participants rated it ≥9 and “optional recommended” if ≥80% rated it ≥7.
Delphi participant characteristics
All Delphi survey participants were asked questions regarding their gender identity, age, ethnicity and country where they were based, as well as whether they had been involved with an OAMP (e.g., as a participant, developing a program, delivering a program or referring to a program). Participants self-identifying as an academic researcher were asked whether they were also a qualified health care professional. Academic researchers and clinician researchers were asked how many years they had been researching OA and how many papers they had published on human OA research. Participants who identified as being health care professionals were asked to describe their discipline (e.g., physical therapist, physician) and number of years in clinical practice. People with OA were asked how many years ago they began experiencing symptoms and affected joint sites.
Statistical analysis
We first computed the number and percent (%) for Delphi Panel Participant Characteristics according to Round of survey. For Round 1, we computed the number and proportion of participants who rated the domain as either “important” or “essential”, using the total number of participants completing Round 1 as the denominator. For Round 2, we computed the number and percent (using the total number of respondents at Round 2 as the denominator) of participants scoring each domain as ≥6 on a 10-point scale. Finally, in Round 3 we computed the number and percent (using total number of respondents at Round 3 as the denominator) of participants scoring a domain as either ≥7 or ≥9. Statistical analyses were performed using SAS version 9.4 (SAS Institute Inc, Cary, NC, USA).
Results
Participants & response rates
A total of 178 participants completed Round 1 of the survey (Table I). We reached or exceeded our aim of n = 36 participants in all categories except for Health Administrators/Policy Makers, for which there were only 5 participants despite concerted efforts by the Working Group to recruit individuals in this category. Participant characteristics by round are shown in Table I. Additional characteristics of participants in different categories are shown in Supplemental Table 1. Fig. 1 shows countries of participants based on the 5 participant categories.
Table I.
Delphi panel characteristics by round
| Characteristic | Round 1 |
Round 2 |
Round 3 |
|---|---|---|---|
| Number (%) | Number (%) | Number (%) | |
|
| |||
| Participant category | n = 178 | n = 107 | n = 85 |
| Academic Researcher | 47 (26.4%) | 39 (36.5%) | 36 (42.4%) |
| Health Professional in Clinical Practice | 52 (29.2%) | 22 (20.6%) | 15 (17.7%) |
| Clinician Researcher | 38 (24.4%) | 21 (19.6%) | 15 (17.7%) |
| Person with Osteoarthritis or Consumer/Patient Advocacy Representative | 36 (20.2%) | 22 (20.6%) | 18 (21.2%) |
| Health Administrator, Policy Maker or Planner | 5 (2.8%) | 3 (2.8%) | 1 (1.2%) |
| Sex | |||
| Female | 109 (61.6%) | 66 (62.3%) | 53 (63.1%) |
| Male | 68 (38.4%) | 40 (37.7%) | 31 (36.9%) |
| Age group | |||
| Under 30 years | 5 (2.8%) | 2 (1.9%) | 1 (1.2%) |
| 30–40 | 49 (27.7%) | 30 (28.3%) | 20 (23.8%) |
| 41–50 | 41 (23.2%) | 25 (23.6%) | 22 (26.2%) |
| 51–60 | 41 (23.2%) | 24 (22.6%) | 18 (21.4%) |
| 61–70 | 30(17.0%) | 17 (16.0%) | 16(19.1%) |
| 71+ | 11 (6.2%) | 8 (7.6%) | 7 (8.3%) |
| Involved in any way with an OAMP | 131 (73.6%) | 85 (79.4%) | 71 (83.5%) |
| Participant country | |||
| Australia | 47 (26.4%) | 28 (26.2%) | 23 (27.1%) |
| Bhutan | 1 (0.6%) | 1 (0.9%) | – |
| Brazil | 2 (1.1%) | 1 (0.9%) | 1 (1.2%) |
| Canada | 6 (3.4%) | 4 (3.7%) | 3 (3.5%) |
| China | 3 (1.7%) | 7 (6.5%) | – |
| Denmark | 8 (4.5%) | – | 4 (4.7%) |
| Egypt | 1 (0.6%) | – | – |
| Finland | 1 (0.6%) | – | – |
| France | 1 (0.6%) | 1 (0.9%) | 1 (1.2%) |
| Germany | 1 (0.6%) | – | – |
| Hungary | 2 (1.1%) | 2 (1.9%) | 2 (2.4%) |
| India | 4 (2.3%) | 2 (1.9%) | 1 (1.2%) |
| Indonesia | 2 (1.1%) | 2 (1.9%) | 1 (1.2%) |
| Ireland | 2 (1.1%) | 2 (1.9%) | 2 (2.4%) |
| Japan | 2 (1.1%) | 1 (0.9%) | 1 (1.2%) |
| Malaysia | 1 (0.6%) | – | – |
| Netherlands | 12 (6.7%) | 6 (5.6%) | 4 (4.7%) |
| New Zealand | 4 (2.3%) | 2 (1.9%) | 2 (2.4%) |
| Norway | 8 (4.5%) | 8 (7.5%) | 8 (9.4%) |
| South Africa | 1 (0.6%) | 1 (0.9%) | 1 (1.2%) |
| Spain | 2 (1.1%) | – | – |
| Sweden | 1 (0.6%) | – | – |
| Switzerland | 1 (0.6%) | 1 (0.9%) | 1 (1.2%) |
| Taiwan | 1 (0.6%) | – | – |
| United Kingdom | 14 (7.9%) | 11 (10.3%) | 11 (12.9%) |
| United States | 50 (28.1%) | 27 (25.2%) | 19 (22.4%) |
Fig. 1.
Participant country by category.
A total of 107 individuals completed Round 2 of the Delphi survey (60% of Round 1 participants). Round 2 response rates within each category were: Academic researchers: 83%, Health care professionals: 42%, Clinician researchers: 55%, People with OA and consumer advocacy representatives: 61%, Health administrators/policy makers: 60%. For Round 3, there were 85 participants (48% of Round 1 and 79% of Round 2 participants). Round 3 response rates within each category were: Academic researchers: 77%, Health care professionals: 29%, Clinician researchers: 39%, People with OA and consumer advocacy representatives: 50%, Health administrators/policy makers: 20%.
Round 1
Of the 75 domains included in the Round 1 survey, there were 25 (33%) for which at least 80% of participants indicated the domain was “important” or “essential.” Table II shows the proportions of all participants rating each domain as “important” or “essential,” and Supplemental Table 2 shows these proportions by the 5 participant categories. Round 1 participants suggested a total of 45 additional domains for inclusion in subsequent rounds. Following Working Group review (based on redundancy and alignment with the scope of the survey), 34 of these domains were included in the Round 2 survey. Table III indicates new domains suggested by participants during Round 1.
Table II.
Round 1 respondents rating items as important or essential
| N (%) | |
|---|---|
|
| |
| Patient impacts | |
| Ability to participate in daily activities | 170 (95.5%) |
| Pain interference | 169 (94.9%) |
| Quality of life | 165 (92.7%) |
| Pain severity | 164 (92.1%) |
| Hip/knee physical function | 162 (91.0%) |
| Patient’s overall assessment of their osteoarthritis symptoms | 159 (89.3%) |
| Adverse events related to the treatments included in the OAMP | 143 (80.3%) |
| Arthritis self-efficacy | 142 (79.8%) |
| Pain frequency | 141 (79.2%) |
| Knee/hip replacement surgery since beginning the program | 139(78.1%) |
| Proportion of patients who meet “response criteria” | 138 (77.5%) |
| Use of pain medications | 135 (75.8%) |
| Sleep quality | 129 (72.5%) |
| Pain coping | 127 (71.3%) |
| Patient acceptable symptom state | 126 (70.8%) |
| Specific strategies used to cope with or manage OA | 126 (70.8%) |
| Self-reported physical activity | 123 (69.1%) |
| Objectively measured physical activity | 121 (68.0%) |
| Depressive symptoms | 117 (65.7%) |
| Self-reported general health status | 117 (65.7%) |
| Pain catastrophizing | 113 (63.5%) |
| Anxiety | 109 (61.2%) |
| Joint stiffness | 106 (59.6%) |
| Sick leave due to hip/knee symptoms | 104 (58.4%) |
| Willingness to undergo knee/hip replacement surgery | 103 (57.9%) |
| Personal costs/financial impacts of OA | 96 (53.9%) |
| Implementation outcomes | |
| Satisfaction with the OAMP from patients who participated | 165 (92.7%) |
| Long-term adherence to what was learned in the OAMP | 161 (90.4%) |
| Reasons patients do not complete the program | 159 (89.3%) |
| Reasons patients do not start the program | 153 (86.0%) |
| Cost effectiveness of the OAMP (e.g., how much the program costs per a certain amount of improvement in OA symptoms) | 147 (82.6%) |
| Number of OAMP sessions attended by patients | 143 (80.3%) |
| Budget impact analysis (expected overall expenses to a health care system if the OAMP is delivered) | 142 (79.8%) |
| Cost of the OAMP for the participant, if any | 140 (78.7%) |
| Reasons providers do not recommend patients to the OAMP | 137 (77.0%) |
| Average cost of the OAMP | 136 (76.4%) |
| Number or proportion of eligible patients who initiate the OAMP | 136 (76.4%) |
| Number or proportion of patients in the OAMP who receive specific aspects of OA care | 134 (75.3%) |
| Satisfaction with OAMP from clinic personnel or administrators | 131 (73.6%) |
| Number or proportion of clinics that implement the OAMP | 123 (69.1%) |
| Wait time for patients to start the OAMP | 120 (67.4%) |
| Number or proportion of eligible patients who initiate the OAMP as a first line intervention before prescription medications | 116(65.2%) |
| Number or proportion of eligible patients who initiate the OAMP based on a clinical diagnosis with use of imaging | 80 (44.9%) |
| Patient characteristics | |
| Joints affected by osteoarthritis | 168 (94.4%) |
| Assessment of overweight status | 167 (93.8%) |
| Most affected joint | 156 (87.6%) |
| Other long-term health conditions | 155 (87.1%) |
| Clinician diagnosis of osteoarthritis | 151 (84.8%) |
| Patient activation (patients’ willingness and ability to take independent actions to manage their health and care) | 146 (82.0%) |
| Age | 140 (78.7%) |
| Exercise tolerance/ability to participate in exercise | 137 (77.0%) |
| Previous joint replacement surgery | 135 (75.8%) |
| OA treatments received prior to the program | 131 (73.6%) |
| Duration of joint symptoms | 127 (71.3%) |
| Work status | 122 (68.5%) |
| Sex | 107 (60.1%) |
| Previous anterior cruciate ligament or meniscus injury | 106 (59.6%) |
| On wait list for knee or hip surgery? | 105 (59.0%) |
| Tobacco/smoking status | 98 (55.1%) |
| Education level | 92 (51.7%) |
| Residential/living status | 80 (44.9%) |
| Ethnicity | 78 (43.8%) |
| X-ray, MRI or other imaging evidence of osteoarthritis | 77 (43.3%) |
| Health insurance status | 77 (43.3%) |
| Financial status | 70 (39.3%) |
| Marital status | 38 (21.3%) |
| OAMP & delivery characteristics | |
| Mode of delivery of the OAMP (e.g., face-to-face, remote/virtual or a combination) | 151 (84.8%) |
| Type of practice (e.g., general/specialty, inpatient/outpatient) | 119(66.9%) |
| Public vs private facility | 103 (57.9%) |
| Geographic region/location | 101 (56.7%) |
| Clinician characteristics | |
| Clinician knowledge about recommended OA care | 168 (94.4%) |
| Clinician attitude toward OA treatment recommendations | 160 (89.9%) |
| Clinician confidence in delivering OA care | 156 (87.6%) |
| Types/disciplines of clinicians involved in program delivery | 135 (75.8%) |
| Patterns of OA care, obtained from electronic health records (referrals, imaging) | 117 (65.7%) |
Items are bolded if at least 80% of participants indicated that it was “important” or “essential”.
Table III.
Round 2 respondents rating items ≥6 (0–10 scale)
| N (%) | |
|---|---|
|
| |
| Patient impacts | |
| Ability to participate in daily activities | 107 (100%) |
| Quality of life | 106 (99.1%) |
| Hip/knee physical function | 105 (98.1%) |
| Pain severity | 104 (97.2%) |
| Pain interference | 104 (97.2%) |
| Adverse events related to the treatments included in the OAMP | 95 (88.8%) |
| Patient’s overall assessment of their osteoarthritis symptoms | 95 (88.8%) |
| Arthritis self-efficacy | 94 (87.9%) |
| Work-related limitations* | 87 (81.3%) |
| Fear of movement* | 83 (77.6%) |
| Misbeliefs about OA and its treatment* | 79 (73.8%) |
| Falls* | 75 (70.1%) |
| Use of walking aids* | 73 (68.2%) |
| Diet* | 68 (63.6%) |
| Pain sensitization* | 67 (62.6%) |
| Caregiving ability* | 66 (61.7%) |
| Surgical complications* | 59 (55.1%) |
| Implementation outcomes | |
| Long-term adherence to what was learned in the OAMP | 103 (95.4%) |
| Reasons patients do not complete the program | 100 (92.6%) |
| Satisfaction with the OAMP from patients who participated | 100 (92.6%) |
| Number of OAMP sessions attended by patients | 99 (91.7%) |
| Reasons patients do not start the program | 98 (90.7%) |
| Budget impact analysis (expected overall expenses to a health care system if the OAMP is delivered) | 94 (87.0%) |
| Cost effectiveness of the OAMP (e.g., how much the program costs per a certain amount of improvement in OA symptoms) | 94 (87.0%) |
| Patient characteristics | |
| Assessment of overweight status | 104 (96.3%) |
| Other long-term health conditions | 104 (96.3%) |
| Joints affected by osteoarthritis | 105 (97.2%) |
| Most affected joint | 99 (91.7%) |
| Clinician diagnosis of osteoarthritis | 97 (89.8%) |
| Patient activation (patients’ willingness and ability to take independent actions to manage their health and care) | 94 (87.0%) |
| Goals and expectations of outcomes of the OAMP* | 94 (87.0%) |
| Health literacy* | 87 (80.6%) |
| Social support* | 80 (74.1%) |
| Access to digital technology* | 73 (67.6%) |
| Occupation type* | 69 (63.9%) |
| Prior knee injury other than ACL/meniscus tear* | 66 (61.1%) |
| Need for interpreter services* | 66 (61.1%) |
| Caregiver responsibilities* | 64 (59.3%) |
| Travel distance to OAMP* | 61 (56.5%) |
| Hobbies* | 56 (51.9%) |
| Religion* | 32 (29.6%) |
| OAMP & delivery characteristics | |
| Mode of delivery of the OAMP (e.g., face-to-face, remote/virtual or a combination) | 101 (94.4%) |
| Types of exercises provided* | 98 (91.6%) |
| Group or individual sessions* | 90 (84.1%) |
| Frequency and duration of sessions* | 89 (83.2%) |
| Number of treatment components at each visit* | 84 (78.5%) |
| Peer support groups involvement in programs* | 82 (76.6%) |
| Patient involvement in the development of the program* | 81 (75.7%) |
| Referral Pathway/Source* | 77 (72.0%) |
| Health insurance coverage of the program* | 67 (62.6%) |
| Staff size* | 66 (61.7%) |
| Clinician characteristics | |
| Clinician knowledge about recommended OA care | 102 (95.3%) |
| Clinician attitude toward OA treatment recommendations | 100 (93.5%) |
| Clinician confidence in delivering OA care | 99 (92.5%) |
| Clinician training for delivering the OAMP* | 97 (90.7%) |
| Clinician knowledge about pain mechanisms* | 84 (78.5%) |
| Clinician training in health coaching or lifestyle medicine* | 78 (72.9%) |
| Clinicians’ years of experience in OA care* | 67 (62.6%) |
| Clinician languages spoken* | 63 (58.9%) |
Items are bolded if at least 80% of participants rated the domain at least a 6 on the numeric rating scale.
Indicates items suggested by Round 1 respondents.
Round 2
Of the 59 domains included in the Round 2 survey, there were 32 (54%) for which at least 80% of participants rated the domain at least a 6 on the 0–10 numeric rating scale. Table III shows the proportions of all participants rating domains at least a 6, and Supplemental Table 3 shows these proportions by the 5 participant categories.
Round 3
Of the 32 domains included in the Round 3 survey, there was 1 (1%) domain, “ability to participate in daily activities”, for which at least 80% of participants gave a rating of 9 or 10 on the 0–10 numeric rating scale; this domain is the only one meeting the threshold for a core outcome domain for evaluating OAMPs. In addition, there were 25 domains (78%) for which at least 80% of participants gave a rating of 7 or greater, meeting the threshold for optional recommended domains (Fig. 2). Table IV shows the proportions of all participants rating domains as 9–10 and as ≥7, and Supplemental Table 4 shows these proportions by the 5 participants categories.
Fig. 2.
Core & optional outcome domains for evaluating Osteoarthritis Management Programs.
Table IV.
Round 3 respondents rating items ≥7 and ≥9 (0–10 scale)
| N (%) Rating Item ≥7 | N (%) Rating Item ≥9 | |
|---|---|---|
|
| ||
| Patient impacts | ||
| Ability to participate in daily activities | 83 (97.7%) | 72 (84.7%) |
| Pain interference | 82 (96.5%) | 67 (78.8%) |
| Quality of life | 80 (94.1%) | 63 (74.1%) |
| Pain severity | 79 (92.9%) | 61 (71.8%) |
| Hip/knee physical function | 80 (94.1%) | 60 (70.6%) |
| Adverse events related to the treatments included in the OAMP | 73 (85.9%) | 40 (47.1%) |
| Arthritis self-efficacy | 78 (91.8%) | 32 (37.6%) |
| Patient’s overall assessment of their osteoarthritis symptoms | 74 (87.1%) | 32 (37.6%) |
| Work-related limitations | 71 (83.5%) | 31 (36.5%) |
| Implementation outcomes | ||
| Long-term adherence to what was learned in the OAMP | 78 (91.8%) | 55 (64.7%) |
| Number of OAMP sessions attended by patients | 74 (87.1%) | 44 (51.8%) |
| Reasons patients do not complete the program | 75 (88.2%) | 39 (45.9%) |
| Reasons patients do not start the program | 75 (88.2%) | 36 (42.4%) |
| Satisfaction with the OAMP from patients who participated | 70 (82.4%) | 33 (38.8%) |
| Cost effectiveness of the OAMP (e.g., how much the program costs per a certain amount of improvement in OA symptoms) | 65 (76.5%) | 26 (30.6%) |
| Budget impact analysis (expected overall expenses to a health care system if the OAMP is delivered) | 64 (75.3%) | 24 (28.2%) |
| Patient characteristics | ||
| Joints affected by osteoarthritis | 80 (94.1%) | 55 (64.7%) |
| Most affected joint | 77 (90.6%) | 50 (58.8%) |
| Assessment of overweight status | 80 (94.1%) | 48 (56.5%) |
| Other long-term health conditions | 80 (94.1%) | 43 (50.6%) |
| Clinician diagnosis of osteoarthritis | 66 (77.7%) | 32 (37.6%) |
| Patient activation (patients’ willingness and ability to take independent actions to manage their health and care) | 67 (78.8%) | 30 (35.3%) |
| Goals and expectations of outcomes of the OAMP | 75 (88.2%) | 28 (32.9%) |
| Health literacy | 58 (68.2%) | 20 (23.5%) |
| OAMP & delivery characteristics | ||
| Mode of delivery of the OAMP (e.g., face-to-face, remote/virtual or a combination) | 79 (92.9%) | 52 (61.2%) |
| Types of exercises provided | 71 (83.5%) | 44 (51.8%) |
| Group or individual sessions | 66 (77.7%) | 40 (47.1%) |
| Frequency and duration of sessions | 75 (88.2%) | 39 (45.9%) |
| Clinician characteristics | ||
| Clinician knowledge about recommended OA care | 81 (95.3%) | 48 (56.5%) |
| Clinician training for delivering the OAMP | 76 (89.4%) | 48 (56.5%) |
| Clinician attitude toward OA treatment recommendations | 75 (88.2%) | 42 (49.4%) |
| Clinician confidence in delivering OA care | 74 (87.1%) | 36 (42.4%) |
The gray shading indicates domains that were condiered “core” (≥80% of participants rated it ≥9) and “optional recommended” (≥80% rated it ≥7).
Discussion
This project aimed to establish consensus on core and optional recommended domains for describing and evaluating OAMPs. We approached this aim through engagement of an international Delphi survey including individuals with varied perspectives on these types of programs. The 3-round Delphi process yielded one core domain (a person-level outcome, ability to participate in daily activities) and 25 optional recommended outcome and descriptive domains that consider patient impacts (n = 8), implementation outcomes (n = 5), participant characteristics (n = 5), OAMP characteristics (n = 3) and clinician characteristics (n = 4).
Only one domain met our a priori criterion for a core domain. The threshold selected by the Working Group (at least 80% of participants rating the domain ≥9 on a 0–10 numeric rating scale) was based on prior Delphi methods27 and intended to identify the domain(s) with a high level of consensus that the domain is critical for evaluating OAMPs. It is noteworthy that among the large number of domains included in this survey, a person-level and person-centered outcome domain was viewed as the most important; this highlights the value placed by the survey participants on patient centeredness and improving the lives of people with OA based on their preferences, values, and needs. As noted above we only sought to identify domains to assess (what to measure in OAMPs) but did not seek to recommend specific measurement tools (that is how to measure outcomes in OAMPs); teams implementing OAMPs may select from among various measures within the relevant domains. This provides teams with flexibility in considering the specific measurement tool that best suits their needs and context. Groups evaluating OAMPs may assess the domain of “ability to participate in daily activities” using a single, broad item or through multi-item measures. One example of a brief measure within this outcome domain is the 4-item PROMIS Ability to Participate in Social Roles and Activities scale31, and there are other measures focusing on different aspects of participation that have been used in the context of rheumatic and musculoskeletal conditions32. We acknowledge that the concept of “daily activities” could include a range of activities, and our survey did not define this more specifically; Groups evaluating OAMPs may consider measures within this domain that capture activities of high relevance to patients and other stakeholders in their context.
Although only one domain met the criterion for a core domain, a relatively large set of 25 domains met the threshold for recommended optional domains. Our threshold for inclusion as a core domain may be considered relatively high. A prior Delphi survey from a JEI work group included items rated ≥7 by ≥80% of respondents in an overall set of core capabilities for health professionals caring for people with OA27. This was the threshold we used for optional domains. Because the domains in our optional set were highly ranked by many respondents, teams are encouraged to consider including domains from each of these categories to comprehensively evaluate their OAMP. We also note that although many domains did not meet our a priori criteria for core or optional recommended domains, this does not necessarily constitute a recommendation against their use. Indeed, some of these domains still achieved a fairly high level of consensus for importance and may be of particular interest to some stakeholder groups (see Supplemental tables). In particular, cost effectiveness of OAMPs and budget impact analyses are likely to be of high interest to health administrators and policy makers, though they did not meet our criteria for core or optional sets (potentially due to the small number of participants representing this group).
Strengths of this research include the international representation of the JEI Working Group and Delphi Panel, engagement with persons with lived experience of OA in the survey development process, inclusion of survey participants with different roles, perspectives and experiences with OAMPs, inclusion of a broad set of draft domains related to OAMP description and evaluation in Round 1, and provision for participants to suggest additional domains for the survey. There are also some limitations. First, we were unable to reach our target of enrolling 36 participants who identified as health administrators or policymakers. This is a critical perspective, and teams planning to implement OAMPs should engage with relevant health administrators and policy makers in their specific clinical setting early in the process to ensure fit and relevance of the program with the local context and priorities for program evaluation. Second, response rates to Rounds 2 and 3 of the survey were somewhat lower than expected. Our response rates were within the range of prior Delphi surveys, though these have varied considerably30. Response rates also differed across participant categories. Specifically, the response rate was highest for academic researchers and similar across health professionals, clinician researchers, and persons with OA/patient advocacy representatives. Among participants who identified as an academic researcher, most (87%) also reported being qualified health care professionals; therefore, clinical perspectives are also represented within this group. In Round 1, there generally was not wide variation across groups with respect to rankings of importance of domains. However, for some domains (e.g., arthritis self-efficacy, sleep quality) academic researchers had a lower proportion of respondents ranking the items as “important or essential.” Therefore, in later rounds, higher representation of academic researchers, compared to other groups, may have resulted in lower likelihood of some items reaching thresholds for inclusion. We have provided data on domain rankings by respondent categories (Supplemental Tables 2–4) so that teams selecting domains for an OAMP evaluation can consider these variations in respondent perspectives. A third limitation is that our Delphi panel had limited participation from individuals in low-income and lower-to-middle income countries, and non-Western regions, potentially because of language barriers. This may limit the generalizability of results for these countries/regions. In addition, many survey respondents, particularly patients and patient advocacy representatives, were from Australia and the US, which may also limit generalizability. Fourth, although participants included a wide range of health care professionals, the majority (70%) were physiotherapists or physical therapy assistants. This is likely due to the critical role of exercise and physiotherapy in the care of OA, as well as the high representation of these health care professionals in this area of research and implementation. However, we acknowledge the more limited representation of other health care professionals in this Panel.
In summary, this international Delphi study yielded recommendations for core (1) and optional (25) domains for evaluating OAMPs. The JEI Implementation Outcomes Working Group therefore recommends that these domains are prioritized, given the high level of consensus for their importance. The fairly large number of optional domains may limit harmonization in the evaluation of OAMPs since it may not be feasible to include all domains, and teams may select different domains. However, we expect the guidance provided here will encourage some overlap in the domains evaluated across OAMPs, therefore facilitating comparison.
Teams implementing OAMPs should also work closely with people with OA, clinicians, health administrators and others in their local context to develop an evaluation plan that considers questions of highest importance. Future work in this area should include consideration of low-income and middle-income countries, comparison of OAMPs based on these recommended outcomes, and further evaluation of the most important optional domains and best instruments for evaluating OAMPs.
Supplementary Material
Acknowledgments
The authors would like to thank the people with OA who contributed feedback during the Delphi survey development process: Denise Allen (US), Matt Burton (US), Bob Gardner (Australia), Loriann Moreno Munoz~ (US), Maria Rosa Papalia (Australia), Julieanne Pofahl (US), and Lou Wissner (US). We also thank the Osteoarthritis Research Society International for assistance with publicizing the Delphi survey. REDCap was supported by UL1TR002489 from the Clinical and Translational Science Award program of the Division of Research Resources, National Institutes of Health. Kelli Allen and Rebecca Cleveland receive support from the National Institute of Arthritis and Musculoskeletal and Skin Diseases Core Center for Clinical Research at the University of North Carolina, Chapel Hill (P30AR072580). Kelli Allen receives support from a VA Health Services Research and Development Research Career Scientist Award (19-332) and the Center of Innovation to Accelerate Discovery and Practice Transformation (ADAPT) (CIN 13-410) at the Durham VA Health Care System. Søren T. Skou is currently funded by a program grant from Region Zealand (Exercise First), and two grants from the European Union’s Horizon 2020 research and innovation program, one from the European Research Council (MOBILIZE, grant agreement No 801790) and the other under grant agreement No 945377 (ESCAPE). Kim Bennell is supported by grant funding from the National Health and Medical Research Council, Medical Research Futures Fund and Medibank Private. She has received personal fees from Wolters Kluwer outside the submitted work. J Haxby Abbott is supported in part by a Programme Grant from the Health Research Council of New Zealand (HRC 22/555). Rana Hinman is supported by a National Health & Medical Research Council Senior Research Fellowship (#1154217). Dr. Jayakumar is supported by The Musculoskeletal Institute, University of Texas at Austin. Jillian Eyles is supported by a Sydney Health Partners Research Translation Fellowship and receives royalties from Wolters Kluwer outside the submitted work. Emma Healey is partly funded by the NIHR ARC West Midlands. Saurab Sharma is supported by the International Association for the Study of Pain John J. Bonica Postdoctoral Fellowship. The views expressed in this paper are those of the author(s) and not necessarily those of the NHS, the NIHR, or the Department of Health and Social Care.
Footnotes
Competing interests
Søren Skou is co-founder of Good Life with Osteoarthritis in Denmark (GLA:D®), a not-for-profit initiative hosted at the University of Southern Denmark aimed at implementing clinical guidelines for osteoarthritis in clinical practice. Furthermore, he has received personal fees from Munksgaard and TrustMe-Ed, outside the submitted work. J. Haxby Abbott was subcontracted by Allan+Clarke Policy & Regulatory Specialists to contribute to the implementation evaluation of musculoskeletal programmes, and of is Director of GLA:D® Aotearoa New Zealand hosted at the University of Otago; he receives no personal fees related to OAMP implementation or evaluation. Allan Abbott is a steering committee member for the Better management of OsteoArthritis (BOA) national health care quality registry. Carin Pratt is a Musculoskeletal Coordinator of the Osteoarthritis Chronic Care Program (OACCP) which is an OAMP in NSW, Australia. Accredited GLA:D Australia Physiotherapist.
Supplementary data
Supplementary data to this article can be found online at https://doi.org/10.1016/j.joca.2023.02.078.
References
- 1.Hawker GA. Osteoarthritis is a serious disease. Clin Exp Rheumatol 2019;37(Suppl 120):3–6. [PubMed] [Google Scholar]
- 2.Abbate LM, Jeffreys AS, Coffman CJ, Schwartz TA, Arbeeva L, Callahan LF, et al. Demographic and clinical factors associated with nonsurgical osteoarthritis treatment among patients in outpatient clinics. Arthritis Care Res (Hoboken) 2018;70: 1141–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Osteras N, Jordan KP, Clausen B, Cordeiro C, Dziedzic K, Edwards J, et al. Self-reported quality care for knee osteoarthritis: comparisons across Denmark, Norway, Portugal and the UK. RMD Open 2015;1:e000136. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Allen KD, Golightly YM, White DK. Gaps in appropriate use of treatment strategies in osteoarthritis. Best Pract Res Clin Rheumatol 2017;31:746–59. [DOI] [PubMed] [Google Scholar]
- 5.Khoja SS, Almeida GJ, Freburger JK. Recommendation rates for physical therapy, lifestyle counseling, and pain medications for managing knee osteoarthritis in ambulatory care settings: a cross-sectional analysis of the National Ambulatory Care Survey (2007–2015). Arthritis Care Res (Hoboken) 2020;72: 184–92. [DOI] [PubMed] [Google Scholar]
- 6.Healey EL, Afolabi EK, Lewis M, Edwards JJ, Jordan KP, Finney A, et al. Uptake of the NICE osteoarthritis guidelines in primary care: a survey of older adults with joint pain. BMC Musculoskelet Disord 2018;19:295. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Eyles JP, Hunter DJ, Bennell KL, Dziedzic KS, Hinman RS, van der Esch M, et al. Priorities for the effective implementation of osteoarthritis management programs: an OARSI international consensus exercise. Osteoarthritis Cartilage 2019;27:1270–9. [DOI] [PubMed] [Google Scholar]
- 8.Allen KD, Choong PF, Davis AM, Dowsey MM, Dziedzic KS, Emery C, et al. Osteoarthritis: models for appropriate care across the disease continuum. Best Pract Res Clin Rheumatol 2016;30:503–35. [DOI] [PubMed] [Google Scholar]
- 9.Bowden JL, Hunter DJ, Deveza LA, Duong V, Dziedzic KS, Allen KD, et al. Core and adjunctive interventions for osteoarthritis: efficacy and models for implementation. Nat Rev Rheumatol 2020;16:434–47. [DOI] [PubMed] [Google Scholar]
- 10.Smith TO, Hawker GA, Hunter DJ, March LM, Boers M, Shea BJ, et al. The OMERACT-OARSI core domain set for measurement in clinical trials of hip and/or knee osteoarthritis. J Rheumatol 2019;46:981–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Dobson F, Hinman RS, Roos EM, Abbott JH, Stratford P, Davis AM, et al. OARSI recommended performance-based tests to assess physical function in people diagnosed with hip or knee osteoarthritis. Osteoarthritis Cartilage 2013;21:1042–52. [DOI] [PubMed] [Google Scholar]
- 12.Rolfson O, Wissig S, van Maasakkers L, Stowell C, Ackerman I, Ayers D, et al. Defining an international standard set of outcome measures for patients with hip or knee osteoarthritis: consensus of the International Consortium for Health Outcomes Measurement Hip and Knee Osteoarthritis Working Group. Arthritis Care Res (Hoboken) 2016;68:1631–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Nero H, Dahlberg J, Dahlberg LE. A 6-week web-based osteoarthritis treatment program: observational quasi-experimental study. J Med Internet Res 2017;19:e422. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Skou ST, Roos EM. Good Life with osteoArthritis in Denmark (GLA:D): evidence-based education and supervised neuromuscular exercise delivered by certified physiotherapists nationwide. BMC Musculoskelet Disord 2017;18:72. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Eyles JP, Mills K, Lucas BR, Williams MJ, Makovey J, Teoh L, et al. Can we predict those with OA who worsen following a chronic disease management program? Arthritis Care Res (Hoboken) 2016. Sep;68:1268–77. [DOI] [PubMed] [Google Scholar]
- 16.Eyles JP, Lucas BR, Patterson JA, Williams MJ, Weeks K, Fransen M, et al. Does clinical presentation predict response to a nonsurgical chronic disease management program for end-stage hip and knee osteoarthritis? J Rheumatol 2014;41: 2223–31. [DOI] [PubMed] [Google Scholar]
- 17.Malay MR, Lentz TA, O’Donnell J, Coles T, Mather Iii RC, Jiranek WA. Development of a comprehensive, nonsurgical joint health program for people with osteoarthritis: a case report. Phys Ther 2020;100:127–35. [DOI] [PubMed] [Google Scholar]
- 18.van der Esch M, van der Leeden M, Roorda LD, Lems WF, Dekker J. Predictors of self-reported knee instability among patients with knee osteoarthritis: results of the Amsterdam osteoarthritis cohort. Clin Rheumatol 2016;35:3007–13. [DOI] [PubMed] [Google Scholar]
- 19.Miller KA, Baier Manwell LM, Rabago D. Multimodal care for knee and hip osteoarthritis: a pilot feasibility study of a novel approach to a common problem. WMJ 2020;119:44–7. [PubMed] [Google Scholar]
- 20.Gwynne-Jones DP, Gray AR, Hutton LR, Stout KM, Abbott JH. Outcomes and factors influencing response to an individualized multidisciplinary chronic disease management program for hip and knee osteoarthritis. J Arthroplasty 2018;33: 2780–6. [DOI] [PubMed] [Google Scholar]
- 21.Gwynne-Jones DP, Hutton LR, Stout KM, Abbott JH. The joint clinic: managing excess demand for hip and knee osteoarthritis referrals using a new physiotherapy-led outpatient service. J Arthroplasty 2018;33:983–7. [DOI] [PubMed] [Google Scholar]
- 22.Roos EM, Gronne DT, Skou ST, Zywiel MG, McGlasson R, Barton CJ, et al. Immediate outcomes following the GLA:D(R) program in Denmark, Canada and Australia. A longitudinal analysis including 28,370 patients with symptomatic knee or hip osteoarthritis. Osteoarthritis Cartilage 2021;29:502–6. [DOI] [PubMed] [Google Scholar]
- 23.Osteras N, Moseng T, van Bodegom-Vos L, Dziedzic K, Mdala I, Natvig B, et al. Implementing a structured model for osteoarthritis care in primary healthcare: a stepped-wedge cluster-randomised trial. PLoS Med 2019;16, e1002949. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Abbott JH, Ward AL, Crane C, Chapple CM, Stout K, Hutton L, et al. Implementation of a ‘Joint Clinic’ to resolve unmet need for orthopaedic services in patients with hip and knee osteoarthritis: a program evaluation. BMC Musculoskelet Disord 2019;20:324. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Jonsson T, Eek F, Dell’Isola A, Dahlberg LE, Ekvall Hansson E. The better management of patients with osteoarthritis program: outcomes after evidence-based education and exercise delivered nationwide in Sweden. PLoS One 2019;14, e0222657. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Thorstensson CA, Garellick G, Rystedt H, Dahlberg LE. Better management of patients with osteoarthritis: development and nationwide implementation of an evidence-based supported osteoarthritis self-management programme. Musculoskeletal Care 2015;13:67–75. [DOI] [PubMed] [Google Scholar]
- 27.Hinman RS, Allen KD, Bennell KL, Berenbaum F, Betteridge N, Briggs AM, et al. Development of a core capability framework for qualified health professionals to optimise care for people with osteoarthritis: an OARSI initiative. Osteoarthritis Cartilage 2020;28:154–66. [DOI] [PubMed] [Google Scholar]
- 28.Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 2011;38: 65–76. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Basedow M, Esterman A. Assessing appropriateness of osteoarthritis care using quality indicators: a systematic review. J Eval Clin Pract 2015;21:782–9. [DOI] [PubMed] [Google Scholar]
- 30.Niederberger M, Spranger J. Delphi technique in health sciences: a map. Front Public Health 2020;8:457. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Hahn EA, DeWalt DA, Bode RK, Garcia SF, DeVellis RF, Correia H, et al. New English and Spanish social health measures will facilitate evaluating health determinants. Health Psychol 2014;33:490–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Bjork M, Bergstrom M, Sverker A, Brodin N. Measures of participation in persons with musculoskeletal conditions. Arthritis Care Res (Hoboken) 2020;72(Suppl 10):486–98. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.