Choosing interventions for hip or knee osteoarthritis - What matters to stakeholders? A mixed-methods study

Summary

Objective

To identify what factors drive choices among interventions for hip or knee osteoarthritis (OA), and to rank the characteristics of interventions.

Methods

In this phased, mixed-methods design involving cross-sectoral and multi-disciplinary stakeholders – healthcare consumers, providers, policy-makers, Māori health advocates and OA experts – we used the Nominal Group Technique in focus groups to generate data. We conducted thematic analysis of the focus group data to inform a framework of categories and sub-categories describing factors and characteristics influencing the choice of OA interventions. We then used a dual-panel, two-round e− Delphi survey to verify the framework and rank the characteristics of interventions.

Results

From six focus groups (n = 38 participants), 364 factors were identified and clustered into 56 themes (mean 9 themes per focus group; range 5–15). Thematic analysis revealed a framework of 3 core categories: characteristics of interventions (10 sub-categories), characteristics of consumers (10 sub-categories) and characteristics of the health system (7 sub-categories). In Delphi round 1, the framework was verified by each of two panels (n = 65, ≥80% acceptability). In round 2, two characteristics of interventions were combined, resulting in 9 characteristics (in decreasing order of importance): effectiveness, appropriateness, quality of the evidence, accessibility, harm, cost, duration, passivity, and immediacy of intervention effect.

Conclusion

Stakeholders make choices among interventions for hip or knee OA within a framework of characteristics of interventions, of consumers, and of the health system. We identified and ranked 9 key characteristics of interventions that stakeholders consider when choosing or recommending interventions for hip or knee OA.

1. Introduction

Clinical practice guidelines (CPGs) for the management of hip and knee osteoarthritis (OA) consistently recommend core management strategies of exercise, weight loss, education and self-management [1,2]. Yet the care received by people with OA is often at odds with these core strategies [3,4]. Successful implementation of interventions and health system strategies for OA are influenced by numerous factors, often involving multiple actors [5]. For example external context (e.g. health policy and infrastructure); organizational culture (e.g. culture and leadership); professional attributes (e.g. beliefs and attitudes to change) [5]; and consumers' and providers’ attitudes about OA and OA interventions [6,7].

Fundamental to an intervention's successful implementation is its ‘fit’ within a local health system (context), which depends on the compatibility of the characteristics of interventions with stakeholders' preferences for those characteristics [5]. The preferences and expectations [8] of health care consumers and providers about characteristics such as treatment effectiveness, cost and accessibility may impact the intervention's overall suitability within a particular context. For these reasons, characteristics of interventions must be considered when developing or implementing local health system strategies, such as Models of Care (MoC) [9]. Preferences for these characteristics are often studied at one level of the health system and are less focussed. For example ‘healthcare decision-making’ among clinical decision-makers and policy-makers [10], or separately among patients [11]. An important knowledge gap remains concerning the characteristics multi-level stakeholders consider, and their preferences for these characteristics.

The aims of this study were to: 1) identify the factors considered by relevant stakeholders, across a range of health care settings, when choosing or recommending OA interventions and express these in a conceptual framework; and 2) define and rank the main characteristics of interventions according to the relative importance placed on them by stakeholders.

2. METHODS

2.1. Design

In this two-phase, mixed-methods study (Fig. 1), phase 1 consisted of focus group discussions conducted using the Nominal Group Technique (NGT) [12]. The NGT data were pooled and thematically analysed to identify priority themes [12,13]. Phase 2 was a dual-panel, two-round, eDelphi study. Round 1 aimed to verify the framework and the focus of round 2 was to rank the importance of the characteristics of interventions. Our design included these validation processes: (i) independent parallel coding of NGT data with reduction into a composite framework to minimise researcher bias (ii) a dual-panel eDelphi which included an external and independent panel to verify the content validity of the Framework.

Fig. 1.

Summary of the study design.

The study was conducted in New Zealand (NZ) between November 2016 and August 2017. The methods and results for phase 1 are presented according to COREQ-32 [14] criteria for reporting qualitative studies (Supplement 1). Study phase 2 follows guidance on reporting and conducting Delphi studies (CREDES) [15] and the recommendations of Diamond and colleagues (2014) [16]. Ethical approval was obtained from the University of Otago Human Ethics Committee (D16-329).

2.2. Sampling and recruitment

2.2.1. Focus groups

We sampled 5 stakeholder groups: health care consumers, providers, policy-makers, Māori health advocates and OA experts. Convenience and snowball sampling generated lists of stakeholders, who were screened against inclusion criteria adapted from a previous qualitative study of cross-sectoral experts in musculoskeletal MoC [17] (Table 1). Purposive sampling of eligible stakeholders was used to maximise the diversity of participant perspectives, to reduce the risk of limited perspectives being represented within stakeholder groups. The Māori health advocates group were included as Māori experience overall disproportionately worse health outcomes relative to the non-Māori population in Aotearoa/NZ [18]. For each of the above stakeholder groups, focus groups with 4–8 participants were formed [19].

Table 1.

Inclusion criteria for the focus group discussions (phase 1).

Stakeholders	Health policy/strategy, advocate or health service/programme delivery in NZ	Osteoarthritis expert in clinical or health services research in NZ or Australia	Clinical practitioner	Consumerd	Māori health advocate
Inclusion criteria	•Experience in health service or programme delivery, coordination, management or funding related to musculoskeletal health care and/or chronic disease (that implicitly includes OA) for at least 1 year at a senior level in NZc •Experience at a seniora level in musculoskeletal and/or chronic disease (that implicitly includes OA) health policy or advocacy for at least 1 year in NZ •At least one participant reflecting each of the sub-categories below	•Awareness of guidelines for OA •At least 5 years experience in undertaking clinical and/or health services research in arthritis or health care-related implementation science at a senior levelb with evidence of peer-reviewed publication(s) in the area •At least one participant reflecting each of the sub-categories below	•Experience in delivery of clinical care for people with arthritic conditions at a senior practitioner levelb for at least 5 years in NZ •Awareness of guidelines for OA •At least one participant reflecting each of the sub-categories below	•Meets the NICE clinical criteria for diagnosis of OA (i.e.: 45 years or over; has activity-related joint pain, and either; has no morning joint related stiffness or morning stiffness lasting less than 30 min) •At least one participant reflecting each of the sub-categories below	•At least one participant reflecting each of the sub-categories below
Sub-categories	•Community services health policy or strategy decision-maker •Public health services health policy or strategy decision-maker (District Health Boards and Primary Health Organisations) •Assessment, treatment and rehabilitation services health policy or strategy decision-maker •National-level advocacy for arthritis health care •Health service management or coordination at secondary hospital setting •Health service management or coordination at primary care setting	•Model of care or clinical practice guideline development expert •Clinical epidemiologist in OA •Academia and education in OA	•General Practice •Physiotherapy •Rheumatology •Orthopaedic surgery •Community pharmacy	•At least 40% male •At least 40% female •National-level advocacy for arthritis health care	•At least one male •At least one female •Māori service provider

^aAt least senior officer or manager level of employment.

^bFellowship level for medical practitioners (e.g. FRNZCGP); senior clinician level for other disciplines; associate professor level for researchers.

^cAt least at the manager or head of department level.

^dSelf-reported diagnosis of OA or determined by the National Institute for Health and Clinical Excellence guideline recommendation for diagnosis of OA [36].

Stakeholders from professional, government and non-government organisations and health care provider agencies were invited to participate by telephone and email. Consumers were targeted using local newspaper advertisements and recruitment flyers at local community centres. Participants from across the public and private health sectors were sampled from a NZ business directory and an online health-service database (https://healthpages.co.nz/). OA experts were initially identified using an online database (http://expertscape.com/) and reviewed by two authors (JC and JHA) with regards to their eligibility. Because of the small population of OA experts in NZ, Australian OA experts were also included on the basis that beliefs about the management of OA should not differ between the two countries given similarities of the health systems in these neighbouring countries and consistency of international clinical practice guidelines [1,2].

2.2.2. eDelphi survey

The dual-panel eDelphi survey was conducted using Qualtrics (Provo, USA). Panel A included phase 1 participants. Panel B, convened to concurrently assess the content validity of the Framework, comprised participants satisfying the same inclusion criteria but who were unable to take part in the focus groups.

3. Protocols & data analysis

3.1. Focus group protocol (phase 1)

In phase 1, focus groups were facilitated using the mixed-method NGT. The NGT structures group discussion in response to a question that can then be prioritised through group consensus in real-time. Focus groups generated data comprising: factors (representing the factors affecting stakeholders’ choice of OA interventions), themes (factors grouped into themes, by participants), top 7 themes ranked by importance (by the participants), audio transcripts and field notes (hereinafter “NGT data”; see Fig. 1). From these data, thematic analysis resulted in a framework (hereinafter “the Framework”) of categories and sub-categories (term descriptors are seen in Fig. 2).

Fig. 2.

Description of terms used in this paper for factors, themes, categories and sub-categories.

Face-to-face and virtual (digital) focus groups were conducted for each stakeholder group separately using the NGT [12]. Each focus group was audio-recorded and transcribed verbatim for analysis. The question posed to participants to discuss was deliberately broad to capture all possible characteristics of interventions: “What factors do you consider when recommending or choosing one treatment option over another, for hip or knee OA?” Consumers were posed the question in the context of their own experience: ”What factors do you use to choose one treatment option over another for your OA?”

The first author (JC) was trained in the NGT by a researcher (CC), experienced in the method; CC facilitated the first focus group, assisted by JC, who facilitated all subsequent sessions. Two hours were allocated for each group, structured using five standard NGT steps [12]: (1) introduction and explanation (~5 min), (2) individual and silent generation of factors (15 min), (3) sharing factors identified in step 2, individually with no group discussion (20 min), (4) group discussion and participant-led clustering of factors into themes (40 min), and (5) voting and ranking of themes (20 min). These 5 steps are detailed in Supplement 2.

In anticipation of the focus groups generating potentially many themes, participants were asked to identify and rank their top 7 themes – on the basis that this number of characteristics is commonly used in health valuation studies employing multi-criteria decision analysis (MCDA) [20]. In the case of OA experts, participating in a virtual environment using a modified NGT, steps 1 and 2 were performed online using Qualtrics. Responses were collated and clustered offline by JC. For step 4, the preliminary clustered ideas were then presented to the OA experts, using Zoom videoconferencing (San Jose, USA) and RealtimeBoard (www.realtimeboard.com).

3.2. Qualitative data analysis

For the thematic analyses we used the abductive approach [21] which builds on the general inductive approach described by Thomas [22]. From this perspective we accepted a priori that the research team's primary frame of reference (evidence-based practice and evidence-based policy-making) [23,24] would influence the thematic analyses.

First, the NGT data generated from the focus groups were organised using the software package Atlas.ti (Ver.7, Berlin, 2015).

Second, JC performed a thematic analysis of the data following the steps described by Thomas (2006) [22], and guided by the advice of McMillan and colleagues (2014) to aggregate diverse and multiple-group NGT data [13]. The iterative process of open-coding raw data began with reading all results and transcripts from the consumer and health care provider focus groups, verbatim. Preliminary codes emerging from the text resulted in the creation of categories and sub-categories (hereinafter, “themes” and “factors” refer to patterns identified in the raw data; “categories” and “sub-categories” refer to the analysts' constructions of the data). The remaining focus group transcripts were coded using this abductively-derived coding framework, until a broad thematic framework of participants' considerations emerged. Next, the analysis focussed on identifying specific NGT data that were related to the overarching aim of the study. These data were open coded by JC until a primary thematic framework of stakeholders’ considerations emerged.

Thirdly, a thematic framework representing participants’ choice-making with respect to OA interventions was developed. Using the same analytical approach undertaken by JC, two researchers experienced in qualitative analysis (AMB, JHA) independently coded and analysed the transcribed NGT results and illustrative quotes to develop two independent frameworks. JC then mapped these two frameworks onto the primary framework to identify areas of concordance and discordance. In two meetings (arbitrated by PH) these 3 analysts reviewed the three frameworks and the results of the mapping exercise to agree on a consensus framework (hereinafter referred to as the Framework) consisting of categories and sub-categories.

Consistent with the a priori research question, i.e. to focus on the characteristics of interventions, the sub-categories describing the characteristics of interventions were further refined by consensus (JC, JHA, AMB) into discrete sub-categories that were generalisable, complete and non-redundant – desirable characteristics of criteria in MCDA [25,26].

3.3. eDelphi protocol (phase 2)

Phase 2 was a two-round, dual-panel online Delphi (eDelphi) survey to verify the Framework and to rank the sub-categories describing characteristics of interventions (Fig. 1). The protocol was based on a previously published process [9].

A two-round eDelphi was planned a priori, regardless of the overall level of agreement with the Framework after round 1. Two panels were planned for verification the Framework and validation of the results. Participants anonymously completed the survey and were eligible for a NZ$100 gift voucher upon completion of both rounds. Email reminders were sent after each round to increase the response rate.

The aims of round 1 were to: i) verify that no characteristics of interventions important to stakeholders were missing from the analysis (Panel A) and; ii) establish the content validity of the Framework (Panel B). Both panels provided ratings of agreement for the overall Framework (assessed using a 5-point Likert scale: strongly agree, agree, neutral, disagree, strongly disagree) and acceptability of the derived sub-categories (assessed using a 3-point Likert scale: completely agree, partially agree, disagree). Free text fields allowed participants to comment on any component of the Framework, and participants were prompted to explain their answer on any disagreed (disagree/strongly disagree) component. Participants were also asked to add any essential characteristics of interventions missing from the Framework. Demographic data were captured in round 1. Round 2 focussed on ranking the importance of the characteristics of interventions. Panels A and B were pooled for this quantitative analysis.

3.4. eDelphi survey data analysis

An a priori consensus for round 1 of the eDelphi was defined as ≥80% complete or partial agreement for the sub-categories (3-point Likert scale: completely agree, partially agree, disagree), and ≥80% agree or strongly agree for the overall framework (5-point Likert scale: strongly agree, agree, neither agree or disagree, disagree, strongly disagree). The qualitative aspect of round 1 comprised content-analysis of participants' free-text comments to identify new categories or sub-categories emerging from either panel. We performed further analysis of free-text comments across all the sub-categories rated as ‘disagree’ from >20% of respondents (per group) to clarify the descriptions of the characteristics of interventions carried into round 2.

The aim of round 2 was to rank the characteristics of interventions by importance. Consistent with the a priori research question, from this point on the eDelphi survey was limited to the characteristics of interventions; other categories and sub-categories of the Framework were not evaluated further. We calculated the average rank of the characteristics of interventions, from most-to least-important, within stakeholder groups and for the overall sample. Level of agreement was assessed using Kendall's W within and across the groups, where 0.1, 0.3, 0.5, 0.7 are very weak, weak, moderate and strong levels of agreement respectively [27]. To ensure the final set of characteristics was representative of all groups (which differed in size), the top two ranked characteristics for each group were included in the final aggregated list of characteristics, regardless of their overall ranking, providing that adequate within-group agreement was demonstrated (Kendall's W > 0.1, p < 0.05). Data were analysed using IBM SPSS Statistics (ver.24, Armonk, NY).

4. RESULTS

4.1. Focus group discussions

Six focus group meetings were held (N = 38, 61% female; n = 3–11 participants per group, median 6 [IQR 3.3]), each lasting approximately 90 min (range 60–120 min). Roles, experience and number participating in each stakeholder group are summarised in Table 2. Steps 2 and 3 of the NGT produced 364 factors. After the removal of duplicates, 258 unique factors emerged across focus group discussions (mean 43 per group; median 39; range 29–62). Step 4 produced a median of 9 (range 5–15) participant-clustered themes per group (total 56 overlapping themes). Step 5 produced themes ranked by importance which broadly overlapped across groups concerning consumer-related factors (e.g. clinical status and consumer preferences), themes related to efficacy and safety of intervention and health system factors (e.g. intervention accessibility and culturally appropriate practices). The OA expert group, conducted by the modified NGT that did not include step 5, produced themes echoed in the other groups. The results of the NGT exercise (the NGT data) are reported in Supplement 2, Table S1.

Table 2.

Demographic characteristics of the focus group (phase 1) and eDelphi survey (Phase 2) participants at rounds 1 and 2.

	Focus groups (Phase 1)			eDelphi Round 1 (Phase 2)						eDelphi Round 2
	(N = 38)			Panel Aa			Panel B (N = 31)			Pooledb
Stakeholder groupc	n (%), age ± SD (range) in years	Female n (%)	Years in current role ± SD (range)	n (%), age ± SD (range) in years	Female n (%)	Years in current role ± SD (range)	n (%), age ± SD (range) in years	Female n (%)	Years in current role ± SD (range)	n (%), age ± SD (range) in years	Female n (%)	Years in current role ± SD (range)
Consumers	11 (29), 69 ± 10 (50–88)	5 (13)	16 ± 16 (1–50)	9 (26), 69 ± 10 (50–88)	5 (15)	14 ± 13 (1–35)	7 (23), 67 ± 9 (50–75)	5 (16)	10 ± 8 (1–25)	15 (24), 68 ± 10 (50–88)	9 (15)	13 ± 11 (1–35)
Māori health advocatesd	5 (13), 37 ± 12 (25–57)	4 (11)	5 ± 4.4 (2–10)	5 (15), 37 ± 12 (25–57)	4 (12)	5 ± 4 (2–10)	-	-	-	5 (8), 37 ± 12 (25–57)	4 (6)	5 ± 4 (2–10)
Health care providerse  General practitioner  Physiotherapist  Orthopaedic surgeon  Pharmacist  Rheumatologist	10 (26), 49 ± 13 (33–67) 4 (40), 55 ± 11 (41–64) 5 (50), 46 ± 14 (33–67) – 1 (10), 39 –	5 (13) 2 (20) 2 (20) – 1 (10) –	22 ± 9 (11–30) 25 ± 7 (15–30) 21 ± 11 (11–30) – 15 –	9 (26), 49 ± 13 (33–67) 3 (33), 56 ± 13 (41–64) 5 (56), 46 ± 14 (33–67) – 1 (11), 39 –	4 (12) 1 (11) 2 (22) – 1 (11) –	21 ± 9 (11–30) 26 ± 8 (15–30) 21 ± 11 (11–30) – 15 –	14 (45), 49 ± 10 (30–66) 4 (29), 47 ± 9 (39–59) 4 (29), 44 ± 8 (30–56) 3 (21), 56 ± 10 (46–66) – 3 (21), 57 ± 4 (52–59)	5 (16) 3 (21) 1 (7) 0 (0) – 1 (7)	20 ± 10 (8–37) 18 ± 8 (12–30) 12 ± 4 (8–15) 24 ± 12 (15–37) – 30 ± 5 (25–35)	22 (36), 48 ± 11 (30–67) 8 (36), 53 ± 12 (39–67) 8 (36), 40 ± 8 (30–55) 2 (9), 51 ± 6 (46–55) 1 (5), 39 3 (14), 57 ± 4 (52–59)	9 (15) 4 (18) 3 (14) 0 (0) 1 (5) 1 (5)	20 ± 9 (8–35) 22 ± 9 (12–30) 15 ± 7 (8–30) 18 ± 4 (15–20) 15 30 ± 5 (25–35)
Health policy-makersf  Heath policy or programme developers  Health system funders  Health service delivery, coordinators or managers	7 (18), 53 ± 10 (34–62) 1 (14), 62 2 (29), 60 ± 1 (59–60) 4 (57), 48 ± 11 (34–59)	5 (13) 1 (14) 1 (14) 3 (57)	6 ± 7 (1–20) 3 ± 0 (0) 20 ± 0 (0) 3 ± 3 (1–7)	6 (18), 52 ± 11 (34–62) – – –	5 (15) – – –	8 ± 7 (1–20) – – –	4 (13), 53 ± 7 (44–59) – – –	4 (13) – – –	14 ± 11 (6–30)g – – –	9 (15), 53 ± 9 (34–62) – – –	9 (15) – – –	10 ± 10 (1–30) – – –
Osteoarthritis experts	5 (13), 50 ± 7 (41–60)h	4 (11)	12 ± 5 (8–15)	5 (13), 50 ± 7 (41–60)h	4 (12)	12 ± 3 (8–15)	6 (19), 52 ± 8 (40–60)h	3 (10)	16 ± 10 (5–34)	11 (17), 51 ± 7 (40–60)	7 (11)	14 ± 8 (5–34)

^aResponse rate of N = 34 (89%).

^bResponse rate of N = 62 (95%).

^cDefined by primary role.

^dTwo stakeholders were interviewed face-to-face.

^eTwo focus group meetings were conducted for the health care provider group.

^fSub-group level data was not collected for eDelphi survey.

^gp = 0.036 (Mann-Whitney U).

^hn = 4 Australians.

Initial independent thematic analysis by the researchers (JC, AMB, JHA) produced three frameworks, each with 3–10 categories, comprising 5–17 sub-categories. Two rounds of discussion leading to consensus resulted in a thematic framework of three over-arching categories: 1) characteristics of consumers (10 sub-categories), 2) characteristics of interventions (10 sub-categories), and 3) characteristics of health systems (7 sub-categories) (Fig. 3). A summary description of the Framework is presented in Table 3(sub-category descriptions and supporting illustrative quotes are in Supplement 2, Tables S2–S3).

Fig. 3.

Thematic representation of the three over-arching categories and sub-categories, derived from the 56 themes generated by the focus discussion, that stakeholders consider in their choice of hip or knee osteoarthritis intervention.

Table 3.

Summary descriptions of the framework of factors influencing stakeholders choice of OA intervention derived from the focus group discussions.

Key Category and summative description	Sub-categories	Illustrative quotes supporting each sub-category
Characteristics of the intervention Stakeholders gathered information about interventions to assess for suitability against a patient's circumstances or feasibility within a health system. The characteristics or attributes of interventions, applicable to any number of interventions, were used to compare and contrast between interventions. Key characteristics of interventions included: evidence for effectiveness and safety, consideration of the benefit and trade-off, cost-effectiveness (to the consumer), immediacy of treatment and access to treatment. Information about the intervention options were also gathered from health professionals, peer/family experience and advertising sources (e.g. newspaper and online advertising) in the decision to select a treatment option for consumers.	•Access to treatment for the consumer. The suitability of access considered in terms of an intervention's distance to nearest provider and wait time. •Active versus passive intervention options. The extent to which an intervention allows a patient to self-manage their condition over the long term, versus passive care. •Cost of the intervention. The financial costs of intervention relevant to the use or provision of health care and society. •Duration of the intervention effect. The ability to delay or avoid more invasive interventions later through improvements in quality of life (e.g. physical function). •Evidence about the intervention appropriateness. Providing the right treatment or services for OA at the right time (e.g. surgical intervention for early- or advanced-stage OA). •Evidence about the intervention effect. Considered on the basis of evidence outside a clinical trial, including short-term and long-term evidence of effectiveness and change in health system and societal costs. •Evidence about the intervention harms. The risk of short- and long-term side effects of intervention. •Immediacy of the intervention effect. The time delay between starting and experiencing the benefits of intervention. •Quality of the evidence (+views and opinions). The extent to which one could be confident that the effects of the treatment or service described were real. Views and opinions concern the value which different people assign to different sources of evidence. For example, peer or family advice and/or personal accounts were viewed as more trustworthy for some participants than health professionals' or advertisement (e.g. newspaper, Facebook) advice, claims or recommendations about the benefits or harms of intervention.	• “Availability; for example, I explored physiotherapy and exercise options but I live an hour or so out of town … to access services whether it be the public system, whether it be close to; in my case travel's a factor.” Consumer #6 •Capacity and confidence and motivation might be different; I might put that as a different ‘cause somebody might have the capacity and confidence, but they're not motivated to change … if you look at the goal-setting care plans, the way they're done, those sorts of things [factors related to self-management] would definitely be taken into consideration.” Osteoarthritis expert #1 • “A whole lot around cost; what the particular cost is. The difference to the lifetime cost system and also the patient's lifetime costs. The value for money. (Health Policy #5) •“I made a note about this just while conversation was going on earlier about surgery and last resorts because from my point of view I would much prefer surgery to a drug medication that goes on and on and on.” Consumer #7 •“I guess in a sense we might say that all the evidence or whatever we might come up with on paper says that this is the right treatment, but actually for certain groups of people in society it might be the wrong treatment because it might be unfeasible because they're rural or can't afford to travel. So therefore it's the wrong treatment for that person or even that group at that time.” Health Policy Maker #2 •“Well, efficacy, simply does the drug work; or does the treatment work? And with more complex interventions, say we've got multidisciplinary clinics, or whatever, then you'd need effectiveness which is showing that it works within a context.” Osteoarthritis expert #2 •“It doesn't matter whether it's a drug that's actually prescribed by the doctor or this Arthrem or any of those other things; you are putting something into your body and things like that. Is it going to be worth it? Is it not? You're weighing this up all the time.” Consumer #4 •“The problem is though you see on the advertisements and things like that and these people stand up and say, ‘This is the best thing since sliced bread,’ and you think ‘Right, okay, I'm going to give this a go.’ How much are they being paid to say that and in actual fact have they had that working for them; because you've got no way of checking. All it is, is you think “Well if it's worked for that person do I spend the money and give that a go,” but you have no knowledge of it.” Consumer #9
Characteristics of the consumer When choosing or recommending an intervention for OA, stakeholders considered it important to match an intervention to the biopsychosocial profile of the patient. Key factors included an assessment of a patient's severity of OA, clinical comorbidity profile, preferences (inclusive of attitudes, beliefs, ability to self-manage, motivation and treatment goals) and access to interventions. Social considerations included support to/from the family and/or community responsibilities.	•Clinical status of individual's osteoarthritis. The recommendation or selection of intervention based on objective tests, patient reported symptoms and age. •Presence of other conditions. For example, diabetes, heart disease and mental health, and the medical care being received for these conditions which may influence intervention choice. •Patient attitudes and beliefs. Intervention options need to reflect a consumer's (and their family's) beliefs about OA, as well as beliefs or expectations about the intervention options. •Capacity to self-manage. Consumer's health literacy, attitudes and beliefs about OA (e.g. perception of pain), and capacity to self-manage and navigate the health system. •Treatment goals. Treatment options need to relate to the immediate and longer-term functional goals of the consumer now and into the future. •Desperation (e.g. from pain). Once consumers felt that they had exhausted the therapeutic options immediately available to them to manage the symptoms of OA they sought to improve their quality of life by trying other interventions or medicines (e.g. herbal remedies) with poor evidence supporting their safety, benefit or interaction with other interventions. •Access to treatment. The consumer's geographic location and the number of treatment options available (to choose). •Affordability for the consumer. The immediate out of pocket expenses to the consumer. •Social support factors. Consideration of consumers' place in the community and wider community and support requirements if a disability is present. •Feasibility to the consumer. Whether or not a treatment option is acceptable and feasible to the consumer to adopt in the short and long term, based on the biopsychosocial circumstances of the consumer, their goals and expectations of treatment, and access to health services.	•“The trouble is there's the objective parts of what you find out about the patient and then there's the subjective part that they contribute which they talk about. So, to me those are the two separations and I would see them as being quite important to me. So, your x-rays, your scans, your blood tests and all that, I see them as being objective versus what the patient says and how incapacitated they say they are, all that sort of stuff which a subjective thing is.” Health care provider #1 •" … a couple of things that always worry me are whether they've got a chronic pain syndrome and whether they've got a chemical addiction already when you're considering what you're gonna do. Those two things influence me quite a bit with what I'm going to suggest that they might contemplate doing.” Health care provider #1 •“Desire. Like after all is said and done you can explain the benefits, present evidence based practice, and if they don't want to take it they just don't want to take it. They've got to want to; they've got to want the treatment. And someone might just want to die and not take the treatment; like I'm over this, I'm sick of being old. I have patients say that: ‘Don't get old dear, don't get old’ … or you might be like the 20th health professional to contact them about this and they just stop … They're just over it.” Māori advocate #1 •“I think health literacy, education and level of patient knowledge all relate to what the patient needs to bring to the treatment or treatment decision.” Health policy #2 •“I put how I'm feeling; it's time of the day sometimes. Like how I'm feeling; if I'm in a good space or something like that and I'm going somewhere and it's painful or whatever I might decide to not do something. I might decide not to get sort of very heavy about it. Or otherwise I may be having a bad day and I weigh up the pros and cons that way. It's how I'm feeling on the day.” Consumer #3 •“Being open for information too rather than just closing your mind off. I think you reach the stage where you will listen to anybody in the hope that something may work.” Consumer #7 •“Yeah so it's not so much about equity; it's about would somebody have to travel a big distance and that would be difficult for them. It's where the costs fall for the treatment option and which of them are most accessible for that person.” Health policy #2 •“The other thing too perhaps and I don't know quite how you put it in, but I'm still part time working and so my choices up to a point is how it can still allow me to work … Because if something's not going to allow me to continue what I'm doing then I wouldn't be interested in it?” Consumer #10 •“I would say family support to and of as well. Like for some Māori it depends on their beliefs and their upbringing but some people would choose a spokesperson; and so everything is dealt with that spokesperson that the patient has nominated as the spokesperson. So you wouldn't necessarily – and this is really weird as a health professional that you talk to someone other than the patient, ‘cause it's really the patient that you're dealing with and their experience. But to some Māori they do choose a spokesperson and so everything gets [9.10]. So let's say it's an older person; they might choose their youngest son, or they might choose the youngest daughter and that's the person that you deal with, with all cares concerning or all treatments concerning the patient. So you actually don't really deal with patient; which is different.” Māori advocate #1
Characteristics of the health system Participants considered whether there was the need for the society to invest in health care for OA and, whether the interventions considered were feasible to adopt or implement from the perspective of the health system and the consumer. Factors considered by stakeholders included the burden of disease profile for OA, socio-political interest, health system capacity, access and equity, culturally appropriate services and, the cost-effectiveness of intervention to society.	•Political landscape – is the disease a compelling problem? Before recommending interventions for OA, policy-makers considered if there was community and political interest in addressing the burden of OA. •Costs and benefits to society. Before recommending treatments for OA, policy-makers considered if there was community and political interest in addressing the burden of OA. •National health system capacity. Policy-makers also considered the capacity of a health system to support OA health care including policy direction, funding availability, evidence in support of funding or implementing a new service or intervention and workforce capacity and capability •Local health system capacity. Health policy makers considered the feasibility for a local health system to provide care services for OA in terms of the costs, communication capacity among care providers, the current availability of services (public and private), access to services and scope for integration with other existing services •Access and equity. Consideration of the burden imposed on the consumer to reach an intervention or service for OA was reasonable (e.g. geographical remoteness, fit with lifestyle) and, whether or not it was affordable to those who need it (fairness/equity of access) •Culturally appropriate services. Whether or not a health care service for OA was culturally appropriate for diverse communities. •Characteristics of the health service provider. The beliefs, experiences and preferences of the health service provider.	•“We've talked about the burden of disease but actually there's a far more less palatable political reality, which is about is there a burning platform? Is there some advocacy for this? Is this both accepted and publicly accepted as being needed? And it might be quite small. So I wonder if burden of disease is only one part of is there a burning platform or public support for this … Sometimes the burning platform is because of the cost to the system. Sometimes it's because of the consumer voice, from some effective advocates, and that's all captured in the how is this framed in the public's mind or little P politically. Because there's a political element to it which might just be local community; I don't mean national politics.” Health Policy #2 •“A whole lot around cost; what the particular cost is. The difference to the lifetime cost system and also the patient's lifetime costs. The value for money. Health Policy #5 •“… I think the question is what resources are available? What funding is available? Because you're likely to cut your service, cut your coat according to your cloth. You're likely to come up with a service that is feasible within the funding available and other resources available.” Health Policy #5 •" … it's actually I guess in a way saying more about the role of the District Health Board. [A DHB] plan for commissioned services in its own district, right? Once it might be a national policy or capability that we're going to provide these services in the country and then the DHBs end up providing those services in their own district according to what's available in their district to do that.” Health Policy #2 •“I wonder if accessibility at a system level becomes kind of fairness or equity; that at the same time as you want to know can the system deliver this, can you set up treatment options that will carry through, there's something about access and about fairness of access that you might want to pick up as well. (health policy #1) … Yeah … I just was thinking it's more than access; it's about fairness of access which I call equity, but I mean we might frame it differently.” Health policy #5 •“… educating the family is important also because that patient potentially is living with them, and so they will be offering the cares in some way, shape or form; so having the family, the patient and the whole team, the MDT, all on the same page. Like Māori don't tend to – this could be generalising but in my experience Māori don't put their loved ones into a rest home; so they're at home living with the younger daughter or the niece or somebody. So having those people involved in all aspects is really helpful ‘cause that will help the patient overall.“ Māori Advocate #2

4.2. eDelphi survey

Delphi round 1 included 65 participants (4 phase 1 participants lost to follow-up in Panel A, N = 34; Panel B, N = 31). In round 2, 3 participants were lost to follow-up due to personal circumstances (95% retention rate). The panels’ demographic characteristics are summarised in Table 2.

4.3. eDelphi round 1 outcomes

The overall acceptability of all the categories of the Framework was confirmed by panels A (91% strongly agree or agree) and B (94% strongly agree or agree) and by the stakeholder groups (Supplement 3, Tables S1–S2). Analysis of pooled panel and stakeholder-group agreement scores for each sub-category revealed consensus across all sub-categories, except for the “desperation” sub-category (77% partial or complete agreement), confirming the content of 26 of the 27 sub-categories (Supplement 3, Table S3). Analysis of agreement within stakeholder groups identified lack of agreement for the sub-categories (Supplement 3, Table S4): [n (%) stakeholder group] 5 (45%) OA experts for “desperation”, 3 (30%) policy-makers for “views relating to the characteristics of the intervention” and 5 (22%) providers for “local health system capacity”.

Analysis of stakeholder's free-text comments to the overall framework did not reveal any new categories or sub-categories. The three sub-categories with >20% disagreement are discussed here. Content analysis of free text comments from 4 of the 5 OA experts (1 did not leave a comment) who disagreed with the “desperation” sub-category posited that consumers may be desperate to try new treatments but did not consider it good practice; i.e. they agreed with the meaning of the sub-category per se. Likewise, the comments from three provider's who disagreed with the “local health system capacity” sub-category (2 did not leave comments) concerned local health system barriers, rather than disagreement with the sub-category. One policy-maker commented about their disagreement for the “views relating to the characteristics of the intervention” sub-category, however it was unrelated.

In summary, the eDelphi panellists agreement with the overall Framework and it's sub-categories (excepting the “desperation” sub-category) indicate strong alignment between the stakeholder-generated synthesis and the qualitative analysis, with validation provided by an independent stakeholder panel. Within the “characteristics of interventions” category, consensus was not reached by the stakeholder groups about the “views relating to the characteristics of the interventions” sub-category. Two participants' comments guided the analysis team to conclude that these views related to the evidence concerning interventions – from a range of sources – therefore these two sub-categories could be merged within the “quality of the evidence” sub-category for round 2. Thus, apart from this one change, no other modifications were made to the Framework, confirming its completeness and content validity, and leaving nine discrete characteristics of interventions (sub-categories) to be ranked by participants.

4.4. eDelphi round 2 outcomes

There was significant but weak agreement in the rankings across the groups (N = 62; between-group W = 0.333; p=<0.00). A weak-to-moderate level of agreement in rankings was observed within stakeholder groups except for the Māori health advocacy group (N = 5), which did not demonstrate adequate within-group agreement (Kendall's W = 0.089, ‘very weak’; p = 0.893). The top two ranked characteristics of interventions across the resulting stakeholder groups were: evidence about the intervention effectiveness, evidence about the intervention appropriateness, quality of the evidence, and immediacy of treatment effect. The Māori advocacy group was the only group to rank immediacy of treatment effect among their top two characteristics; however we were unable to achieve significant agreement within this group. Our failure to observe reliable estimates in this stakeholder group was likely due to the low sample size, despite our extensive efforts, so data from the Māori health advocate group were pooled with those of the healthcare consumer group. The aggregate ranking of the characteristics of interventions, across all groups, was (in decreasing order of importance): effectiveness, appropriateness, quality of the evidence, accessibility, harm, cost, duration, passivity and immediacy of effect (Table 4).

Table 4.

Characteristics of interventions after eDelphi round 2, in decreasing order of importance (top to bottom), for each stakeholder group. Data presented as the mean characteristic and rank by stakeholder group.

Stakeholder group (W; p-value)
Consumers	Māori health advocates	Health care providers	Health policy-makers	Osteoarthritis experts	Pooled
Effect (2.53)	Immediacy (3.80)	Effect (2.24)	Quality (2.33)	Effect (1.83)	Effect (2.40)
Appropriate (3.87)	Effect (4.00)	Appropriate (2.33)	Effect (2.44)	Quality (3.08)	Appropriate (3.18)
Access (4.27)	Appropriate (4.20)	Quality (4.05)	Appropriate (3.22)	Appropriate (3.33)	Quality (4.00)
Duration (4.87)	Access (5.20)	Access (4.14)	Cost (5.44)	Harms (4.08)	Access (4.71)
Quality (5.13)	AvP (5.20)	Harms (5.19)	Access (5.78)	Access (5.25)	Harms (5.34)
Immediacy (5.73)	Cost (5.20)	Cost (5.71)	Harms (5.89)	Cost (5.58)	Cost (5.65)
Cost (5.87)	Duration (5.60)	Duration (6.29)	Duration (6.00)	Duration (6.33)	Duration (5.85)
Harms (5.93)	Quality (5.60)	AvP (6.95)	AvP (6.67)	AvP (7.17)	AvP (6.77)
AvP (6.80)	Harms (6.20)	Immediacy (8.10)	Immediacy (7.22)	Immediacy (8.33)	Immediacy (7.10)
(0.222; 0.001)	(0.089; 0.893)	(0.533; < 0.00)	(0.452; < 0.00)	(0.588; < 0.00)	(0.333; < 0.00)

The mean rank for the characteristic is in parentheses (lower numbers represent greater importance). W= Kendall's W; AvP = Active vs Passive interventions; Effect = Evidence about the intervention effectiveness; Appropriate = Evidence about the intervention appropriateness; Access = Access to the intervention to the consumer; Quality = Quality of the evidence; Duration = Duration of the intervention effect; Immediacy = Immediacy of the intervention effect; Cost = Cost of the intervention; Harms = Evidence about the intervention harms.

5. Discussion

To the authors’ knowledge, this is the first study to investigate, using a cross-sectoral and multi-disciplinary sample, what stakeholders consider when choosing or recommending interventions for hip or knee OA. A Framework of three over-arching categories of factors considered by stakeholders were identified: characteristics of interventions, characteristics of consumers, and characteristics of health systems. We identified, and ranked by importance to stakeholders, nine characteristics of interventions.

The categories and sub-categories of the Framework developed here overlap with other conceptual frameworks [5,28], underlining the Framework's construct validity. A review [5] of the factors influencing implementation success in primary care identified themes, many of which meshed with our Framework, including: “external context”, “organisation”, “professional” and “characteristics of interventions”. Constructs of the Consolidated Framework for Implementation Research [28] also overlapped with our Framework: “intervention characteristics”, “outer setting”, “inner setting” and “characteristics of individuals”. Further, in a study of arthritis MoCs [29] the following concepts were also considered important: “patient self-management”, “provider skills and expertise”, “health care delivery”, “health system”, “community and public awareness” and “primary prevention”. While supporting existing (generic) frameworks [5,28,29], the present Framework provides important context-relevant information about factors relevant to the New Zealand health system for selecting and delivering OA care, which have not been established previously. Establishing context-relevant factors are important in informing implementation initiatives [9].

The nine characteristics of interventions of our Framework span clinical (appropriateness, effectiveness, harms and the quality of the evidence) and health system-related characteristics (accessibility and cost of the intervention) that align with the principles of contemporary MoCs for OA; i.e. “the right care, delivered at the right time, by the right team, in the right place and with the right resources” [9,17]. Alignment of these characteristics with stakeholder preferences is important to optimise the quality of OA care delivered within a health system. The description of the characteristics of interventions as discrete sub-categories that were generalisable, complete and non-redundant makes them particularly useful for informing MCDA methods, which go beyond rank-ordering (such as in this study) to more sophisticated approaches that explicitly evaluate the characteristics quantitatively [30], which may help decision-makers identify which interventions offer the greatest value to the health system and society or, in the case of consumers, the individual.

This research highlights a discordance between what influences stakeholders' choice of intervention relative to the evidence typically provided to guide decision-making (e.g. CPGs). Of the nine characteristics of interventions, ‘effectiveness’ and ‘quality of the evidence’ are routinely evaluated in the traditional evidence-based approach, and ‘appropriateness’ and ‘harms’ are often considered. The identification of these characteristics was, therefore, unsurprising. However, five other characteristics also arose, including: accessibility, cost and passivity of the intervention, immediacy and duration of the intervention effect. These additional characteristics are highly relevant to health consumers and providers, but typically lie outside the evidence-based paradigm and in healthcare provider and policy-makers’ considerations in practice [10]. The GRADE approach to clinical guideline formalisation does encourage guideline committees to also consider these contextual factors when making recommendations, however their consideration remains subjective [31]. These results may therefore carry implications for guideline developers including greater transparency in decision-making. Further, a principle aim of CPGs is to provide recommendations [1,2], which are an indicator of ‘appropriateness’ of the intervention for a given application. However these recommendations are generally formed on the basis of the characteristics of ‘effectiveness’, ‘quality of the evidence’, and ‘harms’, meaning they are non-independent of the characteristic ‘appropriateness’.

Matching stakeholders’ preferences for characteristics of interventions with the interventions provided and delivered in a health system may be fundamental to maximising uptake, in a context where resources are limited but many choices exist. In this study for example, immediacy of the intervention effect was considered more important by the consumer and Māori health advocacy groups, whereas the other (delivery sector) groups ranked it consistently last. These findings reinforce the importance of considering context-specific preferences in efforts to translate evidence into practice and policy [5,32] that may be particularly important in the current policy landscape with calls for a national OA Model of Care and programmatic funding for OA care from the Ministry of Health [33,34].

The Framework's categories reflect the themes generated by the focus groups. Many of the themes overlapped between groups and were not mutually exclusive, highlighting the complex nature of intervention choice. The Māori and health care provider groups, in particular, generated themes specifically about culturally appropriate practices and health services which incorporated “whānau” (family) and community needs (e.g. education about OA) that were not specifically raised by the policy-maker group. Briggs and colleagues [17] note that inclusion of family support factors into MoCs are poorly integrated, perhaps due to inadequate consumer participation in co-design. Due to low sample size, our results with respect to the Māori health advocate group's preferences for characteristics of interventions should be interpreted with caution, however these preliminary results suggest Māori have culturally moderated healthcare considerations which must be considered by healthcare providers in NZ to achieve successful implementation and reduce health disparities, and that providers recognise its importance. The NZ Ministry of Health's Māori Health Strategy [35] exemplifies active Māori participation in planning, development and delivery of appropriate health services for Māori people, reinforcing the benefits of inclusive policy-making.

Our study is not without limitations. Recruitment relied on convenience and snowball methods, which are prone to selection bias, but we filtered participants through purposive sampling which, while intended to enhance diversity, may have inadvertently biased our results towards the views and opinions held by the authors. Moreover, the sample size for each stakeholder group was modest, which may have constrained the breadth of considerations reported in this study when choosing or recommending OA interventions, and the ranked importance of the characteristics of interventions. The necessity to use a modified NGT process for the OA Expert group, conducted remotely due to their geographic dispersion, limited comparison of the NGT results with the other groups. Analysis of NGT data from multiple groups that vary in sample size and produce diverse themes is complex [13], however we included several verification and validation processes in our design to ensure validity, robustness, and generalisability of these results for the NZ population.

Intervention choice is complex and influenced by the characteristics of interventions, consumers and the health system. This finding has implications for developers of multi-level care strategies, such as MoCs. Currently NZ has no such strategy for managing OA, making the reporting of these context specific characteristics timely [33]. Future research could incorporate the nine discrete characteristics of interventions identified in the current study into MCDA, to support complex decision-making, such as those faced by guideline developers.

Author contributions

JC, AMB, PH and JHA contributed to the conception, design and primary analysis of data in the study. JC and CC contributed to preparation and data collection. JC had full access to the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. All the authors contributed to revisions of the manuscript, the interpretation of the data and critically reviewed the manuscript for publication. JHA conceived of the programme of enquiry, and is the corresponding author for this manuscript. All authors read and approved the final manuscript.

Role of the funding source

The funding source had no role in the design or conduct of the study; collection, analysis, or interpretation of the data; or writing of the report.

Declaration of Competing Interest

AMB declares travel support funding from the University of Otago <$10,000. AMB is supported by a fellowship awarded by the Australian National Health and Medical Research Council (113548).

Appendix A. Supplementary data

The following are the Supplementary data to this article: