Strategies for developing and implementing a rheumatoid arthritis healthcare quality framework: a thematic analysis of perspectives from arthritis stakeholders

Abstract

Objectives

To obtain stakeholder perspectives to inform the development and implementation of a rheumatoid arthritis (RA) healthcare quality measurement framework.

Design

Qualitative study using thematic analysis of focus groups and interviews.

Setting

Arthritis stakeholders from across Canada including healthcare providers, persons living with RA, clinic managers and policy leaders were recruited for the focus groups and interviews.

Participants

Fifty-four stakeholders from nine provinces.

Interventions

Qualitative researchers led each focus group/interview using a semistructured guide; the digitally recorded data were transcribed verbatim. Two teams of two coders independently analysed the transcripts using thematic analysis.

Results

Perspectives on the use of different types of measurement frameworks in healthcare were obtained. In particular, stakeholders advocated for the use of existing healthcare frameworks over frameworks developed in the business world and adapted for healthcare. Persons living with RA were less familiar with specific measurement frameworks, however, they had used existing online public forums for rating their experience and quality of healthcare provided. They viewed a standardised framework as potentially useful for assisting with monitoring the care provided to them individually. Nine guiding principles for framework development and 13 measurement themes were identified. Perceived barriers identified included access to data and concerns about how measures in the framework were developed and used. Effective approaches to framework implementation included having sound knowledge translation strategies and involving stakeholders throughout the measurement development and reporting process. Clinical models of care and health policies conducive to outcome measurement were highlighted as drivers of successful measurement initiatives.

Conclusion

These important perspectives will be used to inform a healthcare quality measurement framework for RA.

Strengths and limitations of this study.

• A strength of this study was the inclusion of a diverse group of arthritis stakeholders healthcare providers, persons living with rheumatoid arthritis, clinic managers and policy leaders.

• The results provide key insights into the use of quality frameworks in healthcare and helped to inform the development of a national quality framework for rheumatoid arthritis in Canada.

• The main limitation of this study is that patients were recruited through advocacy organisations and views may not have been representative of all patients.

• Demographic data were also not collected on participants to minimise any identifiable information.

Introduction

A variety of frameworks have been developed to assist with conceptualising and structuring the measurement of the quality of healthcare, the most commonly used or adapted are the Institute of Medicine (IOM) framework,¹ Donabedian’s framework^{2 3} and the Triple or Quadruple Aim Frameworks.^{4 5} The IOM framework¹ defines six domains of healthcare quality (safety, effectiveness, patient-centredness, timeliness, efficiency and equity), Donabedian’s framework,^{2 3} outlining three essential types of measures capturing the structure, processes and outcomes of care.^{2 3} The triple aim framework from the Institute for Health Improvement includes: improving the patient experience of care, the health of populations and reducing the per capita cost of healthcare as central to optimising health systems.⁵ A fourth aim includes improving the experience of healthcare providers.⁶

Quality frameworks of various types are implemented in different countries and health systems and may be associated with incentivisation programmes. For example, in the UK the National Institute for Health and Care Excellence (NICE) publishes quality standards on ‘high-priority’ areas for quality improvement.⁷ The standards are linked to evidenced-based clinical guidelines and can be used to audit and improve care.⁸ Primary care practices in the UK can participate in quality improvement incentive programmes based on NICE indicators through the Quality Outcomes Framework.⁹ In the USA, a National Quality Strategy was published in 2011 built on the triple aim of providing ‘better, more affordable care for the individual and the community’.¹⁰ Currently in the USA physician performance on quality measures is linked to incentives for participating physicians.¹¹ Participation in electronic medical record (EMR)-based registries like the national Rheumatology Informatics System for Effectiveness (RISE) is one way rheumatologists in the USA can monitor quality of care while participating in incentive programmes to improve care delivery¹² and identify gaps in care.¹³ In Canada, the Canadian Quality and Patient Safety Framework for Health Services was recently developed and endorses five goals for safe, quality care including people-centred care, safe, accessible, appropriate and integrated care.¹⁴

In the early 1990’s Kaplan and Norton developed a performance management framework called a balanced scorecard (BSC).¹⁵ The BSC was originally used in the business world and aligns performance metrics and targets to address an organisation’s strategic objectives towards an overall vision. Classically the BSC framework consists of four interacting domains representing different stakeholder perspectives (customer, financial, internal business and innovation and learning).¹⁶ Many healthcare organisations have adapted the BSC framework for use in quality improvement.^17–19 An adaptation of the BSC methodology was used in our province to transform quality of care in hip and knee replacement with great success in a non-incentivised programme.^20–22 This prompted our investigation of the use of a BSC framework in other clinical areas.

Rheumatoid arthritis (RA) is the most common type of autoimmune inflammatory arthritis affecting up to 1% of the population^23–25 and results from the international Global Burden of Disease Study suggest that RA prevalence may be rising.²⁶ In 2016, six performance measures for inflammatory arthritis care were developed to monitor access to care and appropriate treatment in Canada.²⁷ Testing the measures in various provinces revealed significant gaps in access and appropriate treatment at a population level.^{28 29} The impact of these findings on patient outcomes is unknown. Furthermore, current quality measurement programmes in Canada focus largely on acute care or primary care³⁰ and there is little to guide quality monitoring for complex chronic disease such as rheumatoid arthritis. Therefore, as part of a larger project to develop a more comprehensive quality of care framework in collaboration with the Arthritis Alliance of Canada (AAC) and the Canadian Rheumatology Association (CRA), the present study seeks to understand arthritis stakeholders’ perspectives on the structure and content of such a quality framework, while considering specifically the merits and challenges of a BSC framework.

Methods

Design

This project was part of a larger, nationally scoped, mixed-methods programme of research aimed at developing, testing and implementing a quality framework for RA. Here, we report on a subset of the findings focused on impressions of quality measurement frameworks, potential measures for inclusion and perspectives on framework implementation.

Participants, recruitment and sampling method

Stakeholders including healthcare providers HCPs (rheumatologists and allied health professionals, AHPs), people living with RA, clinic managers and policy leaders were recruited between December 2017 and June 2018. Participants were recruited through different means depending on stakeholder type: provincial rheumatology leaders (eg, presidents of regional rheumatology societies and/or rheumatology divisional heads) were asked to identify rheumatologists and clinic managers and regional policy leaders for recruitment. AHPs were identified by directors of publicly funded regional arthritis rehabilitation and therapy programmes and the national Arthritis Health Professions Association. People living with RA were recruited through advocacy organisations including the Canadian Arthritis Patient Alliance, Arthritis Patient Advisory Board, Arthritis Consumer experts and The Arthritis Society. We used purposive sampling to ensure representation of participants based on stakeholder type and region.

Focus group and interview objectives

The objective of the study was to obtain stakeholder perspectives on measurement frameworks in general, including their implementation, facilitators, barriers, benefits and risks as well as potential measures for inclusion in an RA quality framework. Participants were provided with Kaplan and Norton’s BSC framework^{15 16} and it was verbally explained. Participants were then asked to consider the relevance of this framework to RA care (online supplemental data S1).

Data collection

Focus groups and interviews

Separate focus groups were held with different stakeholder types. Three focus groups were conducted with 6–10 participants and telephone interviews with smaller interview groups (IGs) of between one and four people. Focus groups lasted about 90–120 min and interviews lasted about 45–90 min. A researcher with qualitative research skills (KT) and a research associate (MH) trained in qualitative methods cofacilitated all focus groups and interviews using focus group and semistructured interview guides adapted for each stakeholder type (see online supplemental material), these were digitally recorded, and transcribed verbatim.

Data analysis

Transcripts were anonymised and imported into NVivo V.12 for data management and analysis. A six-step thematic analysis^{31 32} was used. We took multiple steps to meet Lincoln and Guba’s³³ concept of trustworthiness including: dependability, credibility, confirmability and transferability. Two independent teams of two coders reviewed the decision-making record and made sure the process was logical. The coding teams met for peer debriefing and feedback on coding and analysis. To ensure dependability and confirmability, we used an audit trail to document decisions. Having two teams coding data independently helped control for biases. Credibility was established with frequent updates and discussion of findings to the CRA Quality Care Committee and the AAC during scheduled teleconferences. Transferability was accomplished from diverse purposive sampling of participants. As well, many team members were trained in multiple disciplines (quality of care, rheumatology, nursing, qualitative methods), which provided balanced perspectives.

Patient and public involvement

Two patients (KE and KT) were partners in all aspect of the study from development of the research question, securing funding, review and interpretation of results and drafting the manuscript and are listed as coauthors on this study. Patient perspectives from consenting individuals living with rheumatoid arthritis were obtained as described above to inform our study results.

Ethical considerations

Ethical approval was granted by the University of Calgary Research Ethics Board (REB16-0556) and all participants provided written informed consent.

Results

Participants

In total, 54 stakeholders participated (12 from British Columbia, 17 Alberta, 2 Manitoba, 12 Ontario, 3 Quebec and 8 from the Atlantic Provinces). Three separate face-to-face focus groups were conducted: AHPs at a rheumatology clinic (n=10), AHPs (n=9) and patients (n=6) both conducted at a national rheumatology conference. Thirteen participants had individual interviews (2 face to face and the remainder by telephone) and 17 people participated in the smaller IGs of between two and four people of the same participant type based on scheduling availability all by teleconference (eight rheumatologists, seven patients with one patient participating in both a focus group and an interview, four clinic managers, nine healthcare administrators/policy leaders, two AHPs). An overview of our findings is shown in figure 1.

Figure 1.

Overview of identified themes to support development of a quality measurement framework for rheumatoid arthritis.

Frameworks for quality measurement

While there was recognition from some participants that a BSC framework might be broad enough to encompass important concepts relevant to RA care, there was concern that the language may not be appropriate for healthcare as the financial domain is prominent in this framework. Participants indicated that although measuring cost of care was important, policy leaders in particular commented that ‘value for money’ was a more important metric and all participants highlighted the overarching importance of patient outcomes.

The policy leaders and HCPs had the most experience with the BSC framework. However, they more commonly endorsed the IOM¹ and the Institute for Healthcare Improvement’s Triple/Quadruple Aim^{4 5} frameworks as being more ‘comprehensive’ for healthcare quality monitoring (IG 13). One advantage to considering a BSC approach, highlighted by policy leaders, was that it lends itself to seeing the relationship between measures and at times the ‘conflict’ between measures, so that it could be used to ‘balance’ potentially competing demands (ie, patient experience, outcomes, costs etc) (IG 16).

Many patient participants had difficulty conceptualising the nature of healthcare measurement frameworks. What was more familiar to patients were websites where patients publicly rated care from HCPs. Some envisioned the concept of a BSC rather like a scorecard for their individual care that could enhance communication between members of their care team to highlight areas in need of attention.

Guiding principles for framework development

As policy leaders were experienced with quality framework development and implementation, they represented the majority of perspectives on this topic. They elucidated key elements for framework development: patients should be at the centre of the framework, and all care should be patient centred, don’t ‘reinvent the wheel’ (leverage existing work), feasibility is important, compare ‘like with like’, develop long-term improvement goals, use a holistic chronic disease approach, ensure relevance to all stakeholders, the importance of patient outcomes and reporting the data in a way that is actionable. All nine elements are described in table 1 with selected quotations.

Table 1.

Guiding principles for quality framework development

Principle	Description and selected quotations
1. Patients should be at the centre of the framework	When developing a quality framework, patients should be the primary focus of the framework. ‘…the patient’s outcomes, the patient’s ease of movement through the system, the patient’s satisfaction, the patient’s ability to access, the patient’s support. To me it all comes back to the patient.’ (Healthcare Provider FG2) ‘I think your best approach is to start from that interaction between patients and clinicians and create things that are meaningful first at that level. And then seeing what aggregates up to help support different decisions and then if you need to be supplemented with other types of measures.’ (Policy Leader IG16)
2. Don’t reinvent the wheel	Participants highlighted the work they had done or were doing in quality measurement across Canada and the successes associated with this work and suggested looking at “what have other people done and see, so you’re not reinventing the wheel.” (Healthcare Provider IG19)
3. Feasibility is important	Selecting measures that are feasible to measure and are of high impact is key to framework development. ‘I think it’s to pick the things that are measurable and have the biggest bang for the buck you know and also have the best outcome for the patient’ (Healthcare Provider IG9)
4. Compare like with like	Some participants suggested that when developing a framework and measuring quality that differences in models of care and practice context or patient populations be considered such that results could be comparable. ‘So compare like with like, so that you don’t lose the message in trying to say the new way of doing things. It’s like the new way of doing things may be different for different environments. So measure Telemedicine against Telemedicine. Measure visiting rheumatology models against visiting rheumatology models.’ (Healthcare Provider IG9)
5. Develop long long-term improvement goals	Emphasizing not only short-term goals and objectives but using the framework to develop longer term objectives for improvement was highlighted: ‘then I think as long as you have that long outlook and that’s again where the framework is so helpful. It gives you some objective benchmarks.’ (Healthcare Provider IG9)
6. A holistic chronic disease approach	Policy leaders had a broader view of measurement and encouraged a more holistic chronic disease approach to framework development given an aging patient population, often with multiple chronic diseases. ‘We’re trying more and more to move away from the Department of Health’s perspective of disease specific and more to an approach where we look at multiple chronic diseases because (Province X) has an aging population and we have a lot of challenges with people having multiple chronic diseases. So we’re just in the process of kind of moving away from disease specific strategies, but that’s not to say that on the operational level in the regional health authorities they would be very focused on specific diseases if patients have just one diagnosis or if one diagnosis is the more predominant one if you will.’ (Policy Leader IG13)
7. Ensuring relevance to stakeholders	In development of quality frameworks, policymakers highlighted the importance of ensuring the measures were relevant and actionable to stakeholders. ‘I think implementing any sort of scorecard or tool, if people don’t see the value or see themselves in it in some way, they have issues with those types of things.’ (Policy Leader IG13) ‘We kind of measure what you need to manage and hopefully that’s what you’re choosing’ (Policy Leader IG16) ‘Sometimes there’s that disconnect between the hundred thousand level, which is the payer and when I use that word I think of government, and the clinical. I think we have to try and find that balance about that is very meaningful to the clinical level because that’s where the improvement happens’ (Policy Leader IG16)
8. The importance of patient outcomes	Policy makers emphasized the importance of valid links between process measures and patient outcomes when developing a quality framework and selecting indicators. ‘you don’t want to look at your scorecard and see we’re measuring a bunch of processes or structures where there’s no clear link to an outcome that we’re trying to move. Because that’s just busywork, right?’ (Policy Leader IG13)
9. Reporting the data in a way that is actionable	While aggregate system-level data is often needed at a policy level to ‘help support sort of the executive level understanding and decision-making’, policy leaders also suggested reporting data in such a way that it can be used directly by health care providers for quality improvement. ‘…for the purpose of putting the data into action is to make it as granular as it needs to be to be actioned.’ (Policy Leader IG13) ‘but I always said make the right thing easier to do. You know just make it seamless and it appears. I know there’s a whole lot of work goes into getting it ready, but basically I need it to appear to me, so that I can easily use it.’ (Policy Leader IG14)

FG, Focus Group; IG, Interview Group.

Candidate themes for quality measures

Some patients reported participating in clinical studies where questionnaires were collected, which they perceived as ‘good because then they’re gathering the information they need to support me…‘ (Patients IG2). In contrast, other patients experienced little information gathering at all ‘I have no questionnaires… there is not any broad-spectrum information gathering at all’ (Patients IG2). Similarly, some HCPs worked in clinics where data collection for research was common, while others perceived challenges in data collection. Table 2 summarises the types of data and potential measures that were highlighted by the participants together with illustrative quotations.

Table 2.

Candidate areas of measurement

Theme	Measures suggested by participants	Selected quotations
Access	Wait times Access to biologic/advanced therapies Access to multidisciplinary care	‘I think that from a system’s standpoint, I mean the things that are relatively low hanging fruit to measure that I think are important are wait times to see a rheumatologist. But again, with the asterisk and the proviso that I think that that only tells part of the story.’ (Healthcare Provider IG6)
Knowing the numbers	Numbers of patients, numbers and distribution of rheumatology workforce	‘ … the number of providers to the number of patients would be well-matched and the patients would be triaged appropriately, so that we’re using our resources as effectively and as timely as possible.’ (Healthcare Provider IG6) ‘I’d be wanting to know like what is the scope or magnitude of the issue? How many people are there with rheumatoid arthritis in my jurisdiction? What percentage of them require ongoing care like at a primary care level? Then I’d be looking at what percentage or how many of them require specialist care?’ (Policy Leader IG14) ‘I get daily reports on COPD and heart failure visits to the ED in the province every morning….but I can’t tell you anything about how many people on (Province X) have rheumatoid arthritis.’ (Policy Leader IG14) ‘We cannot even give you a reliable count of numbers of patients that truly have rheumatoid arthritis. This is as bad as it is.’ (Healthcare Provider IG4)
Healthcare utilisation and Cost	Costs associated with care Emergency visits and hospitalisations Specialist visits/access number of patients leaving the province for treatment	‘If the end goal is going to be patient-centered care for patients with rheumatoid arthritis, in order to provide that care there needs to be a financial analysis to either assess what is going on and find the gaps and address those gaps in order to improve care. I think that, that is a key one but it’s probably one of the hardest ones to figure out what indicators you’re going to use. Do you use direct patient cost? Do you use indirect patient costs? How do you do the analysis?’ (Healthcare Provider FG2) ‘Yeah. And it’s challenging because there are so many hidden costs that are very, very difficult to map out. But I think we’d be really interested in knowing more about that both locally and provincially.’ (Patient IG6) ‘You know it’s hard to quantify prevention and be it to the extent that we can meaningfully quantify economic benefit of intervening clinically earlier around a disease projector.’ (Policy Leader IG18)
Patient outcomes	Composite disease activity scores Joint counts Global scores Functional status (Health Assessment Questionnaire) Pain Fatigue	‘I mean I think we all use some variant of a combination of patient reported outcomes, physician global outcomes and then the objective measures of joint counts and acute phase reactants. And I think that’s our best and most unifying language’ (Healthcare Provider IG6) ‘I get it that’s the standard measurement but to me it seems so antiquated because it’s almost like stereotype that arthritis is going to affect the hands and I know it does for a lot of people but it doesn’t for me.’ (Patient FG1) ‘When we’re talking about our daily living challenges, could we please fill in what our challenging for us because maybe some of those things that are on the list are never going to be a challenge for me but I’ve got eight that are.’ (Patient FG1) ‘It’s not a great questionnaire’ (Healthcare Provider referring to the HAQ FG2) ‘Sometimes even though they (the doctors) fix the RA, the patient still feels crappy…. So it’s the attribution of all of the symptoms may not align between the rheumatologist and the patient and making sure those important items are for the patient’s point of view I think are important on the scorecard. I think largely its fatigue and pain.’ (Healthcare Provider IG1) ‘…but how many times have I ever looked at a fatigue scale? Never, unless I’m doing a research project. And in the realities of time I don’t know if that’s a really good way to measure that.’ (Healthcare Provider IG1) ‘Also for the paperwork I would love for them to cover a little a bit more on fatigue because sometimes I find that to be more debilitating (than stiffness). They are always like, “how’s your pain?” Well, I’m having trouble just feeding myself. That’s a little bit more concerning to me than if a joint is hurting.’ (Patient FG1) ‘Cognitive (abilities) maybe how it affects your social interactions because I know I’m doing better with my disease when I don’t have to turn down so many social things in the evening and how it affects my work. So if there were measures that were pertinent to work, fatigue, socialization’ (Patient FG1)
Mental and emotional health	No specific measures suggested	‘I think that your mental attitude and your mental health has a lot to do with how you cope with it and how well you manage your day. And so, I think that whenever you first get to a rheumatologist or to a clinic that should be something that should be one of the most important things aside from the pain that you’re working through. I think that mental health is critical for the well-being of the patient.’ (Patient IG2) ‘Because it’s another one of those sneaky symptoms that you don’t realize it’s happening until you all of a sudden wake up one day and realize you haven’t gone out of the house in two weeks. And all you can do there is sit there and cry’ (Patient IG11)
Adherence	Adherence to medications	‘…and the other thing we do that I think is good is like an adherence for medication screen.’ (Healthcare Provider IG1)
Patient experience with care	Questionnaire/survey	‘But I think a lot of us are talking a lot more about that patient experience and the healthcare provider satisfaction or ability to provide that. I would love to then figure out ways for those concepts to be measured in indicators.’ (Healthcare Provider FG2) ‘Off the top of my head I can’t think of any satisfaction questionnaires that I have been given by the practitioner themselves or sent as a follow- up for an appointment. I have for practitioners that I didn’t think was great. I have gone online and given them reviews.’ (Patient IG8)
Comorbidities	Cardiovascular screening Osteoporosis screening	The capacity to do such advanced screening and monitoring in routine clinics was described as “limited”. With one healthcare provider stating that ‘… we’re still missing the boat as far as managing comorbidity concerns.’ (Healthcare Provider IG17)
Radiograph and laboratory testing	Inflammatory markers other monitoring lab tests X-ray results	‘So the numbers that are very important to me are just testing to see my CRP, like how my inflammation is going.’ (Patient IG3)
Safety	Adverse event reporting Measuring transitions between care settings	‘And we haven’t too much on safety yet in usual practice. Some of us do report if there’s been an adverse event associated with some of the drugs that we use. But that’s voluntary and not everybody does it.’ (Healthcare Provider IG17) ‘it’s a key area that we’ve been thinking about at a system level is having better measures of transitions when patients are crossing settings of care. The reason is that for safety reasons, for experience reasons, for effectiveness reasons, that that’s where we see probably the biggest room for improvement.’ (Policy Leader IG13).
Patient workforce participation	Workforce productivity and participation	‘Work productivity is something that I would love to have on a balanced scorecard. I think you mentioned that at the beginning but we’ve got a lot of patients who are working and we want to keep them there. I think we do because we’re treating them earlier. We’re treating them better but it would be nice to capture that.’ (Healthcare Provider FG3)
Healthcare provider satisfaction	Survey/questionnaire	‘Because if they’re overburdened anymore then it’s not going to work either. And we’ve got to understand the system has limited resources.’ (Patient FG1) ‘I think other staff opinions too are important to collect. We could do that for the same reasons. You know they may have another suggestion they haven’t brought forward yet. They may have a concern and they haven’t mentioned it, so we haven’t had the chance to explain why we haven’t done something or people are not able to do something.’ (Manager IG7)
Healthcare inefficiency	Rates of ‘no-shows’ for clinic appointments Waitlist management	‘If we could post even what no show appointments are doing to that waitlist when patients are complaining they can’t get in. If they’re aware of what that does to the system.’ (Healthcare Provider FG3) ‘…there’s a hundred inefficiencies that patients and HCPs come across within institutions and the communities. Things are being duplicated. People are on multiple waitlists. There’s all kinds of messy things that are happening. How can you capture the inefficiencies within the system?’ (Healthcare Provider FG2)

COPD, Chronic Obstructive Pulmonary Disease; ED, Emergency Department; FG, Focus Group; HAQ, Health Assessment Questionnaire; HCP, Health Care Provider; IG, Interview Group.

Access

Measuring wait times was felt to be an important system-level measure of care quality by all participant types. Time to access a rheumatologist was felt to be important as well as time to treatment. Access to medications, in particular to advanced therapies, was highlighted as important by patients. In addition, the work by Arthritis Consumer Experts, a national organisation providing free evidence-based information and educational programming for patients with arthritis, was mentioned as central in tracking access to advanced therapy nationally through their Arthritis Medications Report Card,³⁴ which provides provincial ratings of access to advanced rheumatology therapies. Access to multidisciplinary HCPs was also a major theme (discussed further below).

Knowing the numbers

In relation to healthcare planning and advocacy, the participants indicated that it was important to understand the distribution of the rheumatology workforce and the numbers of patients requiring care as these elements influence access to care, although challenges in tracking these elements were frequent. For example, some policy leaders reported that they were unaware of the numbers of patients with RA in their province/region and had no processes in place to access such information efficiently. HCPs in different provinces also commented on the inability to define the RA population, usually due to inefficient or non-existent systems for capturing patient diagnosis.

Healthcare utilisation and costs

Costs associated with care were important for all participants; however, they indicated that cost was a potentially difficult measure to address, especially in relation to cost savings with early and improved treatments. Policy leaders described having ‘a good grasp of the numbers in terms of dollars and cents’ (Policy Leader IG18); however, they highlighted the ongoing challenges of balancing financial and clinical outcomes. Measures of healthcare utilisation included emergency visits, hospitalisations and healthcare provider access. Policy leaders and HCPs also discussed measuring the number of patients leaving the province for care in other jurisdictions as a cost that should be tracked.

Patient outcomes

Patients and HCPs described a variety of patient-reported outcomes and composite disease activity scores that are used in rheumatology. Tender and swollen joint counts, patient global rating and composite disease activity scores including the DAS28 were commonly reported as useful for measuring outcomes as part of treat to target strategies. As computer calculated disease activity scores are generally required, the feasibility of routine disease activity calculation was limited for some providers. These disease-specific measures were highlighted as important for drug reimbursement.

The Health Assessment Questionnaire (HAQ)³⁵ is a measure of functional status routinely collected in rheumatology clinics with many HCPs and patients describing routine use. However, patients and providers reported some concerns with the HAQ as they felt it did not adequately reflect the patient’s burden and impact of disease. While quality of life measures (eg, 36-item Short Form Survey, SF-36³⁶) were mentioned by healthcare providers, it was only in context of data collection for research studies, suggesting lesser day to day clinical use.

Pain and fatigue were symptoms that patients felt were important to capture; however, providers questioned the relationship of these outcomes to the care provided and also noted that improved disease activity did not always result in improved pain and fatigue. Furthermore, measures such as fatigue were uncommonly used outside of research projects and some HCPs were sceptical as to the utility of such measures especially given time constraints in practice. Cognitive functioning was another important clinical symptom that patients felt should be monitored. For example, a number of patients referred to ‘brain fog’ as a particularly debilitating symptom that had a significant impact on physical function and social interactions.

Mental and emotional health

Capturing the impact of disease on mental, emotional health and coping was of primary importance to patients. However, the application of specific tools for systematically monitoring mental health infrequently occur. Descriptions from a healthcare provider perspective provided insight that mental and emotional concerns are more often captured through patient–provider interactions than through screening surveys or specific measures.

Concordance with treatment plan

Concordance with treatment plan was discussed by HCPs as important, with some stating this was simply discussed with patients, while in other clinics formal questionnaires of treatment ‘adherence’ are used. Patients discussed the cost of medications as well as delays in getting treatments approved and funded which could contribute to undertreatment.

Patient experience with care

While all participant groups felt measuring patient experience with care was important, they reported it was not commonly measured in routine clinical care. When patient experiences were measured it generally was done as part of a research study or if mandated by the hospital. Patients reported that they were infrequently asked about their experience, but some did report their experience with providers using online public reviews.

Comorbidities

The burden of comorbidities in patients with RA including osteoporosis and cardiovascular disease was recognised by HCPs, with some reporting routinely screening and monitoring for cardiovascular risk factors and osteoporosis risk factors using specialised online forms in risk reduction clinics. The capacity to do such advanced screening and monitoring in routine clinics was described as ‘limited’ by other HCPs. Many HCPs agreed at a minimum there should be communication of the increased risk of comorbidities to the primary care provider and/or patient.

Radiographs and laboratory testing

While HCPs recognised that radiographs of joints have some utility (especially early on in the disease course when evaluating for erosive disease), formalised scoring systems were not used frequently in clinical practice and were deemed of limited clinical utility. Patients, however, viewed knowing their laboratory and radiograph test results, and communication of the results among their HCPs, as important in following disease progression.

Less frequently discussed themes: safety, workforce participation and healthcare provider satisfaction, healthcare inefficiencies

There were four less-frequently discussed themes including safety, patient workforce participation and healthcare provider satisfaction and healthcare inefficiencies. While safety was infrequently mentioned by participants as an area of measurement in rheumatology, it was noted by HCPs that adverse medication events are sometimes reported through a voluntary process. One healthcare provider routinely conducted a biologics survey for patients ‘…to make sure they know how to safely take them’ (Healthcare Provider IG10). Transitions when patients were moving between settings of care (eg, posthospital discharge) was mentioned as a possible safety concern by a policy leader who suggested measuring aspects of safe transitions, including evaluating the timely sharing of information and medication reconciliation.

HCPs suggested that important metrics to consider would be work productivity and workforce participation. Potentially, tracking the volume of patients on disability and those on provincial low-income support programmes was also suggested as potential measures worth tracking.

Formal measurement of healthcare provider satisfaction was also discussed infrequently. While a healthcare provider commented that both patient and provider satisfaction were important, that they were not necessarily of equal importance and had different meaning. One manager viewed staff opinions as important to collect as a means of addressing suggestions or concerns for clinic and workplace improvement.

One healthcare provider suggested it would be important to measure lost opportunities, such as patient cancelling/not appearing at appointments without being able to reschedule another patient to attend during the allotted time. Similarly, there are inefficiencies in the system when physicians cancel clinics and do not provide a suitable appointment replacement opportunity. The balance of scheduling new appointments over providing follow-up appointment care was discussed. Another healthcare provider commented on patients being on multiple rheumatologists’ waitlists as an example of inefficiency that could be tracked.

Challenges of measurement

A number of challenges for measurement and reporting were identified by HCPs (table 3). Concerns around data access, privacy, accuracy and data linkage were often described. In some centres, data are routinely collected through EMR for patient care, although this is not universal. Additional data collection solely for the purpose of quality improvement was viewed unfavourably, due to clinical demands. HCPs also raised concerns about how the measures would be used, and by whom, and whether there would be any ability to make changes if deficits were identified.

Table 3.

Perceived barriers to implementation of a measurement framework

Barrier	Selected quotations
Data availability, access, accuracy, linkage and privacy	‘How do you trend your patients when you have data living in a computer and living on paper? It’s like a two-tiered world’ (Healthcare Provider FG3). ‘In the academic centers, us and all of the others, we don’t have any electronic medical records.’ (Healthcare Provider IG7) ‘It’s really challenges because even within (Province X) the RA data for rheumatoid arthritis is in over eight or nine databases across the province. And a lot of it is still missing and a lot of the clinical kind of data that is needed to drive it and measure some of the outcome pieces.’ (Healthcare Provider IG16) ‘Territorial nature of physicians. They don’t like to share anything.’ (Patient FG1) ‘And like I did fill out a three-page form in order to be able to… communicate with me in that way and I could see that privacy and confidentiality would be a big challenge in terms of that (performance measure reporting).’ (Patient IG8)
Data collection (time and resource constraints)	‘If there is a requirement for the physician to fill in stuff beyond what they’re already doing. Like we’re already busy, busy, busy’ (Healthcare Provider IG1) ‘it’s a huge burden to check off all the boxes and fill in all the outcome measures and all of that kind of stuff and still give attention to the person in front of me.’ (Healthcare Provider FG2)
Concerns about how measurement could be used	‘I heard the rheumatologists very clearly saying that… they were worried that if Health Canada had it then what would Health Canada do or what would insurance companies (do).’ (Healthcare Provider FG2) ‘I would be afraid that it would be used by managers in the wrong way’ (Healthcare provider IG4) ‘I think some people get nervous about the medical/legal impact of (measurement reporting)’ (Healthcare Provider FG3)
Physician attitudes on practice feedback through measure reporting and practice change	‘People are sensitive to being criticized, so of course as a physician you are going to feel crappy if you get a bad score and you may turn off some potential people. Also people are collecting a lot of stuff now and feel like they’re being Big Brothered up the wazoo.’ (Healthcare Provider IG1) ‘Look, if you’ve been doing the same thing for twenty years and it’s almost like somebody tells you you need to raise your kids differently. I mean I’ve only been in for ten years, but I’d be like no, this is the only way I know how to do it. For some kid to go talk to a seasoned rheumatologist to say you’re doing it all wrong I think is going to rub most people very much the wrong way.’ (Healthcare Provider IG10) ‘I know from working with the practitioners, like they are not interested in receiving any feedback. They were grandfathered into the system and they do it their way and they’re going to do it until they retire.’ (Patient IG8) This is especially true of ’the old guard’
Futility (real or perceived inability to make practice changes)	‘…aren’t we losing our time producing scorecards saying that no one is meeting the standard.’ (Healthcare Provider IG2 referring to challenges meeting wait times for care) ‘No matter how much we gather, data we have, if that’s not in their plan.’ (Healthcare Provider FG2 referring to challenges obtaining funding to improve care)
Concern measure not reflective of care	‘I don’t think that really necessarily tells about the quality. Like it doesn’t capture the quality of care that I’m providing.’ (Healthcare Provider when discussing a measure of disease activity IG15) ‘I don’t think that it could carry a lot of weight whenever there isn’t standardized care across the country.’ (Healthcare Provider IG2)
Misinterpretation of results	‘Patients might misinterpret it. I might misinterpret it. Anybody could misinterpret it. You have to be really clear on what the numbers mean’ (Healthcare Provider FG3)

FG, Focus Group; IG, Interview Group.

HCPs also expressed concerns about being measured on ‘outliers’, including patients that were ‘particularly complex’, where outcome measures such as low disease activity or remission were difficult to meet. They suggested a number of factors that could be considered when reporting results that would provide context, such as baseline disease activity and disease duration. While the concept of risk adjustment of measures was not directly discussed, it appeared that HCPs were concerned about potential differences in patient populations that could make comparisons between practices challenging and/or misleading. Being measured on processes of care was more acceptable as was using aggregate data for reporting purposes.

Facilitators and strategies for framework implementation

Participants highlighted strategies that were important to ensure effective quality framework implementation. These are described in table 4 along with selected quotations. At a system level, financial and political will were highlighted as major facilitators. Policy leaders in two provinces detailed successes in measuring surgical wait times for hip and knee replacement surgeries following federal incentives for improving access to care and suggested that a similar process might be developed for RA. At a clinic level, nurse-led models of care were described by some healthcare practitioners as being facilitators for data entry which could support tracking of clinical outcomes conducive to measure reporting.

Table 4.

Strategies for effective quality framework implementation

Strategy	Selected quotations
Financial and political will	‘At the outset, back in 2004 when the last federal accord was stood up and there was money allocated to improving wait times to hip and knee replacement surgery, there was no such thing as a national benchmark in terms of wait times for hip and knee replacement. However, because there was money attached to---federal money attached to provinces meeting those targets; very, very quickly it became the benchmark. And today nobody questions those federal or national benchmarks as being reasonable, attainable or whether it is the standard…’ (Policy Leader IG18)
Nurse/allied healthcare provider-led models of care that assist with data entry to support tracking of outcomes	Regarding nurse-led model of care: ‘My nurses have already done the history, the joint assessment and they’re just a phenomenal workhorse. They have made excellent forms…. I walk in and I see the joint assessment…. Before my patients even see the nurse they go into a kiosk and they do a touch screen form. They give their HAQ; they let us know various things about morning stiffness, hospital visits, change of medicines….’ (Healthcare Provider IG10)
Knowledge translation to ensure effective messaging of results	‘Like having maybe a third party look over it and to kind of bring things together.’ (Patient IG8)
Ensuring constructive feedback	‘I think constructive feedback is really useful to provide, but unconstructive feedback is not very useful.’ (Patient IG8) ‘And let them know it’s about system improvement and it’s not about finding out that they’ve got a bad practice.’ (Healthcare Provider IG9)
Clinical champions	‘you could have clinical champions for examples to present their own data and say here’s what I found and here’s where the data made me unhappy. Here’s where I think it’s valid and here’s what we’ve done to think about improvement and that can encourage other people to talk about it.’ (Policy Leader IG13)
Confidence in methods brings confidence in results	‘you probably want the people whose performance is directly being measured to have bought into the methodology, the data. Like work out all the kinks and maybe transparently say we’re going to move towards public reporting, but before we do that we want to work with you to make sure we’ve got these measures right and we’ve got the analysis right, we’ve got the interpretation right’ (Policy Leader IG13)

FG, Focus Group; IG, Interview Group.

Other facilitators included having effective knowledge translation strategies for messaging of results, using quality measure results as sources of constructive feedback, having clinical champions for quality improvement and robust methods for quality measurement to ensure confidence in results.

Discussion

The perspectives of arthritis stakeholders have been used to inform the development of a healthcare quality framework for RA. Importantly, although we had initially embarked on a project to develop a BSC-based quality of care framework for RA, following the input from arthritis stakeholders we have determined that a BSC framework did not resonate with them and was not reflective of their needs for quality improvement. While the BSC framework has been widely applied in healthcare, there have been challenges including the need to modify traditional domains of the scorecard to ‘fit’ the healthcare context.¹⁹ Our participants indicated that the IOM framework (or a similar healthcare framework) had greater relevance as it required no further adaptation and would be more easily accepted by HCPs. This study has additionally provided important perspectives to consider when developing measures to populate the framework including 13 potential areas for measurement and highlights measurement gaps as well as potential implementation challenges including potential physician reticence to measurement and barriers in accessible data. The study also highlights potential strategies to enhance measurement framework success including political will/incentivisation at a health system-level and clinic-level factors including nurse/AHP-led models of care to facilitate data collection to foster quality improvement.

Our study identified 13 aspects of patient care that could be used for quality measurement including patient outcomes. This work is complementary to existing qualitative studies evaluating important outcomes for patients with RA. Ahlmén et al evaluated patient perspectives on RA outcomes and found four main themes including: normal life, physical capacity, independence and well-being.³⁷ In a similar study Carr et al outlines seven themes: physical, general well-being, independence, return to normality, emotional impact, fear of the future and that the relative importance of outcomes changes over time depending on circumstances.³⁸ In our study, all identified patient outcomes were related to these previously described larger themes (eg, physical function is related to the larger themes of independence and physical health). The challenge from a quality measurement perspective is that many of these patient-important outcomes (eg, general well-being, fatigue, quality of life) are infrequently measured at the point of care. Indeed, a recent systematic review of published quality measures in RA outlines very few quality measures addressing patient outcomes such as pain and quality of life.³⁹ We believe future work to incorporate patient-important outcomes in daily clinical care and quality improvement efforts is central to improving patient-centred care.

An additional barrier related to patient outcome measurement is that HCPs were concerned about being measured on ‘outliers’. Indeed, as outlined in a recent American College of Rheumatology White Paper on Performance Outcome Measures,⁴⁰ risk adjustment and outcome attribution are among a number of important considerations when developing outcome measures.

As previously described when evaluating a set of system-level performance measures across different models of care in a number of Canadian provinces, data to measure access to care and treatment for patients with RA are not always readily available for quality monitoring.⁴¹ In some centres disease activity and patient-reported outcomes are not tracked in EMRs, necessitating the development of a secondary platform for tracking patient outcomes.⁴² With increasing EMR use by rheumatologists and primary care physicians in Canada, it is anticipated that these data sources could be leveraged in the future for more comprehensive monitoring of quality of care in rheumatic diseases.^{43 44} In the USA, the EMR-based RISE registry provides an excellent example of passive collection of data for quality improvement.¹²

Frequently cited successful quality framework and measurement initiatives included the evaluation of hip and knee surgical wait times and care in a number of Canadian provinces.^{21 22} These successes had a common driver: political will and funding. Similarly, in the UK and in the USA, political policy and financial incentives have led to largescale adoption of arthritis quality measures, although sometimes with mixed results.⁴⁵ In Canada, despite the recognition of RA as a chronic disease with substantial economic impact, quality of care metrics and patient outcomes are not routinely collected and reported.

This study has provided many useful insights for the development of our RA quality measurement framework; however, there are several limitations. It is possible that perspectives of individuals included may not be representative of other patients and HCPs across Canada. We did not collect demographic data on individuals to minimise any identifying information, this could impact transferability of our results to other specific patient populations. However, the involvement of patient partners in all aspects of study design, interpretation and reporting of results helps ensure meaningful patient engagement in this work. Importantly, while we identified many important areas of measurement in RA, it is likely that not all will be included in the final measurement framework due to data availability and the feasibility of measurement. Finally, the list of barriers and benefits to measurement frameworks may not be exhaustive as they represent our stakeholders experiences and perspectives.

Conclusions

The present study summarises stakeholder perspectives to inform the development and implementation of a quality framework for RA in Canada. Importantly, it also highlights that meaningful measurement and quality improvement should be driven with significant stakeholder engagement throughout the process of measure development, testing and reporting. Further work around advocacy for appropriate data sources, analysis, reporting and models of care to support measure collection and quality improvement are required before widespread implementation of the framework. A patient-specific tool to help patients navigate the elements of high-quality care described in the framework may also be a valuable output to consider in future knowledge translation of this work.