Abstract
Objective
To assess adherence to 3 system‐level performance measures in a national early rheumatoid arthritis (RA) cohort.
Methods
Patients enrolled in the Canadian Early Arthritis Cohort (2007–2015) who met 1987 or 2010 American College of Rheumatology/European League Against Rheumatism criteria with <1 year of symptom duration and ≥1 year of followup after enrollment were included. Performance measures assessed were the percentage of RA patients seen in yearly followup, and the number of gaps between visits of >12 or >14 months, the percentage of RA patients treated with a disease‐modifying antirheumatic drug (DMARD), and days from RA diagnosis to initiation of a DMARD. Results are shown stratified by enrollment year to assess for temporal changes in performance.
Results
A total of 1,763 early RA patients were included (mean age 54 years, 73% female, and 82% white). At enrollment, mean ± SD disease duration was 6 ± 3 months, and Disease Activity Score in 28 joints was 5.1 ± 1.5. Over 8 years, the proportion of patients seen in annual followup declined from 100% to 91%. Over followup, 42% of patients had 0 gaps in care of >12 months, and 64% had 0 gaps >14 months. The percentage of DMARD‐treated early RA patients was and remained high (95–87%), and the percentage receiving DMARDs within 14 days of diagnosis was 75%. Median time‐to‐DMARD therapy was 1 day, indicating DMARDs were initiated at diagnosis (90th percentile 93 days).
Conclusion
There was evidence of high adherence to system‐level performance measures in this early RA cohort following a protocol. Small declines in performance were noted with increasing length of patient followup. Our findings are useful for performance measure benchmarking.
Introduction
Measuring health care quality has received increasing attention over the last 2 decades since landmark reports from the Institute of Medicine described important gaps in health care 1, 2. In rheumatology, organizations such as the American College of Rheumatology (ACR) in the US and the National Institute for Health Care Excellence (NICE) in the UK have developed quality indicators 3, 4 and quality standards 5 as tools for monitoring and improving the quality of care provided to patients with rheumatoid arthritis (RA).
Significance & Innovations.
There was evidence of a high level of adherence to performance measures for early treatment and followup in early rheumatoid arthritis (RA) patients from rheumatology clinics across Canada who were enrolled in a prospectively followed cohort.
Measuring adherence to performance measures using an established early RA cohort is feasible and may be used for benchmarking in other settings.
The Arthritis Alliance of Canada (AAC) is a not‐for‐profit organization bringing together arthritis stakeholder groups from across Canada to participate in projects to improve the lives of patients living with arthritis. The AAC has developed an approach to models of care for inflammatory arthritis 6 and a framework for model‐of‐care evaluation that promotes provision of high‐quality care. As part of this evaluation framework, a set of 6 system‐level performance measures for inflammatory arthritis care capturing early access to care and treatment for inflammatory arthritis patients were developed 7. The measures capture the following concepts: waiting times for rheumatologist consultation, percentage of patients seen by a rheumatologist, percentage of patients seen in annual followup, percentage of patients treated with a disease‐modifying antirheumatic drug (DMARD), time‐to‐DMARD therapy, and rheumatologists per capita 7. To date, the measures have not been tested in regular rheumatology practice settings, as quality of rheumatology care is not routinely monitored in Canada outside of the context of research.
The objective of the present study is to report on 3 of the system‐level performance measures (percent seen in yearly followup, percent taking a DMARD, and time‐to‐DMARD start) in early RA patients enrolled in a pan‐Canadian study (the Canadian Early Arthritis Cohort [CATCH]) 8. The remainder of the measures were not evaluated, as the data were not available and/or the measure was not appropriate to measure in CATCH (e.g., waiting times were not captured in CATCH, rheumatologists per capita has been evaluated elsewhere 9, and the percentage of patients seen by a rheumatologist is a measure captured using population administrative data).
Patients and methods
CATCH is a prospective observational study of patients with early RA seen in academic and community rheumatology clinics across Canada since its inception in January 2007 8. Patients in CATCH are recruited by a member of their health care team and receive usual care and treatment at the discretion of their rheumatologist, but sites are encouraged to target treatment to attain remission. Members of the CATCH site primary investigators are shown in Appendix A. Patients are followed according to a standard protocol with assessments at baseline (at enrollment) and every 3 months for the first year, every 6 months, in year 2, and annually thereafter. Data from visits outside of these standard assessments are not captured in CATCH. At each visit a standardized assessment is completed, including (but not limited to) measurement of disease activity, evaluation of medications, and patient‐completed validated questionnaires on symptoms and physical function. Standard‐of‐care laboratory tests are drawn for each protocol visit at the discretion of the provider 8.
Inclusion criteria
Patients from CATCH were included in the study if they met either 1987 ACR 10 or 2010 ACR/European League Against Rheumatism (EULAR) criteria for RA with <1 year of symptom duration at enrollment 11. Patients who had not yet reached their expected 1‐year followup date or those missing the date of first treatment start were also excluded, as they would not have enough followup time to be eligible for performance measurement. Followup time began upon enrollment on or after January 1, 2007 and continued until patient withdrawal from the study or the end of the study period (May 9, 2016).
Analysis
Descriptive statistics were calculated for baseline characteristics. Chi‐square tests were used to examine differences in performance over time. Control charts were created for the measures depicting performance over time compared to overall mean and with upper and lower control limits set at 3 SDs above and below the mean. Three system‐level performance measures were operationalized and reported.
Percentage of patients seen in yearly followup by rheumatologists
The percentage was calculated using fixed 12‐ and 14‐month windows. Each patient's followup time began at enrollment and continued until the patient withdrew from the study, was lost to followup, or the study observation period ended, whichever came first. The proportion of patients meeting this measure was calculated using the denominator (e.g., expected visits: 1 for each calendar year) and the observed visits (if they had at least 1 during the year) as the numerator. The number of gaps of >12 and >14 months duration (to allow for a lag‐time in appointment scheduling) between consecutive visits was also calculated and reported as a proportion of the total number of years of followup for all patients in the cohort.
Percentage of patients taking a DMARD or biologic DMARD during the measurement year
The percentage of patients using DMARDs was calculated between the enrollment date and the end of followup on a yearly basis for each patient. The denominator included the number of patients in the cohort during the measurement year, and the numerator included the number of patients with at least 1 DMARD or biologic DMARD prescription during the year, as documented in the patient record.
Time‐to‐DMARD therapy
Time was calculated by measuring the time between the physician‐reported date of RA diagnosis and the date the first DMARD was started, and was reported as the median and 90th percentiles. Additionally, the percentage of patients treated within the Wait Time Alliance 12 benchmark of 14 days was also reported. For this outcome, patients with a missing diagnosis date were excluded. The DMARDs/biologic DMARDs included in this measure are listed in Supplementary Appendix A (available on the Arthritis Care & Research web site at http://onlinelibrary.wiley.com/doi/10.1002/acr.23439/abstract).
The measures are reported yearly between 2007 and 2015, with the exception of time‐to‐DMARD therapy, which is reported until 2014, as a complete year of followup was necessary to determine time‐to‐DMARD therapy, and individuals with a diagnosis in 2015 may not have had a complete year of followup in 2016 to meet this measure. All results were also stratified by year of enrollment in the cohort.
Ethics
The present study was approved by the University of Calgary Research Ethics Board (REB15‐2271). The CATCH study was approved by local research ethics boards at each center, and all participants provided written informed consent.
Results
A total of 1,763 early RA patients met inclusion criteria for this study (Figure 1). Enrollment characteristics were similar to other early RA cohorts, as patients had a mean age of 54 years and 73% were female (Table 1). The mean ± SD disease duration at enrollment was 6 ± 3 months and the median followup was 4 years (range 1–9 years).
Figure 1.
Inclusion criteria for the study. There were 1,763 Canadian Early Arthritis Cohort (CATCH) patients included in the analytic sample at baseline (final box in the figure). Over time there was some attrition due to withdrawals or patients who were lost to followup: 11 in the first year, 68 between the first and second years, 65 between the second and third years, 59 between the third and fourth years, 50 between the fourth and fifth years, and 65 at ≥5 years. ACR = American College of Rheumatology; EULAR = European League Against Rheumatism; RA = rheumatoid arthritis; DMARD = disease‐modifying antirheumatic drug.
Table 1.
Baseline characteristics of patients included in the study from the Canadian Early Arthritis Cohorta
| Characteristics | Total sample (n = 1,763) |
|---|---|
| Age, mean ± SD years | 54 ± 15 |
| Female | 1,283 (73) |
| Disease duration, mean ± SD months | 5.7 ± 3.0 |
| Meeting ACR/EULAR 2010 classification criteria | 1,575 (89) |
| Meeting 1987 ACR classification criteria | 1,235 (71) |
| DAS28‐ESR, mean ± SD | 5.1 ± 1.5 |
| TJC68, mean ± SD | 13 ± 9 |
| TJC28, mean ± SD | 9 ± 7 |
| SJC66, mean ± SD | 10 ± 8 |
| SJC28, mean ± SD | 8 ± 6 |
| Physician global assessment, mean ± SD | 4.9 ± 2.5 |
| Patient global assessment, mean ± SD | 5.8 ± 2.9 |
| HAQ DI, mean ± SD | 1.1 ± 0.7 |
| Anti‐CCP positive | 750 (43% overall or 60% of nonmissing)b |
| RF positive | 1,075 (61) |
| Smoking | |
| Never | 741 (42) |
| Current | 315 (18) |
| Former | 699 (40) |
| Education greater than high school | 906 (51) |
| Baseline treatmentc | |
| Nonbiologic DMARDs | 1,380 (78) |
| Biologic DMARDs | 40 (2) |
| Small molecule inhibitors or other immunosuppressive agents | 0 (0) |
| Treatment over course of followupc | |
| Nonbiologic DMARDs | 1,619 (92) |
| Biologic DMARDs | 342 (19) |
| Small molecule inhibitors or other immunosuppressive agents | 6 (0.3) |
Values are the number (%) unless indicated otherwise. ACR = American College of Rheumatology; EULAR = European League Against Rheumatism; DAS28‐ESR = Disease Activity Score in 28 joints using the erythrocyte sedimentation rate; TJC = tender joint count; SJC = swollen joint count; HAQ DI = Health Assessment Questionnaire disability index; anti‐CCP = anti–cyclic citrullinated peptide; RF = rheumatoid factor; DMARDs = disease‐modifying antirheumatic drugs;
There are 511 missing.
Treatments included in these categories are shown in Supplementary Appendix A, available on the Arthritis Care & Research web site at http://onlinelibrary.wiley.com/doi/10.1002/acr.23439/abstract. Due to low numbers, the numbers of patients taking small‐molecule inhibitors or other immunosuppressive agents have been combined for reporting.
The percentage of patients seen in yearly followup is shown in Table 2 (control chart available in Supplementary Appendix A, available on the Arthritis Care & Research web site at http://onlinelibrary.wiley.com/doi/10.1002/acr.23439/abstract). The overall percentage seen in yearly followup declined from 100% in 2008 to 85% in 2015 (P = 0.24). However, when analyses were stratified by year of enrollment, this trend was explained mostly due to an increasing proportion of patients with longer cohort followup in later years (e.g., more people in later years with longer followup who had a greater opportunity to get lost to followup over time). There were few statistically significant differences in followup rates when the stratified sample was reviewed, with the exception of the patients enrolled in 2011, who had higher rates of missed followups over time.
Table 2.
CATCH patients seen in yearly followup, stratified by enrollment yeara
| 2008 | 2009 | 2010 | 2011 | 2012 | 2013 | 2014 | 2015 | P | |
|---|---|---|---|---|---|---|---|---|---|
| Numeratorb | 91 | 338 | 581 | 756 | 933 | 1,023 | 1,060 | 1,056 | – |
| Denominatorc | 91 | 352 | 619 | 833 | 1,029 | 1,168 | 1,239 | 1,238 | – |
| Proportion, % | 100 | 96 | 94 | 91 | 91 | 88 | 86 | 85 | 0.24 |
| Enrollmentd | |||||||||
| 2007 | 91 (100) | 80 (89) | 66 (83) | 53 (79) | 50 (76) | 45 (79) | 37 (77) | 39 (91) | 0.74 |
| 2008 | – | 258 (98) | 228 (91) | 183 (83) | 176 (86) | 154 (81) | 134 (80) | 113 (84) | 0.38 |
| 2009 | – | – | 287 (99) | 253 (93) | 227 (87) | 202 (84) | 187 (88) | 152 (83) | 0.40 |
| 2010 | – | – | – | 267 (98) | 239 (93) | 199 (84) | 173 (80) | 162 (84) | 0.19 |
| 2011 | – | – | – | – | 241 (99) | 210 (92) | 151 (75) | 132 (74) | 0.01e |
| 2012 | – | – | – | – | – | 213 (99) | 190 (92) | 148 (83) | 0.26 |
| 2013 | – | – | – | – | – | – | 188 (99) | 158 (92) | 0.45 |
| 2014 | – | – | – | – | – | – | – | 152 (97) | – |
Values are the number (%) unless indicated otherwise. CATCH = Canadian Early Arthritis Cohort.
The numerator represents the total number of patients in CATCH seen yearly in the cohort.
The denominator represents the total number of patients in CATCH with an expected followup in each calendar year, excluding patients who withdrew from followup during the calendar year.
Patients meeting the followup measure, stratified by year of enrollment.
Statistically significant.
Similarly, the number of gaps in care also increased with increasing length of followup (Table 3), with the most common pattern of care being no gaps of >12 months between years 1 and 3 of followup, but by year 4, 50% of patients had at least 1 gap, and by year 6 over 50% had 2 or more gaps in care. When a 14‐month window was used, over half of the patients had no gaps until year 5 of followup, with increasing numbers of gaps seen after 5 years (Table 3). Overall, using a 14‐month window, 64% of patients had no gaps in care over followup compared to 42% using a 12‐month window.
Table 3.
Gaps in care of >12 and >14 months in Canadian Early Arthritis Cohort patients followed for up to 9 yearsa
| Number of gaps >12 months | Number of gaps >14 months | |||||||
|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 2 | ≥3 | 0 | 1 | 2 | ≥3 | |
| Total sample | 747 (42) | 479 (27) | 356 (20) | 181 (10) | 1,136 (64) | 485 (28) | 123 (7) | 19 (1) |
| Years in cohort (no.) | ||||||||
| 1 (172) | 165 (96) | 7 (4) | 0 | 0 | 170 (99) | 2 (1) | 0 | 0 |
| 2 (268) | 241 (90) | 25 (9) | 2 (1) | 0 | 249 (93) | 18 (7) | 1 (<1) | 0 |
| 3 (270) | 187 (69) | 78 (29) | 5 (2) | 0 | 232 (86) | 35 (13) | 3 (1) | 0 |
| 4 (243) | 90 (37) | 122 (50) | 27 (11) | 4 (2) | 162 (67) | 75 (31) | 6 (2) | 0 |
| 5 (242) | 38 (16) | 111 (46) | 78 (32) | 15 (6) | 124 (51) | 104 (43) | 13 (5) | 1 (<1) |
| 6 (242) | 17 (7) | 81 (33) | 112 (46) | 32 (13) | 91 (38) | 110 (45) | 35 (14) | 6 (2) |
| 7 (175) | 5 (3) | 37 (21) | 78 (45) | 55 (31) | 56 (32) | 83 (47) | 33 (19) | 3 (2) |
| 8 (118) | 4 (3) | 17 (14) | 50 (42) | 47 (40) | 40 (34) | 47 (40) | 22 (19) | 9 (8) |
| 9 (33)b | 0 | 1 (3) | 4 (12) | 28 (85) | 12 (36) | 11 (33) | 10 (30) | 0 |
Values are the number (%). Patients with gaps in followup are stratified by the years they were part of the cohort.
The year of followup was rounded to the closest integer, therefore a small number of patients who enrolled in early January 2007 and had a followup by mid August 2015 were included here.
The percentage of patients treated with a DMARD was high throughout all years of evaluation between 2007–2015, with a nonsignificant decline over the calendar years from 95% in 2007 to 87% in 2015, likely again due to a greater proportion of the sample with longer RA duration in later calendar years (e.g., with more opportunity to be off treatment) (Table 4 and the control chart shown in Supplementary Appendix A, available on the Arthritis Care & Research web site at http://onlinelibrary.wiley.com/doi/10.1002/acr.23439/abstract). Stratification by year of enrollment showed similar rates of DMARD use in each strata (e.g., patients with 1 year of followup had similar rates of DMARD use across calendar years, shown on the diagonals of Table 4).
Table 4.
CATCH patients treated with a DMARD (or biologic DMARD), stratified by year of enrollmenta
| 2007 | 2008 | 2009 | 2010 | 2011 | 2012 | 2013 | 2014 | 2015 | P | |
|---|---|---|---|---|---|---|---|---|---|---|
| Numeratorb | 86 | 342 | 614 | 842 | 1,015 | 1,143 | 1,240 | 1,250 | 1,078 | – |
| Denominatorc | 91 | 353 | 635 | 879 | 1,058 | 1,208 | 1,311 | 1,341 | 1,246 | – |
| Proportion, % | 95 | 97 | 97 | 96 | 96 | 95 | 95 | 93 | 87 | 0.34 |
| Enrollmentd | ||||||||||
| 2007 | 86 (95) | 90 (99) | 79 (94) | 71 (93) | 59 (89) | 54 (90) | 50 (96) | 43 (93) | 36 (88) | 1.0 |
| 2008 | – | 252 (96) | 257 (98) | 233 (96) | 199 (95) | 177 (90) | 163 (91) | 141 (92) | 105 (78) | 0.65 |
| 2009 | – | – | 276 (96) | 279 (97) | 255 (95) | 230 (92) | 211 (91) | 184 (91) | 144 (84) | 0.87 |
| 2010 | – | – | – | 259 (95) | 266 (98) | 241 (98) | 210 (93) | 189 (94) | 164 (88) | 0.91 |
| 2011 | – | – | – | – | 236 (97) | 237 (98) | 209 (95) | 169 (88) | 134 (79) | 0.25 |
| 2012 | – | – | – | – | – | 204 (95) | 210 (99) | 184 (92) | 142 (81) | 0.33 |
| 2013 | – | – | – | – | – | – | 187 (99) | 187 (99) | 156 (93) | 0.79 |
| 2014 | – | – | – | – | – | – | – | 153 (98) | 151 (97) | 0.91 |
Values are the number (%) unless indicated otherwise. CATCH = Canadian Early Arthritis Cohort; DMARD = disease‐modifying antirheumatic drug.
The numerator represents the total number of patients in CATCH treated with a DMARD or biologic DMARD in the cohort (for definitions of DMARD or biologic DMARD, see Supplementary Appendix A, available on the Arthritis Care & Research web site at http://onlinelibrary.wiley.com/doi/10.1002/acr.23439/abstract).
The denominator represents the total number of patients in CATCH eligible for DMARD treatment in each calendar year, excluding patients who withdrew from followup during the calendar year.
Patients meeting the DMARD measure, stratified by year of enrollment.
The median time‐to‐DMARD start for all patients with an available date of diagnosis and a treatment start after diagnosis (n = 1,303) was 1 day with a mean ± SD of 34 ± 88 days (Table 5 and the control chart shown in Supplementary Appendix A, available on the Arthritis Care & Research web site at http://onlinelibrary.wiley.com/doi/10.1002/acr.23439/abstract). When stratified by year of cohort entry, a trend toward decreasing time to starting a DMARD was seen (Table 5) (P = 0.02). Similarly, there was a nonsignificant trend toward an increased proportion of patients treated within the 14‐day benchmark, with 64% and 68% of patients who started DMARD treatment after diagnosis meeting this benchmark in 2007 and 2014, respectively. The percent treated within 14 days was higher at 79% when the 429 patients who received DMARD treatment prior to having a formal rheumatologist diagnosis of RA were included in this calculation.
Table 5.
Time to disease‐modifying antirheumatic drug (DMARD) therapy, stratified by enrollment year in the Canadian Early Arthritis Cohorta
| Sample size, Rx ≥ Dxb | Time to DMARDb | % treated within 14 days | ||||
|---|---|---|---|---|---|---|
| Mean ± SD, days | Median, days | 90th percentile | Rx ≥ Dxb | Include Rx < Dxc | ||
| Total | 1,303 | 34.1 ± 87.6 | 1.0 | 93.0 | 66 | 75 |
| Enrollment | ||||||
| 2007 | 61 | 37.9 ± 93.2 | 1.0 | 100 | 64 | 76 |
| 2008 | 187 | 40.8 ± 98.2 | 1.0 | 121 | 63 | 74 |
| 2009 | 219 | 44.7 ± 122.4 | 1.0 | 111 | 66 | 74 |
| 2010 | 217 | 33.1 ± 85.2 | 2.0 | 83 | 65 | 72 |
| 2011 | 174 | 38.8 ± 91.0 | 1.0 | 123 | 67 | 76 |
| 2012 | 170 | 29.5 ± 68.9 | 1.0 | 88 | 66 | 73 |
| 2013 | 140 | 18.8 ± 35.3 | 1.0 | 66 | 71 | 78 |
| 2014 | 103 | 23.1 ± 51.2 | 0.0 | 69 | 68 | 79 |
| P | – | 0.02d | – | < 0.0001 | 1.0 | – |
Rx = treatment; Dx = diagnosis.
Limited to patients treated at or after diagnosis (Rx ≥ Dx).
Including patients treated before diagnosis (n = 429).
Statistically significant.
Discussion
The CATCH cohort is a multicenter cohort representative of early RA patients treated in settings of usual care in rheumatology practices across Canada. In this study, we observed high rates of adherence to the 3 performance measures that could be captured in CATCH, including high rates of yearly followup, DMARD use, and a high percentage of patients treated with a DMARD within the benchmark of 2 weeks. While overall rates of adherence to the measures were high, a small decline in yearly rates of rheumatologist visits and DMARD use was observed with increasing length of cohort followup. Conversely, a trend toward an increased rate of patients treated with DMARDs within the 14‐day benchmark was observed with increasing calendar year.
To our knowledge, this study is the first in which nationally endorsed system‐level performance measures 7 have been evaluated in Canadian patients. The use of real‐world data has helped inform measure development, as operationalization of the measures has identified a number of issues. In the following section, a rationale for updated specifications of some of the measures is discussed. A summary of the final recommended updates is shown in Supplementary Appendix A (available on the Arthritis Care & Research web site at http://onlinelibrary.wiley.com/doi/10.1002/acr.23439/abstract). First, when the measure capturing the percentage of patients in yearly followup was operationalized, both a fixed and a rolling time interval were planned to measure the percentage. Rolling intervals capture the time between patients’ 2 consecutive visits, resetting the clock so that the second visit is used as the new anchor time point for the subsequent followup visit and so on. This method, in theory, should provide a more reliable estimate of whether there were gaps in followup visits of >12 months between 2 consecutive visits. However, when trying to report on adherence to the measure by calendar year, using rolling intervals, it became apparent that a patient could have met and missed the measure within the same calendar year, so that reporting the adherence data by calendar year was impossible. Limitations also exist to using a fixed‐interval approach, as was done for this analysis. For example, a patient may have almost 24 months between 2 consecutive visits, but because the visits fall within 2 consecutive calendar years, the measure will be met in both years. To address this possibility, a secondary analysis was conducted to identify the number of gaps in care of >12 or >14 months duration between rheumatologist visits, which is a more sensitive indicator of gaps in care. Both methods are recommended for reporting this measure.
Additionally, although the percentage seen in the yearly followup measure was originally designed to be captured and reported on a yearly basis, a 14‐month window seemed more realistic. For example, a physician may request a yearly followup with a patient, but due to a scheduling preference this appointment gets booked in 13 or 14 months, such that the performance measure would not be met, yet this length of time may not truly reflect a gap in care. Additionally, in some Canadian provinces physicians may be remunerated more for followup visits billed at an interval greater than 1 year. This fact means there may be an incentive to booking patients just over 1 year, which may also not represent a true gap in care. Indeed, the adherence to the measure improved, and the number of gaps declined significantly, when using a 14‐month rather than a 12‐month window. Ideally, a 14‐month interval for assessing followup would be optimal, as in clinical practice insurers sometimes require 12 months between appointments, so that a 12‐month interval is impossible to meet. Thus, the measure results reported for a given calendar year require 12‐month look‐back and look‐forward periods if the measure is intended for high stakes applications, to provide the most accurate performance assessment. However, measuring adherence to 14‐month windows of followup on a yearly basis over time (e.g., >1 year) becomes problematic for reporting, as the windows of expected followups will eventually no longer fall neatly within calendar years for reporting. Additionally, further evaluation should be conducted to determine whether adherence to a 12‐month versus a 14‐month window of followup impacts any patient outcomes.
For the time‐to‐DMARD measure, over 400 patients were treated with a DMARD before their physician‐reported date of diagnosis. This treatment may have been due to a variety of reasons. For example, patients may not have fulfilled RA criteria or had undifferentiated or palindromic presentations and were treated with a DMARD before a formal diagnosis of RA was made. Also the paradigm for the diagnosis of patients recruited before the 2010 ACR/EULAR criteria 11 were published may have meant that patients were being treated off‐label before a formal RA diagnosis was made according to the older ACR criteria 10. When calculating the median and 90th percentile, as recommended when reporting on this measure 7, this calculation incurred negative values for the patients treated before diagnosis. Therefore, the performance measure was reported in 2 ways. The first included only those patients treated after the diagnosis of RA was made, as ensuring no delays is critical in patients referred without a diagnosis and without treatment, and this would represent an appropriate target for quality improvement if long wait times were observed. Secondly, the performance measure included all patients with an available diagnosis and treatment start date, and considering the 14‐day benchmark met when treatment is initiated before diagnosis. Reporting both of these metrics may be useful to understand practice patterns in health systems. A final consideration for this measure is the potential for gaming (e.g., physicians delaying a documented diagnosis of RA to decrease the time‐to‐DMARD start). While an unlikely consequence of measurement, such a practice is a possibility in high‐stakes measurement. Of note, physicians participating in CATCH were not aware they were being evaluated on measure performance.
The quality of care in rheumatology is not routinely measured in Canada outside of the context of local quality improvement projects and research using administrative data 13, 14, 15 or chart reviews 16, 17. In contrast, in the US, rheumatology quality measures are reported in national programs used for physician quality reporting tied to reimbursement. The ACR Rheumatology Clinical Registry 18 was used by rheumatologists for the Physician Quality Reporting System and demonstrates that registries are useful vehicles for quality monitoring and reporting. Recently, the ACR has developed the Rheumatology Informatics System for Effectiveness (RISE) registry 19. RISE passively extracts electronic medical records (EMR) data from participating practices for quality measurement and reporting. The ACR recently developed a set of electronic quality measures for use in RISE to capture access to treatment, as well as disease activity and functional status monitoring and tuberculosis screening prior to biologic or small molecule therapy 3, 4. The DMARD performance measure from the present set 7 was designed to be harmonized with the ACR DMARD measure, to ensure comparable measures between countries; however, no additional overlap exists between the measure sets.
Current data from RISE on adherence to the ACR's e‐measure for DMARD use revealed similarly high rates of DMARD use (91% adherence) in a denominator of over 57,000 RA patients 4. In contrast, significant gaps in DMARD use are still reported in population‐based studies 15, 20, 21, even among patients under the care of a rheumatologist 20. This difference could be because rheumatologists participating in CATCH and RISE may be more aware of quality measures and more likely to adhere to them than nonparticipating rheumatologists. The difference also may reflect an enrollment bias, whereby physicians may be less likely to enroll patients in cohorts who are not expected to follow up or comply with medication recommendations (although unlikely with RISE, as it is EMR‐based enrollment). Alternatively, the completeness of administrative data compared to cohort and EMR data may also play a role in the disparate results.
Delays in DMARD starts have been associated with worse outcomes for patients, including increased radiologic progression and need for surgical interventions and lower remission rates 22, 23, 24, 25. Most significant delays to therapy occur due to delays in access to rheumatology, and time‐to‐DMARD start once a patient is seen by a rheumatologist is often negligible 26, a finding confirmed by our study. In comparison, wait‐time benchmarks for DMARD start times are more stringent in our Canadian set than in the NICE quality standards from the UK, yet higher rates of adherence are reported in our study (75% within 14 days of diagnosis in CATCH compared to 53% treated within 6 weeks of referral in the UK) 27. Unfortunately, the complementary measure of waiting time to rheumatology care, that captures the time between rheumatologist referral and first visit for patients with RA 7, was not captured in CATCH and is being evaluated in other data sources, including triage databases across the country. In the UK, adherence to the NICE quality standard wait‐time benchmark of 3 weeks for rheumatologist consultation was poor (38%) 27, likely contributing to the lower rates of timely treatment seen, as the measure for time‐to‐treatment was anchored at the referral date.
Guidelines, quality standards, and quality indicators recommend that RA patients be seen at least yearly in followup to evaluate their disease status, screen for comorbidities, and review medications 5, 7, 28, 29. There has unfortunately been little published evaluation on RA patients who may have gaps in care or get lost to followup and on the consequences of lack of rheumatologist care. In a population‐based study from British Columbia, Canada, lack of continued rheumatology care was associated with lower rates of DMARD use (22% without any rheumatology care over the prior 5 years compared to 92% with continuous care) 14. While the present study is limited, as visit intervals were done per protocol, which may have biased the results, there was still a small decline in followup rates over time. This decline, however, may not be associated with a decline in care, as patients possibly withdrew or missed appointments with CATCH but continued seeing their rheumatologist. In addition, patients who missed followups may have been in low disease activity or remission, and missing care may not have been associated with poorer outcomes. Further study is warranted to evaluate the impact of gaps in care on patient DMARD use and most importantly on patient outcomes.
While this study represents the most comprehensive examination of the recently published system‐level performance measures for inflammatory arthritis in Canada, there are a number of limitations. While treatment is at the discretion of the treating rheumatologist, care received may not be entirely representative of usual care, due to selection bias from participating rheumatologists and patients. Physicians agreeing to participate in CATCH may be more motivated to treat patients according to guidelines and best practices. Also, through the research conducted, they may receive more practice feedback than the average rheumatologist in Canada, possibly leading to higher rates of adherence to the measures. Furthermore, patients who dropped out of CATCH or who had missing data possibly could have created a reporting bias, although numbers of dropouts were small after the first year (between 11 and 68 per year). Additionally, adherence to the performance measure capturing the percentage seen in yearly followup observed in this study may represent an overestimation compared to usual care, as followup is dictated by a protocol.
The original performance measures descriptions also included some exclusion criteria, which could not be easily operationalized in the context of CATCH. For example, for the DMARD measure, patients with a pregnancy during the measurement year should be excluded from measurement; however, this variable was inconsistently captured in the data and, given the small numbers, likely had minimal impact on results and was not formally applied in this analysis. Additionally, given the source of the data, DMARD use represents DMARDs prescribed rather than DMARDs taken, as information on patient nonadherence to DMARD initiation or treatment is not available. Also, only 3 of the performance measures could be captured in this data set. While 2 of the performance measures required different data sources (percentage of patients seen by a rheumatologist and rheumatologists per capita), the addition of referral dates to CATCH data capture would allow the waiting‐time measure to be reported. Lastly, while the measures were developed considering existing national and international guidelines for RA, they were developed for the Canadian context and may not be widely generalizable. Despite these limitations, this work provides a clear set of benchmarks for further evaluation of these measures in other data sources, as this cohort likely represents a best‐case scenario.
This study represents the first time the system‐level performance measures have been tested and highlights the fact that testing performance measures is crucial prior to implementation, due to potential issues in measure operationalization and/or interpretation using real‐world data. Further testing of the measures in different data sources is planned, including in administrative data, EMR data, and other clinical cohorts. Additionally, future studies examining patient and provider factors that impact measure performance will be important to conduct, to better understand how to improve any observed gaps in care. While the rates of adherence in this cohort were high, lower adherence will perhaps be observed with other data sources, but the current results may be used in benchmarking.
Author contributions
All authors were involved in drafting the article or revising it critically for important intellectual content, and all authors approved the final version to be submitted for publication. Dr. Barber had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study conception and design
Barber, Lacaille, Marshall, Barnabe, Hazlewood, Thorne, Ahluwalia, Bykerk.
Acquisition of data
Bartlett, Boire, Haraoui, Hitchon, Keystone, Tin, Pope, Denning.
Analysis and interpretation of data
Barber, Schieir, Bykerk.
Role of the study sponsor
Companies providing financial support for the Canadian Early Arthritis Cohort had no role in the study design or in the collection, analysis, or interpretation of the data, the writing of the manuscript, or the decision to submit the manuscript for publication. Publication of this article was not contingent upon approval by any of the financial supporters.
Supporting information
Acknowledgments
The authors thank Lyne Nadeau, MSc, and Marie‐France Valois, MSc, the CATCH analysts for this study.
Appendix A. Canadian Early Arthritis Cohort Primary Site Investigators
Principal Investigator Vivian Bykerk (email bykerkv@hss.edu), Frederic Morin (Centre de Recherche Musculo‐Squelettique, Trois Rivieres, Quebec), Édith Villeneuve, Boulos Haraoui (Institut de Rhumatologie de Montréal, Montreal, Quebec), Susan Bartlett, Murray Baron, Ines Colmegna, Sabrina Fallavollita (McGill University, Montreal, Quebec), J. Carter Thorne (Southlake Regional Health Centre, Newmarket, Ontario), Derek Haaland (The Waterside Clinic, Barrie, Ontario), Michel Zummer (Université de Montréal, Montreal, Quebec), Gilles Boire (Université de Sherbrooke, Sherbrooke, Quebec), Louis Bessette (Université Laval, Quebec), Pooneh Akhavan, Ed Keystone, Laurence Rubin (University of Toronto, Toronto, Ontario), Carol Hitchon (University of Manitoba, Winnipeg, Manitoba), Bindu Nair (University of Saskatchewan, Saskatoon, Saskatchewan), Glen Hazlewood, Chris Penney (University of Calgary, Calgary, Alberta), Shahin Jamal (University of British Columbia, Vancouver, British Columbia), Janet Pope (Western University, London, Ontario), Raman Joshi (William Osler Health Center, Brampton, Ontario).
Supported by a grant from the Canadian Initiative for Outcomes in Rheumatology Care. The Canadian Early Arthritis Cohort study was designed and implemented by the investigators and supported through unrestricted research grants from Amgen, Pfizer Canada, UCB Canada, AbbVie, Eli Lilly Canada, Sanofi Genzyme Canada, Bristol‐Myers Squibb Canada, Hoffmann‐LaRoche, and Janssen Biotech.
Contributor Information
Claire E. H. Barber, Email: cehbarbe@ucalgary.ca
Canadian Early Arthritis Cohort Investigators:
Frederic Morin, Édith Villeneuve, Murray Baron, Ines Colmegna, Sabrina Fallavollita, Derek Haaland, Michel Zummer, Louis Bessette, Pooneh Akhavan, Laurence Rubin, Bindu Nair, Chris Penney, Shahin Jamal, and Raman Joshi
References
- 1. Institute of Medicine . To err is human: building a safer health system (brief report). 1999. URL: http://www.nationalacademies.org/hmd/Reports/1999/To-Err-is-Human-Building-A-Safer-Health-System.aspx.
- 2. Institute of Medicine . Crossing the quality chasm: a new health system for the 21st century (brief report). 2001. URL: http://nationalacademies.org/HMD/Reports/2001/Crossing-the-Quality-Chasm-A-New-Health-System-for-the-21st-Century.aspx. [PubMed]
- 3. Yazdany J, Myslinski R, Miller A, Francisco M, Desai S, Schmajuk G, et al. Methods for developing the American College of Rheumatology's electronic clinical quality measures. Arthritis Care Res (Hoboken) 2016;68:1402–9. [DOI] [PubMed] [Google Scholar]
- 4. Yazdany J, Robbins M, Schmajuk G, Desai S, Lacaille D, Neogi T, et al. Development of the American College of Rheumatology's rheumatoid arthritis electronic clinical quality measures. Arthritis Care Res (Hoboken) 2016;68:1579–90. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. National Institute for Health Care Excellence . Rheumatoid arthritis in over 16s: quality standard. 2013. URL: https://www.nice.org.uk/guidance/qs33.
- 6. Ahluwalia V, Frank C, Zummer M, Mosher DP. A pan‐Canadian approach to inflammatory arthritis models of care. 2014. URL: http://www.arthritisalliance.ca/.
- 7. Barber CE, Marshall DA, Mosher DP, Akhavan P, Tucker L, Houghton K, et al. Development of system‐level performance measures for evaluation of models of care for inflammatory arthritis in Canada. J Rheumatol 2016;43:530–40. [DOI] [PubMed] [Google Scholar]
- 8. Bykerk VP, Jamal S, Boire G, Hitchon CA, Haraoui B, Pope JE, et al. The Canadian Early Arthritis Cohort (CATCH): patients with new‐onset synovitis meeting the 2010 ACR/EULAR classification criteria but not the 1987 ACR classification criteria present with less severe disease activity. J Rheumatol 2012;39:2071–80. [DOI] [PubMed] [Google Scholar]
- 9. Barber CE, Jewett L, Badley EM, Lacaille D, Cividino A, Ahluwalia V, et al. Stand up and be counted: measuring and mapping the rheumatology workforce in Canada. J Rheumatol 2017;44:248–57. [DOI] [PubMed] [Google Scholar]
- 10. Arnett FC, Edworthy SM, Bloch DA, McShane DJ, Fries JF, Cooper NS, et al. The American Rheumatism Association 1987 revised criteria for the classification of rheumatoid arthritis. Arthritis Rheum 1988;31:315–24. [DOI] [PubMed] [Google Scholar]
- 11. Aletaha D, Neogi T, Silman AJ, Funovits J, Felson DT, Bingham CO III, et al. 2010 rheumatoid arthritis classification criteria: an American College of Rheumatology/European League Against Rheumatism collaborative initiative. Arthritis Rheum 2010;62:2569–81. [DOI] [PubMed] [Google Scholar]
- 12. Wait Time Alliance (WTA) . Time to close the gap: report card on wait times in Canada (2014). 2014. URL: http://www.waittimealliance.ca/wta-reports/2014-wta-report-card/.
- 13. Schmidt TJ, Avina‐Zubieta A, Sayre EC, Abrahamowicz M, Esdaile JM, Lacaille D. Quality of care for cardiovascular disease prevention in RA: compliance with lipid screening guidelines [abstract]. J Rheumatol 2015;42:1290. [Google Scholar]
- 14. Lacaille D, Lipson R, Thompson D. Rheumatologist care in rheumatoid arthritis: are we dropping the ball? J Rheumatol 2015;42:1264–65. [Google Scholar]
- 15. Lacaille D, Anis AH, Guh DP, Esdaile JM. Gaps in care for rheumatoid arthritis: a population study. Arthritis Rheum 2005;53:241–8. [DOI] [PubMed] [Google Scholar]
- 16. Barber CE, Esdaile JM, Martin LO, Faris P, Barnabe C, Guo S, et al. Gaps in addressing cardiovascular risk in rheumatoid arthritis: assessing performance using cardiovascular quality indicators. J Rheumatol 2016;43:1965–73. [DOI] [PubMed] [Google Scholar]
- 17. Keeling SO, Teo M, Fung D. Lack of cardiovascular risk assessment in inflammatory arthritis and systemic lupus erythematosus patients at a tertiary care center. Clin Rheumatol 2011;30:1311–7. [DOI] [PubMed] [Google Scholar]
- 18. American College of Rheumatology . RCR (rheumatology clinical registry). URL: http://www.rheumatology.org/I-Am-A/Rheumatologist/Registries/RCR.
- 19. Yazdany J, Bansback N, Clowse M, Collier D, Law K, Liao KP, et al. Rheumatology informatics system for effectiveness: a national informatics‐enabled registry for quality improvement. Arthritis Care Res (Hoboken) 2016;68:1866–73. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Widdifield J, Bernatsky S, Paterson JM, Thorne JC, Cividino A, Pope J, et al. Quality care in seniors with new‐onset rheumatoid arthritis: a Canadian perspective. Arthritis Care Res (Hoboken) 2011;63:53–7. [DOI] [PubMed] [Google Scholar]
- 21. Shafrin J, Ganguli A, Gonzalez YS, Shim JJ, Seabury SA. Geographic variation in the quality and cost of care for patients with rheumatoid arthritis. J Manag Care Spec Pharm 2016;22:1472–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Feldman DE, Bernatsky S, Houde M, Beauchamp ME, Abrahamowicz M. Early consultation with a rheumatologist for RA: does it reduce subsequent use of orthopaedic surgery? Rheumatology (Oxford) 2013;52:452–9. [DOI] [PubMed] [Google Scholar]
- 23. Lard LR, Visser H, Speyer I, vander Horst‐Bruinsma IE, Zwinderman AH, Breedveld FC, et al. Early versus delayed treatment in patients with recent‐onset rheumatoid arthritis: comparison of two cohorts who received different treatment strategies. Am J Med 2001;111:446–51. [DOI] [PubMed] [Google Scholar]
- 24. Möttönen T, Hannonen P, Korpela M, Nissilä M, Kautiainen H, Ilonen J, et al. Delay to institution of therapy and induction of remission using single‐drug or combination–disease‐modifying antirheumatic drug therapy in early rheumatoid arthritis. Arthritis Rheum 2002;46:894–8. [DOI] [PubMed] [Google Scholar]
- 25. Van Aken J, Lard LR, le Cessie S, Hazes JM, Breedveld FC, Huizinga TW. Radiological outcome after four years of early versus delayed treatment strategy in patients with recent onset rheumatoid arthritis. Ann Rheum Dis 2004;63:274–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Jamal S, Alibhai SM, Badley EM, Bombardier C. Time to treatment for new patients with rheumatoid arthritis in a major metropolitan city. J Rheumatol 2011;38:1282–8. [DOI] [PubMed] [Google Scholar]
- 27. Ledingham JM, Snowden N, Rivett A, Galloway J, Ide Z, Firth J, et al. Achievement of NICE quality standards for patients with new presentation of inflammatory arthritis: observations from the National Clinical Audit for Rheumatoid and Early Inflammatory Arthritis. Rheumatology (Oxford) 2017;56:223–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Bykerk VP, Akhavan P, Hazlewood GS, Schieir O, Dooley A, Haraoui B, et al. Canadian Rheumatology Association recommendations for pharmacological management of rheumatoid arthritis with traditional and biologic disease‐modifying antirheumatic drugs. J Rheumatol 2012;39:1559–82. [DOI] [PubMed] [Google Scholar]
- 29. National Institute for Health and Clinical Excellence (NICE) . Rheumatoid arthritis in adults: management. 2009. URL: https://www.nice.org.uk/guidance/cg79.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials