Rheumatoid arthritis quality measures and radiographic progression

Abstract

Objective

Documentation of quality measures (QMs) in rheumatoid arthritis (RA) is used as a surrogate for measure of quality of care, but the association of this documentation with radiographic outcomes is uncertain. We examined documentation of RA QMs, for disease activity and functional status and the association with radiographic outcomes.

Methods

Data were analyzed for 438 RA patients in a longitudinal cohort with complete data on van der Heijde-modified Total Sharp Score (TSS). All rheumatologist (N = 18) notes in the electronic medical record during a 24-month period were reviewed for RA QMs. Any mention of disease activity categorized as low, moderate, or high was considered documentation of the QM for disease activity. Functional status QM documentation included any mention of the impact of RA on function. Change in TSS was quantified with progression defined as ≥1 unit per year. We compared percent of visits with an RA QM documented and mean change in TSS.

Results

The mean age in the cohort was 56.9 years, disease duration was 10.8 years, baseline DAS28 score was 3.8 (±1.6), 67.7% were seropositive, and 33.9% used a biologic DMARD. Radiographic progression was observed in 28.5%. Disease activity was documented for 29.0% of patient visits and functional status in 74.7%; neither had any significant relationship to mean TSS change (both P > 0.10).

Conclusion

The documentation of RA QMs was infrequent and not associated with radiographic outcomes over 24 months.

Quality measurement and improvement in health care are gaining increasing recognition as key factors in health care reform. The Centers for Medicare and Medicaid Services has emphasized the importance of quality of care through the development of initiatives, such as the Physician Quality Reporting System (PQRS). PQRS provides a financial incentive for physicians who report on quality measures (QMs) in a variety of medical conditions, including rheumatoid arthritis (RA). QMs are the specific and measurable actions or results of care that have been established in a variety of domains [1].

QMs are translated from quality indicators into practical tools that allow for quantification of care and direct comparison between provided care and established criteria. QMs in several common conditions require specific evidence-based treatments for selected patient populations—for example, the use of aspirin in the management of acute myocardial infarction or ophthalmologic examinations for patients with diabetes mellitus [2,3]. Many of the QMs currently in use for RA rely on documentation of processes of care, such as physician-derived assessments of disease activity. Disease activity assessments, such as the Disease Activity Score (DAS28) and Clinical Disease Activity Index (CDAI), are not routinely used by rheumatologists in clinical practice in the US, but data support their use to guide treatment decisions, such as treat to target [4,5].

Patient-reported outcomes (PRO) are a valuable tool to incorporate the patient’s perspective in disease management. The shift towards a greater emphasis on patient-reported outcomes is supported by evidence [6]. Involving patients in their own care significantly improves health care outcomes, decreases health care utilization, and increases patient satisfaction, particularly in patients with chronic disease [7]. PROs are important tools in RA as well, allowing for assessment of disease burden and facilitating treatment decisions [6]. Both the Arthritis Foundation Quality Indicator 2004 Starter Set and PQRS, a voluntary reporting program of RA QMs through the Centers for Medicare and Medicaid Services, include functional status assessments in the core set of measures for RA [8].

The link between RA process measures used as QMs, such as documentation of disease activity and functional status and RA outcomes, such as radiographic progression, is not clear. Our study examined the documentation in rheumatologist notes of RA QMs on disease activity and functional status. The aims of the present study were as follows: (1) to examine variation in rheumatologist documentation of two RA QMs, assessment of disease activity, and assessment of functional status and (2) to evaluate the relationship between rheumatologist performances on RA QM documentation with radiographic progression assessed by the change in van der Heijde-modified Total Sharp Score (TSS).

Patients and methods

Study population

This study was conducted among subjects enrolled in the Brigham Rheumatoid Arthritis Sequential Study (BRASS), a longitudinal observational cohort of RA patients at an academic medical center [9]. Only patients who had at least two bilateral hand radiographs from 2003 to 2008, a minimum of 2 years apart, were included in this study, for a total of 438 patients. Although all patients in the study had hand radiographs performed, only a subset had van der Heijde-modified Total Sharp Scores performed. Institutional Review Board approval was obtained for this study, and all patients provided written informed consent for participation in BRASS.

Primary outcome (dependent variable)

Radiographic progression was the primary outcome of interest and was calculated using the van der Heijde-modified Total Sharp Score method (TSS), by four trained radiologists [10]. Practical necessity required multiple readers but all radiologists were blinded to the sequence of radiographs and to clinical data. The intra-class correlation coefficient was 0.93 for baseline and 0.85 for change scores based on a sample of 40 patients scored by two of the radiologists [11]. The TSS is the sum of the total erosion score (range: 0–160) and the total joint space narrowing score (range: 0–120), with a range from 0 to 280 [12]. A clinically significant difference for change in TSS is 5 [13,14]. The annual radiographic progression rate was calculated by taking the change in TSS divided by the number of years between the first and second hand radiograph:

$$\mathrm{Annual\; progression\; rate}=\mathrm{\Delta TSS}(\mathrm{TSS\; value\; at\; date\; of\; hand\; x}­\mathrm{ray}2-\mathrm{TSS\; value\; at\; date\; of\; hand\; x}­\mathrm{ray}1)/\mathrm{time}(\mathrm{years})$$

The TSS was categorized as no progression (<1 U/year) or progression (≥ 1 U/year).

Predictors (independent variables)

We assessed two QMs as possible correlates of the change in TSS. The first RA quality measure was assessment and documentation of disease activity and the second RA quality measure was assessment and documentation of functional status. The disease activity QM was defined as any mention of disease activity assessment in the electronic medical record (EMR), with details categorizing disease activity into low, moderate or high; or the use of a disease activity score-28 joint (DAS28) tool in the EMR. The functional status QM was characterized as any mention of how RA impacts function. For example, function may have included basic activities of daily living (washing, bathing, and dressing), activities out of the home (shopping), or work. The use of a validated instrument to measure disease activity or functional status was not required.

A trained research assistant (TN) performed a standardized chart review of each patient’s EMR over a 24-month period to extract the two RA QMs. The patient’s EMR 12 months prior to the first hand radiograph and the 12 months after were reviewed, for a total of 24 months. A subset of patient charts (N = 25 with 181 visit notes) were reviewed by both the research assistant (TN) and the principal investigator (SD) to ensure consistency of results. A Cohen’s κ statistic was calculated to evaluate the inter-rater reliability of the chart review abstraction for RA QMs on disease activity and functional status. The Cohen’s κ to evaluate inter-rater reliability for disease activity was 0.54, for functional status was 0.63, and the overall κ for both was 0.63 (N = 181 notes).

Covariates

Participants completed questionnaires at the time of enrollment and then every 6 months while enrolled in the BRASS cohort. Baseline, for the purposes of this study, was defined as the questionnaire date closest in time to the date of the first hand radiograph. The questionnaire included information about age, gender, disease duration, functional status, and medications. The treating rheumatologists diagnosed all patients with RA, the 28-joint count and disease activity, measured as DAS-28-CRP [15]. Information from the functional status questionnaires and disease activity assessments done for the BRASS cohort were not available to the treating rheumatologists at the point of care. Laboratory tests performed at the time of enrollment, included rheumatoid factor and cyclic-citrullinated peptide antibody. Medications at baseline were categorized into non-biologic and biologic DMARDs. Corticosteroid use was dichotomized as current user versus non-user at baseline. A detailed chart review captured the total number of rheumatology visit notes.

Statistical analysis

Descriptive analyses were used to calculate the baseline characteristics of the study sample. Means ± standard deviation, medians with interquartile range, and/or frequencies were calculated depending on the variable. We evaluated differences in baseline characteristics between patients included in our study versus those who were excluded from our study but part of the larger BRASS cohort who did not have radiographic data. We compared the association between the frequency of visit notes with RA QMs documented, none versus some of the time, and the mean change in TSS, using analysis of variance. The annual progression rate data were not normally distributed, so we used Wilcoxon rank sum tests to evaluate differences in the mean change in TSS between patients with the RA QMs documented and patients without the RA QMs documented. Secondary analyses also examined these relationships in subgroups of patients, by serologic status and disease activity. In order to account for clustering of patients within physicians, a generalized linear model with generalized estimating equation was utilized. All analyses were conducted using SAS (Cary, NC, version 9.2).

Results

We studied 438 subjects with disease duration 10.8 years, 81.1% female and 67.7% seropositive. The baseline DAS-28-CRP score was 3.83 and Multi-Dimensional Health Assessment Questionnaire (MDHAQ) score was 0.56. Less than one-third were taking corticosteroids at baseline, with 71.6% using non-biologic DMARDs and 33.9% biologic DMARDs (Table 1). The main differences between the study population and subjects in the BRASS cohort who did not have radiographs with TSS was that the study patients had a shorter disease duration (10.8 versus 15.1 years) and slightly less were seropositive (67.7 versus 72.4%) (both P > 0.05).

Table 1.

Baseline characteristics of the rheumatoid arthritis study population

Variable	Total study sample, N = 438 (N% or mean ± SD)	BRASS patients not in our study sample, N = 733 (N% or mean ± SD)	Patients with at least one QM, N = 353 (80.6%) (N% or mean ± SD)	Patients with no QM, N = 85 (19.4%) (N% or mean ± SD)
Age (years)	56.9 (13.1)	56.6 (14.7)	56.2 (13.1)	60.1 (12.5)
Gender (female)	354 (81.0)	608 (83.0)	288 (81.6)	66 (78.6)
Disease duration (years)	10.8 (10.4)	15.1 (13.0)	10.2 (10.0)	13.3 (11.7)
Disease activity score (DAS28)	3.8 (1.6)	4.0 (1.6)	3.8 (1.6)	4.1 (1.4)
Functional status score (MDHAQ)	0.6 (0.5)	0.7 (0.5)	0.6 (0.5)	0.6 (0.4)
Rheumatoid factor (RF) positive	258 (60.3)	434 (65.5)	213 (61.2)	45 (56.3)
Cyclic-citrillunated peptide (CCP) antibody positive	256 (59.5)	446 (65.0)	206 (59.5)	50 (59.5)
Seropositive (either RF or CCP positive)	294 (67.7)	502 (72.4)	241 (69.0)	53 (63.1)
Steroid usea	130 (29.8)	234 (31.9)	109 (30.9)	21 (25.0)
Non-biologic DMARD	313 (71.6)	505 (69.0)	250 (70.8)	63 (75.0)
Biologic DMARD	148 (33.9)	287 (39.2)	115 (32.6)	33 (39.3)
DAS28 ≤ 3.2	156 (35.6)	236 (32.2)	135 (38.2)	21 (24.7)
Total Sharp Score	32.2 (47.8)	N/A	29.0(43.8)	45.4 (60.5)

^aSteroid use defined as current user at baseline.

There was substantial heterogeneity in the number of visits per patient per year by specific rheumatologists, with means ranging from 7.8 to 10.3 visits per year. The median is based on a wide range with a minimum of 2 and maximum of 21 visits per year, based on how the patient was doing clinically and rheumatologist preference; it does not reflect visits with a nurse. There was also variation between rheumatologists in the number and percent of notes within a 12-month period that had disease activity documentation, functional status documentation, or both. As illustrated in Table 2, the percent of visits with documentation of disease activity ranged from 0.0 to 8.7 and functional status ranged from 12.2 to 45.7.

Table 2.

Variation in rheumatologist documentation of RA QMs

MDa	Study sample patients (N = 378)	Patient visits per year median (min–max)	RA QM1 Nb	RA QM2 Nc	Both RA QM1 and QM2	% Of visits disease activity (min–max)	% Of visits functional status (min–max)
1	60	9 (3–17)	5	33	4	1.5 (0.0–33.3)	20.6 (0.0–92.3)
2	20	9 (6–12)	0	13	0	0.0 (0.0–0.0)	12.3 (0.0–40.0)
3	83	8 (2–17)	37	52	11	8.7 (0.0–50.0)	14.7 (0.0–75.0)
4	20	9 (5–17)	6	12	4	4.0 (0.0–23.1)	16.2 (0.0–58.8)
5	24	10 (3–21)	1	16	0	0.6 (0.0–14.3)	12.2 (0.0–42.9)
6	32	7 (4–19)	6	32	4	4.0 (0.0–60.0)	42.2 (9.1–83.3)
7	125	7 (3–18)	55	111	45	8.5 (0.0–83.3)	45.7 (0.0–100.0)

^aOnly rheumatologists with a minimum of 20 patients enrolled in the study were included in this table.

^bRA QM1 = notes with disease activity assessment at least once in 12 months.

^cRA QM2 = notes with functional status assessment at least once in 12 months.

There was no association between the percent of visits with a RA QM documented and the mean change in TSS, at 24 months of follow-up (Table 3). The baseline TSS was 32.2 ± 47.8 with 29.5% having radiographic progression (≥ 1 unit change per year) over the 24-month follow-up period. The distribution of patient visits with at least one documentation of disease activity in 24 months was skewed, with 71.0% of patient visits without any documentation of disease activity. Among patients with least one documentation of disease activity in the 24-month period, 32.0% had DAS28 ≤ 3.2 at baseline, indicating low disease activity or remission.

Table 3.

Association between percent of visits with RA QM documented and mean annual change in TSS during chart review

RA QM	% Of visits	Change in TSS (mean ± SD)	P value
Disease activity
None	71.0	0.04 ± 6.52	0.15
Sometimes	29.0	0.15 ± 4.37
Functional status
None	25.3	0.38 ± 4.47	0.12
Sometimes	74.7	−0.03 ± 6.41

The distribution of patient visits with at least one documentation of functional status in 24 months was more uniformly distributed. Scatter plots depicting the documentation of RA QM by percent of visits versus the change in TSS did not show any relationship. Among patients who were seropositive (N = 292), who were seronegative (N = 136), who had low disease activity (DAS28 ≤ 3.2, N = 156), and who had moderate/high disease activity (DAS28 > 3.2, N = 258), there was no significant association between change in TSS and disease activity documentation or functional status documentation (Table 4).

Table 4.

Association between mean change in TSS and RA QM documentation

	Disease Activity (TSS)		Functional status (TSS)
	Mean ± SD	P value	Mean ± SD	P value
Seropositive (N = 292)
None	0.19 ± 7.46	0.12	0.51 ± 5.35	0.09
Sometimes	0.22 ± 4.96		0.10 ± 7.24
Seronegative (N = 136)
None	−0.27 ± 3.39	0.86	0.14 ± 1.69	0.80
Sometimes	−0.01 ± 2.52		−0.33 ± 3.57
Low disease activity (N = 156)
None	−0.35 ± 7.70	0.04	−0.45 ± 5.78	0.35
Sometimes	−0.37 ± 3.47		−0.33 ± 6.67
Moderate/high disease activity (N = 258)
None	0.26 ± 5.86	0.76	0.63 ± 3.72	0.40
Sometimes	0.44 ± 5.01		0.17 ± 6.30

Discussion

In this study, we evaluated the variation in rheumatologist documentation of two RA QMs, disease activity, and functional status. We also assessed the relationship between RA QM documentation and radiographic progression, among a cohort of RA patients with established disease. We found substantial variation across rheumatologists, with respect to how often patients were seen in the office and how frequently either RA QM or both were documented. Overall the documentation of disease activity was infrequent. The documentation of functional status was more frequent, but was not recorded using a standardized tool and was subject to interpretation by the chart abstractor. There was no association between documentation of these two RA QMs and radiographic progression, during the 24-month follow-up period.

QMs are used for quality improvement, accountability, transparency, and research efforts [1]. A review of health care quality indicators for RA found that the most are process indicators and there is a need for development and validation of outcome indicators [16]. The link between process and outcome measures in RA, as with many other chronic diseases, can be difficult to observe. Several studies evaluating the correlation between performance on process measures for common inpatient conditions, such as congestive heart failure and pneumonia, and the outcome of decreased mortality, showed only modest relationships [17]. The complexity of health care delivery and the many attributes that converge to determine health outcomes may be difficult to disentangle from the effect of adherence to a process-based QM. As health care reform efforts focus on value, defined as patient outcomes per dollar spent, traditional process measures of quality of care may not be adequate [18]. Implementing QMs that are clinically meaningful and promote efficient, cost-effective care presents a major challenge.

There were several important limitations in our analysis. First, we did not consider the physical examination as a component of disease activity measurement. This is because there was heterogeneity in how rheumatologists documented the physical examination and specifically the joint count and lack of routine use of a standardized disease activity measurement tool in our practices. We utilized the documentation in the rheumatologists’ notes of remission; low, moderate, or high disease activity; or the use of a standardized, validated disease activity tool, as the definition to meet the RA QM on disease activity assessment. The disadvantage of this approach is that we may have likely underestimated documentation of disease activity. However, even the use of the clinical exam can be problematic as the interpretation from a note of a thickened joint may indicate active inflammation versus chronic synovitis.

A second limitation is that the academic practice from which the study data were drawn does not have an agreed upon RA outcome measure. We did not find that rheumatologists consistently used any one standardized instrument to measure disease activity (Table 2). In addition, the moderate κ statistic of 0.63 for both disease activity and functional status demonstrates the difficulty in extracting these particular RA QMs if a quantitative standardized tool is not used for measurement. If our practice consistently used the same tool to document RA disease activity or functional status, the agreement between chart abstractors would likely be stronger. Since so few rheumatologists use a standardized tool to document RA outcomes, it did not make sense to utilize this as an outcome measure. Some rheumatologists record a DAS28; however, the practice is now emphasizing the use of the Clinical Disease Activity Index (CDAI) through the use of an electronic tool that also creates a note in the EMR [19]. The practice has also initiated the use of a computer tablet-based MHAQ for RA patients to use in the waiting room prior to appointments, which creates a note in the EMR after the physician reviews the data with the patient during the office visit. Both the CDAI and MHAQ efforts were begun after the study period.

In this study, the percent of patient visits, with at least one documentation of the RA QM during 24 months of chart review, was 29.0% for disease activity and 74.7% for functional status. Our study population was examined between the years 2003 and 2008. At this time, the PQRS RA QMs had not been widely rolled out. Thus the low documentation of RA disease activity and functional status was during a time period that pre-dated the PQRS QMs. While the Arthritis Foundation’s 2004 starter set of QMs existed, they were not widely recognized.

Although we did not find a positive association between the documentation of two RA QMs and the outcome of radiographic progression in this study, we believe that utilizing disease activity and functional status assessments in the routine care of RA patients is essential to providing high-quality care [20]. Functional status has been shown to strongly correlate with all-cause mortality risk in patients with RA and has been shown to be a better marker of disease outcome and mortality than imaging or laboratory data [21,22]. By incorporating a quantitative PRO into the routine assessment of RA patients, such as the MDHAQ, rheumatologists can help patients become part of the “shared decision-making” process [23]. Decision support tools that allow patients to better understand risks and benefits of biologic therapy may also play a key role in achieving remission and low disease activity. For example, patients may rate their disease activity as more severe than their physician, leading to discrepancies between the patient and physician perspectives on RA management [24,25]. It is conceivable that if RA QMs for disease activity and functional status are performed regularly with the use of a standardized instrument, that benefits in patient outcomes may be seen.

In the randomized clinical trial literature, radiographic progression is often the primary outcome measure; however, improvements in functional status are frequently correlated [26]. For example, recent trials of adalimumab versus abatacept showed similar outcomes for radiographic progression and Health Assessment Questionnaire Disability Index (HAQ-DI) [27]. The question remains what is the right set of QMs that accurately reflects the value of care for RA patients? The American College of Rheumatology has developed recommendations for which disease activity tools to use in clinical practice: CDAI, DAS28 (ESR or CRP), Routine Assessment of Patient Index Data with three measures, Simplified Disease Activity Index, Patient Activity Scale (PAS), and PAS-II [28]. Disease activity assessment is also an important aspect of evolving treat-to-target management strategies in RA [29,30]. In the BeST study, there was an association between patients who received more aggressive treatment strategies and improvements in functional outcomes and radiographic progression [30]. However in current clinical practice, documentation of QMs may be poor, our tools to precisely review charts to collect QM data may not be optimal, and there can be a disconnect between clinical remission and radiographic progression—erosions can still develop in patients who are in remission [31]. Thus at present, disease activity assessment may be one of the most optimal outcome measures for RA.

In conclusion, we did not demonstrate a relationship between the documentation of RA disease activity and functional status with radiographic outcomes, as assessed by the first generation of RA QMs for PQRS. However, these QMs are likely important processes of care that ought to be incorporated into routine practice, particularly as more sophisticated QMs are developed. Further studies in other practices among other patient groups should be pursued to test these QMs—for instance, early RA cohorts may be more likely to demonstrate the link between these QMs and TSS progression given the potential for radiographic progression early in the course of disease. Developing better systems for collecting and documenting this information, and involving patients in the use of this information in treatment decision making is likely to improve outcomes.