Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2012 Dec 1.
Published in final edited form as: Arthritis Rheum. 2011 Dec;63(12):3649–3660. doi: 10.1002/art.30605

Quality measurement and improvement in rheumatology: Rheumatoid arthritis as a case study

Sonali P Desai 1, Jinoos Yazdany 2
PMCID: PMC3228271  NIHMSID: NIHMS317785  PMID: 22127687

Introduction

In recent years, we have seen significant efforts to understand health care quality in the United States, including rheumatic diseases. Health care reform, as demonstrated through the Affordable Care Act (ACA) and Health Information Technology for Economic and Clinical Health (HITECH) Act, has further accelerated this rapidly evolving field. (1) The formidable tasks of improving quality, enhancing equity, and ensuring value in U.S. health care, remain largely unsolved.

Rheumatologists are affected by programs aimed at driving quality improvement, including performance measurement, public reporting, and tiered payment systems. Many of these programs have focused on rheumatoid arthritis (RA), a condition with rising prevalence and increasing health care costs. In this paper, we review the literature regarding quality measurement and improvement in RA with four aims: 1) to define quality in RA, 2) to delineate the history of RA quality indicator/ measure development, 3) to outline lessons learned from prior RA quality measurement experiences, and 4) to frame the future of RA quality initiatives. In reviewing this literature, we strive to highlight both the achievements and failures in developing and implementing quality measures for RA.

Defining quality in RA

Many quality of care organizations play a role in RA quality measures. These include physician groups (American Medical Association [AMA]’s Physician Consortium for Performance Improvement [PCPI]), government agencies (Centers for Medicare and Medicaid Services [CMS]’s Physician Quality Reporting System [PQRS] and Medicare Stars Program), private health care insurers, and quality measure endorsement organizations (National Quality Forum [NQF]), among others (Table 1)(2). Each of these stakeholders has an important role in measure development, field-testing, endorsement, and reporting for RA. The American College of Rheumatology (ACR) has collaborated with these stakeholders to generate the RA PQRS measures, and has developed detailed policies and procedures for endorsing potential quality measures. (3)

Table 1. Stakeholders in RA quality.

Group Example RA quality activities
Physician
Specialty
Societies (e.g.,
American
Medical
Association
Physician
Consortium for
Performance
Improvement
(PCPI)
Medical
leadership in
defining health
care quality,
quality measures
development, and
physician
recognition
programs
Collaboration with NCQA and ACR
leadership for PCPI approval of 5 RA
measures
Non-profit quality
organizations
National
Committee for
Quality
Assurance
(NCQA)
Healthcare Effectiveness Data and
Information Set (HEDIS) measures, such
as DMARD use in RA
Medical
Certification
Boards (e.g.,
American Board
of Internal
Medicine [ABIM])
Continuous
Professional
Development,
Practice
Improvement
Modules (PIMs)
ABIM role in developing PIMS, which
include a RA module
Government Centers for
Medicare and
Medicaid
Services (CMS)
Physician Quality Reporting System
(PQRS) with RA measure set (6 QIs)
Third party
payors
Measurement
programs with
incentives and
disincentives to
health care
providers
Tiering, pay-for-performance (P4P), prior
authorization programs
Quality Measure
Endorsers and
Accreditors (e.g.,
National Quality
Forum [NQF],
Joint
Commission,)
Endorsement and
approval of
measurement
sets
NQF-endorsed RA measures

The Institute of Medicine (IOM) defines quality of care as “the degree to which health care services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.” (4) Defining and measuring quality in health care is sometimes a difficult process, as only certain aspects of health care are amenable to measurement. Often, these specific and measurable aspects of care are characterized as quality indicators (QIs). A quality indicator (QI) is often framed in an “IF, THEN” format. For example, the QI on disease modifying anti-rheumatic drugs (DMARD) use in RA is “IF a patient has an established diagnosis of RA, THEN the patient should be treated with a DMARD unless contraindication to DMARD, inactive disease or patient refusal is documented”. In this example, the QI defines a process of care (DMARD use in patients with RA) that is strongly linked to an outcome (improved health outcomes among patients).

Translating a QI into a quality measure (QM) allows the measurement of performance in clinical practice. A quality measure (QM) has more detailed specifications that define the exact numerator and denominator for the QI being evaluated: “the percentage of patients age 18 and older who were prescribed, dispensed, or administered at least one ambulatory prescription for a DMARD during the 12-month reporting period”. (5) During the QI process, consideration is given to practical aspects of measurement, such as inclusion criteria (e.g. limiting measures to adult patients with RA), and exclusion criteria (e.g. excluding pregnant women and those with certain co-morbidities).

A QM is therefore a tool that enables the user to quantify the quality of a selected aspect of care by comparing it to a criterion. (6) QMs, and the QIs from which they are derived, are often developed from a combination of literature searches, evidence-based guidelines and recommendations, and expert opinion, through the use of a systematic approach. (7-10) Using RA as an example, potential QMs are categorized around five domains of quality: structure, process, outcome, access, and patient experience. (11-13) Examples of QMs referable to each of these five domains would be: structure – the number of rheumatology clinics in which disease activity score 28 (DAS28) values for RA patients are electronically documented; process – DMARD use in RA; outcome – the number of RA patients in clinical remission one year after diagnosis; access – the percentage of patients with RA who see a rheumatologist within three months of symptom onset in a particular geographic area; and patient experience – RA patient satisfaction with rheumatologist encounter.

QMs can be utilized for at least four purposes: quality improvement, accountability, transparency, and research. (15, 16) The QM on DMARD use in RA has been utilized for all of these purposes: quality improvement - identifying which patients in a practice are not on DMARD therapy and developing strategies for increasing DMARD use, accountability – measuring DMARD use routinely as part of the PQRS RA measure set, transparency – assessing differences in performance by health plans (HEDIS), and research – studying rates of DMARD therapy across different RA populations. Thus, QMs can be important tools to improve clinical practice, standardize measurement across practices, and answer challenging research questions.

A desirable QM has three essential attributes: importance, scientific soundness, and feasibility. (6) Importance relates both to the clinical relevance and the overall public health impact. Scientific soundness encompasses discrete characteristics of a measure, such as validity, reliability and risk-adjustment. Feasibility denotes the practical considerations involved in measuring the QM: are the data readily available, can the measurement process be implemented, and is the cost and burden of data collection known? In the following sections, we use the Physician Quality Reporting System RA measure set to exemplify how these three QM attributes can be applied. (14)

RA quality indicator and quality measure development

The evolution of quality measurement efforts in RA, like other examples throughout the health care system, has been a step-wise, gradual process over the last decade. The first effort can be traced back to research performed by Maclean et al. at UCLA/RAND. (17) In a large national study of RA patients, using process of care measures, researchers demonstrated suboptimal health care quality across the domains of arthritis, co-morbid disease and health maintenance. Higher quality scores were observed if specialists were involved in care. (18) This study was instrumental in defining the extent of the quality problem for patients with RA and laid the foundation for future studies defining and applying RA QIs.

In 2004, a formal QI set for RA was put forth with funding from the Arthritis Foundation (AF) (19, 20) This set was rigorously developed using the RAND/UCLA appropriateness method, which combines systematic literature reviews and expert panel ratings for potential QIs. The resulting 27 RA QIs represent a minimal standard to assess quality of care among RA patients, spanning processes of care such as imaging studies, medication use, exercise, surgery, laboratory monitoring and vaccinations.

This QI development effort in the United States also set a precedent for subsequent international efforts. In 2009, RA QIs focusing on disease course monitoring were developed in the Netherlands, using a modified Delphi method. (21) The 18 final RA QIs were rated according the grade of supporting evidence. (22) These QIs encompassed process measures (N=10), structure measures (N=5), and outcome measures (N=3) for use in daily clinical rheumatology practice. All of the structure and outcome measures (N=8) were supported by level D evidence, indicating reliance on expert opinion. The primary focus of these QIs was a single aspect of rheumatologic care: management of the RA disease course; the generalizability and feasibility of these QIs remains to be field-tested. (8)

Quality measures in RA have been derived largely from two approaches: modifying the early QI efforts outlined above, and developing new measures from clinical guideline recommendations. Many of these efforts were in response to national programs seeking to endorse or apply RA measures in the health care system. RA Quality measures currently used in the U.S. health care system are listed in Table 2. These measures were largely derived from the original 2004 AF measure set and the 2008 RA treatment guidelines. (10, 19, 20) A critical review of this list reveals that many of the measures that have gained early national prominence, have done so because they represent concepts that are easily measured with current data infrastructure. For example, the only measures submitted to the NQF to date reflect data contained within administrative claims. Many broad areas that are critical in evaluating the larger construct of health care quality in RA, such as the appropriate assessment and management of disease activity and functional status are not included, but efforts to examine these important clinical metrics are underway, as discussed below.,

Table 2. Rheumatoid arthritis (RA) quality measures (QMs) used in the United States.

Quality
indicat
or
Specification NQF
endorsed
HEDIS PQRS ACR
Tuberc
ulosis
screeni
ng
Adult patients with RA
who have
documentation of a
tuberculosis (TB)
screening performed
and results interpreted
within 6 months prior to
receiving a first course
of therapy using a
biologic DMARD
No
Periodic
assess
ment of
disease
activity
Adult patients with RA
who have an
assessment and
classification of disease
activity at least once
within 12 months.
No
Functio
nal
status
assess
ment
Adult patients with RA
for whom a functional
status assessment was
performed at least once
within twelve months.
No
Assess
ment
and
classific
ation of
disease
prognos
is
Adult patients with RA
who have an
assessment and
classification of disease
prognosis at least once
within 12 months.
No
Glucoco
rticoid
manage
ment
Adult patients with RA
who have been
assessed for
glucocorticoid use and,
for those on prolonged
doses of prednisone >
10 mg daily (or
equivalent) with
improvement or no
change in disease
activity, documentation
of glucocorticoid
management plan
within 12 months.
No
DMARD
use in
RA
IF a patient has an
established diagnosis of
rheumatoid arthritis,
THEN the patient
should be treated with a
DMARD unless
contraindication to
DMARD, inactive
disease or patient
refusal is documented.
RA
treatme
nt
acceler
ation
IF a patient has
rheumatoid arthritis and
is being treated with a
DMARD and there is
evidence of increased
disease activity or there
is evidence of
progression of RA bony
damage over a 6-month
period of time, THEN
one of the following
should be done: change
DMARD dose or route
of administration,
change DMARD, add
an additional DMARD,
start or increase dose of
glucocorticoids or
provide local
glucocorticoid
injection(s), unless the
patient refuses or all of
the above are
contraindicated.
No
Hydroxy
chloroq
uine
annual
eye
exam
The percentage of
patients with RA who
received
hydroxychloroquine
during the
measurement year and
had a fundoscopic
examination during the
measurement year or in
the year prior to the
measurement year
Methotr
exate
CBC
within
12
weeks
Adult patients with RA
who were prescribed at
least a 6-month supply
of methotrexate during
the measurement year
and received a CBC
test within 120 days (3
months + 1 month
grace period) following
the earliest observed
methotrexate
prescription claim
Methotr
exate
serum
creatini
ne
within
12
weeks
Adult patients with RA
who were prescribed at
least a 6-month supply
of methotrexate during
the measurement year
and received a serum
creatinine test in the
120 days (3 months + 1
month grace period)
after the earliest
observed methotrexate
prescription claim.
Methotr
exate
LFT
within
12
weeks
Adult patients with RA
who were prescribed at
least a 6-month supply
of methotrexate during
the measurement year
and received a LFT in
the 120 days (3 months
+ 1 month grace period)
following the earliest
observed methotrexate
prescription claim.
RA new
DMARD
baseline
CBC
Adult patients with a
diagnosis of RA who
received appropriate
baseline CBC testing
within 90 days before to
14 days after the new
start of sulfasalazine,
methotrexate,
leflunomide,
azathioprine, D-
Penicillamine,
intramuscular gold, oral
gold, cyclosporine, or
cyclophosphamide
during the
measurement year.
RA new
DMARD
baseline
LFT
Adult patients with a
diagnosis of RA who
received appropriate
baseline LFT (AST or
ALT) within 90 days
before to 14 days after
the new start of
sulfasalazine,
methotrexate,
leflunomide,
azathioprine,
cyclosporine or
cyclophosphamide
during the
measurement year.
RA new
DMARD
baseline
serum
creatini
ne
This measure identifies
adult patients with a
diagnosis of RA who
received appropriate
baseline serum
creatinine testing within
90 days before to 14
days after the new start
of methotrexate,
leflunomide,
azathioprine, D-
Penicillamine,
intramuscular gold,
cyclosporine, or
cyclophosphamide
during the
measurement year.
New RA
baseline
ESR or
CRP
within 3
months
This measure identifies
adult patients newly
diagnosed with RA
during the first 8 months
of the measurement
year who received ESR
or CRP lab tests either
4 months (3 months +
1-month grace period)
before or after the initial
diagnosis.
RA
annual
ESR or
CRP
This measure identifies
adult patients with a
history of RA who have
received ESR or CRP
lab tests during the
measurement year.

HEDIS = Health Plan Employer Data and Information Set, PQRS = Physician Quality Reporting System, DMARD = disease-modifying anti-rheumatic drug, CBC = complete blood count, LFT = liver function test, CRP = C-reactive protein, ESR = erythrocyte sedimentation rate

Health Plan Employer Data and Information Set (HEDIS)

The quality measure that has been applied to largest number of Americans with RA pertains to the use of DMARDs. This was the first rheumatology-specific QM incorporated into the Health Plan Employer Data and Information Set (HEDIS). HEDIS encompasses performance measures used in the managed care and insurance industry, providing annual data for QMs across health plans.

A recent analysis of the HEDIS RA DMARD measure by our group has shown that among 90,000 Medicare managed care enrollees with RA, between 2005-2008, 63% received a DMARD. (23) These data provide a birds-eye view of DMARD use in the United States, and suggest disparities across populations. For example, older individuals, Blacks and those with low socioeconomic status were significantly less likely to use a DMARD. In addition, we found wide variations in performance based on health plan with rates ranging from 16-87% even after adjusting for case-mix. Similarly, performance in different geographic regions ranged from 52-71% after adjustment for patient characteristics. Key questions regarding the role of patient characteristics (patient preference, co-morbidities, disease activity) and health care access (accessibility of subspecialty care, cost-sharing for medications and other health plan policies) remain and can only be answered when more clinically detailed data streams become available.

National Quality Forum (NQF)

Over the last several years, the NQF has played an increasingly important role as a national endorsement body for performance measure sets. The NQF was established by an act of Congress in 1999, and structured as a public-private partnership with broad participation from all parts of the health care system, including national, state, regional and local groups that represent patients, purchasers, employers, health care professionals, provider organizations and health plans, among others. Once measures pass the standardized endorsement process of the NQF, they are often disseminated widely throughout the health care system, with the goal of improving the quality of American health care. There have been nine RA QMs endorsed by NQF (in addition to the original RA DMARD measure), seven regarding drug safety and two evaluating baseline and annual erythrocyte sedimentation rate (ESR) or C-reactive protein (CRP) testing.

Each of these nine NQF-endorsed measures was submitted by Resolution Health, Inc., a private company that provides services to both health insurance plans and employers. The drug safety measures are summarized in Table 2. Most of these drug safety QMs can be easily measured through claims (billing) data available to health insurance companies, thus increasing their feasibility.

Although feasible to measure, several methodological questions remain regarding these recently endorsed measures. For instance, consideration of the measurement period around a particular QM cannot be understated. A study assessing a QM for follow-up lab monitoring through the EHR for RA patients initiating DMARDs, found that performance on this QM was 35%, 48% or 60% depending on the lab interval measurement methodology used. (24) This study illustrates the importance of developing and standardizing the methodology used in lab monitoring QMs.

Questions such as the appropriate interval for patients taking various doses remain unanswered (e.g. do patients with no co-morbidities taking 7.5 mg of methotrexate a week require identical monitoring to those with co-morbid conditions taking 20 mg per week?). In a recent systematic review, the scientific basis for most laboratory monitoring practices for rheumatic drugs was found to be weak and largely based on expert consensus. (25) In order to utilize lab monitoring QMs in routine clinical practice, several steps must occur: first is the development of a rigorous scientific evidence base to support monitoring recommendations, second is the determination of the appropriate modality (i.e. CBC and LFTs) and timing (the minimal interval for performing testing) of monitoring, and third is the methodologic research on measure specification. These considerations are crucial in defining the next generation of drug safety monitoring QMs in RA.

Physician Quality Reporting System (PQRS) and Registry reporting for RA QIs

PQRS is a Centers for Medicare and Medicaid Services (CMS) voluntary quality reporting program with bonus payment incentives. (26) There are six RA QIs that comprise the RA measure set for PQRS (Table 3). Five of the six PQRS RA measures are not yet endorsed by the NQF; they were developed in collaboration between the ACR and NCQA as well as the AMA’s PCPI and based on the ACR’s 2008 recommendations for RA.

Table 3. Physician Quality Reporting System (PQRS) RA QI measure set and snapshot of 2009 PQRS data.

QI 2009 RCR Data
N and %
measure met
Feasibility Scientific Soundness Comments Solutions for
improvement
Reliability Validity
Tuberculosis
screening
N=1,714 (92%
measure met)
Moderate High Moderate
  • Dependent on clear documentation of Biologic DMARD being 1st course

  • 2008 ACR guidelines do not specify that PPD screening occur ≤ 6 months of medication start as compared to QI which denotes ≤ 6 month timeframe

  • Modification of measure specification around timeframe of PPD screening (i.e. ≤ 12 months for example)

Periodic
assessment
of disease
activity
High N=929
(15% measure
met)
Moderate
N=2,205 (35%
measure met)
Low N=3,157
(50% measure
met)
Moderate High High
  • Dependent on rheumatologist use of standardized tool (i.e. DAS28, CDAI, etc.) to classify disease activity as remission, low, moderate, or high

  • Ensure that all rheumatologists integrate the use of a standardized disease activity tool into routine clinical practice

Functional
status
assessment
N=6,131 (79%
measure met)
Moderate High High
  • Dependent on rheumatologist use of standardized tool (i.e. HAQ, MDHAQ, etc.)

  • Ensure that all rheumatologists integrate the use of a standardized functional status assessment tool into routine clinical practice

Assessment
and
classification
of disease
prognosis
Good N=4,051
(52% measure
met)
Poor N=2,042
(26% measure
met)
Not assessed
N=1,667 (22%
measure met)
Low Low Low
  • Dependent on rheumatologist use of “good” versus “poor” language to describe RA

  • Data elements to define prognosis can vary over time (i.e. disease activity) and may have been assessed years prior to date of QI measurement (i.e. RF/CCP measurement at initial diagnosis)

  • The classification of RA prognosis into “good” versus “poor” is not part of routine clinical practice

  • Modification of measure specification to outline parameters for determining “good” versus “poor” instead of relying on actual terms

Glucocorticoi
d (GC)
management
Data not
available
Low Moderate Moderate
  • Dependent on incorporation of multiple discrete data elements into complex algorithm (i.e. active glucocorticoid prescription, specific dose and duration of GC use, indication for GC, documentation of GC management plan in note)

  • GC prescriptions in medical record may not reflect current use of GCs by patients and may not corroborate with GC dose/duration listed in rheumatologist notes

  • Encourage standardized documentation of GC prescriptions entered in medical record by rheumatologists

  • Consider significant modification of actual measure specification

DMARD use
in RA
N=7,260 (93%
measure met)
N/A

The role of clinical registries in facilitating quality reporting is growing. A clinical patient registry is an electronic system that helps record key aspects of patient care, including QMs. Collecting meaningful clinical data through the use of registries is a powerful way to measure and improve quality as the data captured is richer than administrative data. The strengths of utilizing clinical registries are patient tracking over time, observation of trends for directed quality improvement initiatives, and participation in national quality reporting programs such as PQRS. Since claims (billing) data can be fraught with error and manual chart review can be expensive and time-consuming, registries offer the ability to more readily capture the fundamental data elements required for quality reporting. That being said, in order to report to registries, manual chart review is required and understanding how EHRs can successfully interface with registries for ease of reporting is an important area.

Many specialty societies, the American College of Cardiology and the American Thoracic Society for example, are participating in registry reporting. They have conducted research and implemented impressive quality improvement programs through their registries, and these can certainly serve as a useful model for the field of rheumatology moving forward. The ACR has developed and launched the Rheumatology Clinical Registry (RCR) to facilitate quality reporting for PQRS for practicing rheumatologists. (27) Data from 2009 PQRS reporting reflects over 7,800 RA patients cared for by 240 providers from 125 practices. (28) A snapshot of five of the six RA QIs reported in the measure set is shown (Table 3). The RCR carries great promise for facilitating quality reporting and providing an overall representation of the quality of RA care. The RCR currently is not a nationally representative sample of RA patients that may inform clinical research since the sample derives from self-selected practices that participate in quality reporting. As participation grows, the role of the RCR in both quality initiatives and research will likely grow.

Many of the measures included in the PQRS RA measure set have not been extensively field-tested or validated, and several challenges in the measurement of the existing RA measures have emerged. Concerns have been raised about both the scientific soundness and feasibility of some of the PQRS RA measure set. Table 3 outlines some of the challenges encountered to date applying the PQRS RA measures, in the context of importance, scientific soundness (reliability, validity), and feasibility.

The role of electronic health records (EHRs) in RA QI measurement

Although EHRs hold promise in improving quality, many challenges remain ahead in realizing their full potential. In order to effectively assess performance on quality measures, the data elements required for numerator and denominator specifications must be easily extractable from the EHR. (29) An evaluation of the Veterans Affairs (VA) EHR, where all patient data is stored electronically, studied 3 RA QIs: DMARD use in RA, RA core data set, and RA treatment acceleration. (30) Some of the key requisite data elements, such as joint count, functional status assessment, and quantitative evaluation of x-rays are not typically entered in the EHR as structured data; thus they cannot be obtained easily from the EHR for quality reporting. Aside from these practical considerations, larger questions also remain about the most effective means of utilizing EHRs to improve quality. (31) (32)

In the case of RA, health information technology can be developed to assist rheumatologists with joint counts and disease activity assessments in rheumatology practice. (33-35) It is imperative that EHRs integrate tools to facilitate the documentation, measurement and reporting of QMs to help rheumatologists care for RA patients. In addition, the alignment of these EHR tools with electronic registry reporting, such as the RCR, are important steps to help busy, practicing clinicians caring for patients with RA participate in national quality improvement reporting initiatives.

The successful use of EHR tools does require bioinformatics expertise, time, and funding for development; and physician training, workflow considerations and added time during the office visit for implementation into routine clinical practice.(36) (37) Once these changes are made and adapted in clinical practice, providers may find that caring for practice of RA patients is easier. For example, a practice can decide to use a standardized disease activity assessment tool at each RA patient visit, enter the disease activity score in the EHR, and engage the patient in observing trends in RA disease activity over time.(38) Meaningful use of an EHR is the ability of the EHR to be used in a meaningful manner, to allow exchange of health information electronically for quality improvement and to be utilized for submission of clinical quality measures. (39) Not only are these innovations potentially helpful in shared medical decision making between the patient and provider, but they are important for achieving meaningful use of the EHR. (40)

Lessons learned from prior RA quality measurement experience

In order to develop the next generation of RA QMs, we must critically appraise the literature on existing RA measures. Above, we have summarized some of the key lessons learned from QMs that have gained national prominence in RA, including those endorsed by NQF, NCQA/HEDIS, NCQA/PCPI and the ACR. Most of the RA measures currently in use are process based and not outcome based. In chronic diseases such as RA, it can be difficult to develop, measure, and define QIs assessing outcomes of care. In addition, outcomes may take years to occur and are influenced by a multitude of factors often not under the direct control of the patient, provider, or health care system. However, despite these challenges, the ultimate goal of RA treatment strategies is to improve a variety of outcomes for RA patients, and we anticipate that outcome measures will likely be introduced in the coming decade. In a recent landmark editorial, Michael Porter explained that providing high-value in health care delivery rather than simply focusing on quality, involves incorporating outcomes of care as they relate to cost, while centering care around the individual patient.(41)

Importance of detailed specification of QMs

One currently used RA quality measure is laying the groundwork for outcome measures in RA. The RA treatment acceleration measure is outlined in Table 2. This process measure is in part based on objective outcomes: disease activity and radiographic progression. Several studies have assessed performance on this process of care, with adherence ranging from 50-85%. (42-44) Important lessons in the specification of this measure have become apparent: variation in measure performance may relate to the level of disease activity (moderate or severe), duration of time patient is at disease activity level (months), and time allotted to the rheumatologist for an intervention to occur. (42) That is, if a patient has severe disease activity that is sustained over a longer period of time, or if this measure is assessed at six months rather than three months, performance on the measure will likely be higher. There are several factors that may influence the decision of a rheumatologist to escalate RA care, in addition to disease activity or radiographic progression. (45) Future testing of the RA treatment acceleration measure will require detailed specifications in order to standardize this QI over different populations.

Applicability of RA QMs to rheumatologists versus primary care physicians

In addition to the experiences gained in measure development, specification and validation, the experience to date has also shown that different measures may have applicability to different structural levels of the health care system. For example, as outlined in Table 4, performance on the RA DMARD measure varies from 30-94%, depending on study design (administrative data versus manual chart review), geographic location, type of insurance plan, time period, and health delivery system (Table 4). (42, 44, 46-49) An important lesson learned from application of this QM is that its utility in assessing quality in those who have access to subspecialty care (e.g. those already under the regular care of rheumatologists) is likely limited, as 90% of these patients are taking a DMARD. This measure may be more useful when applied to nationally representative samples of patients with RA where access to subspecialty care may be uneven and geographic and socioeconomic determinants of DMARD use may become apparent.

Table 4. Performance on three rheumatoid arthritis (RA) quality measures (QMs).

RA QMs Population Findings
DMARD use in RA
  • Claims data

  • Medical record review

RA treatment
acceleration*
  • Medical record review

  • 46% with moderate disease activity had treatment acceleration (42)

  • 80% with severe disease activity had treatment acceleration (42)

  • 85% RA treatment acceleration (44)

  • 50-64% treatment acceleration (43)

RA core data set**
  • Medical record review

  • 69% RA core data set (44)

  • Joint exam: 98%, functional status: 83%, global assessment: 91%, ESR or CRP: 66%, pain: 92% (43)

*

See Table 2.

**

RA core data set = IF a patient has a diagnosis of RA, THEN each of the following should be documented within 3 months of diagnosis and at appropriate time intervals thereafter: a joint exam, functional status, disease activity (presence/ absence of synovitis, ESR/ CRP) and pain (20)

The future of RA quality initiatives

In the preceding sections, we have highlighted various RA quality measure development efforts and the current associated data systems used in U.S. measurement efforts. As our experience grows, both measure and data system development will be an iterative process, where we build on successful efforts and discard those that were well-intentioned but less feasible or valid. At the national level, RA measures that are NQF-endorsed are already being applied by payors and the government to assess quality. As the number of QMs grows and physician performance incentive programs expand, it will become essential to find ways to incorporate quality measurement into routine clinical practice. An example of a performance incentive program is pay-for-performance (P4P), a mechanism designed to tie a proportion of a physician’s remuneration to performance on pre-specified QMs. (50) Several challenges have been identified in engaging specialist physicians in performance incentive programs: the development of meaningful quality metrics, the question of accountability between different providers caring for the same patient, the nuances of appropriate risk-adjustment, the information technology infrastructure to extract quality data, and the need for robust EHRs capable of facilitating quality measurement. (51)

Despite these potential limitations, there are several key steps to measure and improve the quality of care provided to their RA patients. These steps include: identifying areas where there are gaps in quality, understanding the clinical practice process changes that must occur, learning the methods for process redesign, and espousing performance measurement. (52) Continuous quality improvement (CQI) methods such as Plan-Do-Study-Act (PDSA) cycles can be employed to perform numerous small tests of change with the goal of making improvements without disrupting large processes of care. (36, 53, 54) An example of this would be integrating patient history and patient and physician-reported disease assessments dictation templates for all RA patients. (55)

On a broader scale, system-level process improvement involves thinking about the experience of health care delivery for a particular disease, such as RA, across the entire continuum of care – how does a patient with RA have his/her needs addressed by the health care system? (37) RA is the prototypical chronic disease, cared for by multiple physicians (primary care physician, rheumatologist, orthopedist, etc.) and health care providers (physician, nurse, medical assistant), and in various health care delivery settings (outpatient – rheumatology clinic, physical therapy, inpatient). The Wagner model of chronic illness provides a framework for thinking about how to improve the quality of care for RA patients. (56) This model stresses four key components: 1) provide patients with self-management support to increase self-efficacy, 2) reorganize the practice system and team function to help meet chronic illness patient needs, 3) develop and implement evidence-based guidelines with education and reminders for providers, and 4) augment information systems to help create patient disease registries for tracking and providing feedback on performance (Figure 1). (57)

Figure 1. Wagner model of chronic illness.

Figure 1

This model of chronic illness stresses four key components: 1) provide patients with self-management support to increase self-efficacy, 2) reorganize the practice system and team function to help meet chronic illness patient needs, 3) develop and implement evidence-based guidelines with education and reminders for providers, and 4) augment information systems to help create patient disease registries for tracking and providing feedback on performance. It serves as a framework for guiding RA quality improvement efforts.

The foundation of Wagner’s model is a productive interaction between the patient and the practice team. One can apply the Wagner model to measure and improve upon existing RA QIs to ultimately enhance the quality of care provided to RA patients (Table 5). For example, one way to reorganize the practice system and team to meet needs of the RA patient would be to think about how best to capture information on disease activity assessment. Routinely measuring and acting upon changes in disease activity is a central focus of RA management; therapeutics have evolved such that the goal of therapy is clinical remission. (58) Two of the RA measures, disease activity assessment and treatment acceleration, rely upon the regular assessment of disease activity in routine clinical practice. If patients could provide a global health assessment and self-report joint count either prior to the visit (i.e. web-based tool) or at the point-of-care (i.e. paper or electronic questionnaire), this data could be more readily assimilated into the overall RA management plan during the office visit. (59)

Table 5. Wagner model of chronic illness applied to RA for improving quality of care through process improvement.

Wagner model
component
RA QI Example of process
improvement
Patient self-management
support
Functional status
assessment
Routine questionnaires
(paper or electronic) at
each RA office visit for
data collection and
tracking of trends over
time with patient and
physician feedback using
standardized instruments
(HAQ, MHAQ, etc.)
Reorganizing practice
system and team to meet
needs of chronic illness
patient
Disease activity
assessment

RA treatment
acceleration
Web-based or point-of-
care routine disease
activity evaluation using
standardized instrument
(DAS28, CDAI, RADAI,
etc.)
Evidence-based
guidelines with provider
education
DMARD use Clinical decision support
physician reminders in
EHR for patients with RA
not on DMARD therapy
Creating patient disease
registries
Glucocorticoid
management
Outreach efforts by nurse
case manager to registry
of all RA patients on
long-term glucocorticoids
for osteoporosis
prevention, screening
and treatment

Conclusions

There is consensus among all stakeholders that we must work to improve quality and value in the U.S. health care system. Quality measurement and improvement efforts hold great promise and have begun to revolutionize the care provided in certain settings. The dramatic decrease in catheter-related bloodstream infections in hospitals across the United States is a prominent example of how implementation of a standardized checklist can significantly improve the quality of care. (60) In order for the potential to be realized in other areas, especially in outpatient care for most chronic diseases, we have our work cut out for us. Basic methodological issues include the development of sound important measures. Practical concerns are barriers in creating a data infrastructure and large-scale systems improvement for instituting meaningful change.

The federal government has outlined a National Quality Strategy that includes seven key elements: safer care, effective care coordination, person and family centered care, prevention and treatment of leading causes of mortality, supporting better health in communities, and making care more affordable. (61) The mission of the NQF is aligned with the National Quality Strategy to achieve these goals. For patients with RA, a prevalent chronic disease with fluctuations in disease course over time, the first step is the development of a set of important, scientifically sound yet feasible QMs. Rheumatologists should lead the way in scrupulously developing and field-testing new measures that can be submitted to organizations such as the National Quality Forum for endorsement. Active engagement in measure development and submission will ensure that clinically meaningful measures that more closely reflect high quality of care in RA are included in these key measure sets. Health information technology innovations should be integrated with EHRs to facilitate QM documentation, measurement and performance improvement in routine clinical practice. Collaborations among stakeholders, both within the United States and internationally, around RA management guidelines can help solidify the evidence-base from which QMs are generated. As the healthcare landscape continues to change rapidly, the field of rheumatology can be poised to take a leadership role in defining, measuring and improving the quality of care for RA patients.

Acknowledgements

We are grateful to Dr. Daniel H. Solomon for his insightful comments on this manuscript.

Support:

Dr. Desai is supported by the American College of Rheumatology (ACR) Research and Education Fund (REF) Physician Scientist Development Award. Dr. Yazdany is supported by the Arthritis Foundation (AF), the Agency for Healthcare Research and Quality (AHRQ) / National Institute of Arthritis and Musculoskeletal and Skin Diseases (NIAMS), the Resource Evaluation and Allocation Committee at the University of California, San Francisco, and the Rosalind Russell Center for Arthritis.

References

  • 1.2010 November 7; http://www.healthcare.gov/index.html.
  • 2.Foundation KF. What’s in the Stars? Quality Ratings of Medicare Advantage Plans. 2010 2009. [Google Scholar]
  • 3.Cohen S, Gabriel S, Moynihan E. The quality movement: rheumatologists need to be prepared. Practice view. 2006;1:1–11. [Google Scholar]
  • 4.Lohr KN. Medicare: A Strategy for Quality Assurance. National Academy Press; Washington D.C.: 1990. [PubMed] [Google Scholar]
  • 5. [Accessed on: November 18, 2010];RA DMARD therapy. Accessed at: http://www.rheumatology.org/practice/office/pqri/2010_measures/measure_108/measure108_description.pdf.
  • 6. [Accessed on November 7, 2010];National Quality Measures Clearinghouse: Selecting and using measures. Available at: http://qualitymeasures.ahrq.gov/selecting-and-using/using.aspx.
  • 7.Campbell SM, Braspenning J, Hutchinson A, Marshall MN. Research methods used in developing and applying quality indicators in primary care. Bmj. 2003;326(7393):816–9. doi: 10.1136/bmj.326.7393.816. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Vliet Vlieland TP, Huizinga TW. Quality indicators in rheumatology: valid for whom? Ann Rheum Dis. 2009;68(12):1797–9. doi: 10.1136/ard.2009.116582. [DOI] [PubMed] [Google Scholar]
  • 9.Solomon DH, Gabriel SE. Quality Measures 101: what every rheumatologist should know. Clin Exp Rheumatol. 2007;25(6 Suppl 47):18–21. [PubMed] [Google Scholar]
  • 10.Saag KG, Teng GG, Patkar NM, Anuntiyo J, Finney C, Curtis JR, et al. American College of Rheumatology 2008 recommendations for the use of nonbiologic and biologic disease-modifying antirheumatic drugs in rheumatoid arthritis. Arthritis Rheum. 2008;59(6):762–84. doi: 10.1002/art.23721. [DOI] [PubMed] [Google Scholar]
  • 11.Donabedian A. The quality of care. How can it be assessed? Jama. 1988;260(12):1743–8. doi: 10.1001/jama.260.12.1743. [DOI] [PubMed] [Google Scholar]
  • 12.National Quality Measures Clearinghouse [Accessed on: November 7, 2010]; Available at: http://www.qualitymeasures.ahrq.gov/browse/by-domain.aspx.
  • 13.Gardner LA, Snow V, Weiss KB, Amundson G, Schneider E, Casey D, et al. Leveraging improvement in quality and value in health care through a clinical performance measure framework: a recommendation of the american college of physicians. Am J Med Qual. 25(5):336–42. doi: 10.1177/1062860610366589. [DOI] [PubMed] [Google Scholar]
  • 14.KG Saag, Yazdany J, Alexander C, Caplan L, Coblyn J, Desai SP, et al. Defining quality of care in rheumatology the American College of Rheumatology white paper on quality measurement. Arthritis Care Res (Hoboken) doi: 10.1002/acr.20369. [DOI] [PubMed] [Google Scholar]
  • 15.Solberg LI, Mosser G, McDonald S. The three faces of performance measurement: improvement, accountability, and research. Jt Comm J Qual Improv. 1997;23(3):135–47. doi: 10.1016/s1070-3241(16)30305-4. [DOI] [PubMed] [Google Scholar]
  • 16.Chassin MR, Loeb JM, Schmaltz SP, Wachter RM. Accountability measures--using measurement to promote quality improvement. N Engl J Med. 363(7):683–8. doi: 10.1056/NEJMsb1002320. [DOI] [PubMed] [Google Scholar]
  • 17.MacLean CH, Louie R, Leake B, McCaffrey DF, Paulus HE, Brook RH, et al. Quality of care for patients with rheumatoid arthritis. Jama. 2000;284(8):984–92. doi: 10.1001/jama.284.8.984. [DOI] [PubMed] [Google Scholar]
  • 18.Yazdany J, MacLean CH. Quality of care in the rheumatic diseases: current status and future directions. Curr Opin Rheumatol. 2008;20(2):159–66. doi: 10.1097/BOR.0b013e3282f50ec4. [DOI] [PubMed] [Google Scholar]
  • 19.MacLean CH, Saag KG, Solomon DH, Morton SC, Sampsel S, Klippel JH. Measuring quality in arthritis care: methods for developing the Arthritis Foundation’s quality indicator set. Arthritis Rheum. 2004;51(2):193–202. doi: 10.1002/art.20248. [DOI] [PubMed] [Google Scholar]
  • 20.Khanna D, Arnold EL, Pencharz JN, Grossman JM, Traina SB, Lal A, et al. Measuring process of arthritis care: the Arthritis Foundation’s quality indicator set for rheumatoid arthritis. Semin Arthritis Rheum. 2006;35(4):211–37. doi: 10.1016/j.semarthrit.2005.08.004. [DOI] [PubMed] [Google Scholar]
  • 21.van Hulst LT, Fransen J, den Broeder AA, Grol R, van Riel PL, Hulscher ME. Development of quality indicators for monitoring of the disease course in rheumatoid arthritis. Ann Rheum Dis. 2009;68(12):1805–10. doi: 10.1136/ard.2009.108555. [DOI] [PubMed] [Google Scholar]
  • 22.Guyatt GH, Sackett DL, Sinclair JC, Hayward R, Cook DJ, Cook RJ, Evidence-Based Medicine Working Group Users’ guides to the medical literature. IX. A method for grading health care recommendations. Jama. 1995;274(22):1800–4. doi: 10.1001/jama.274.22.1800. [DOI] [PubMed] [Google Scholar]
  • 23.Schmajuk G, Trivedi AN, Solomon DH, Yelin E, Trupin L, Chakravarty EF, et al. Receipt of disease-modifying antirheumatic drugs among patients with rheumatoid arthritis in Medicare managed care plans. Jama. 305(5):480–6. doi: 10.1001/jama.2011.67. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Agnew-Blais JC, Coblyn JS, Katz JN, Anderson RJ, Mehta J, Solomon DH. Measuring Quality of Care for Rheumatic Diseases Using an Electronic Medical Record. Ann Rheum Dis. 2008 doi: 10.1136/ard.2008.089318. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Schmajuk G, Yazdany J. Drug Monitoring in Systemic Lupus Erythematosus: A Systematic Review. Semin Arthritis Rheum. doi: 10.1016/j.semarthrit.2010.07.010. [DOI] [PubMed] [Google Scholar]
  • 26. [Accessed on: November 7, 2010];PQRI Overview. Available at: http://www.cms.hhs.gov/PQRI/
  • 27. [Accessed on: November 11, 2011];Rheumatology Clinical Registry: Data-driven, optimal care. Available at: http://www.rheumatology.org/practice/clinical/rcr.asp.
  • 28.Kazi S, Elizabeth A. American College of Rheumatology. Atlanta, GA: Nov, 2010. Tindall, Kristin McNiff, Itara Barnes. Early experience with the American College of Rheumatology (ACR) Rheumatology Clinical Registry (RCR) November 2010. [Google Scholar]
  • 29.Kern LM, Dhopeshwarkar R, Barron Y, Wilcox A, Pincus H, Kaushal R. Measuring the effects of health information technology on quality of care: a novel set of proposed metrics for electronic quality reporting. Jt Comm J Qual Patient Saf. 2009;35(7):359–69. doi: 10.1016/s1553-7250(09)35051-5. [DOI] [PubMed] [Google Scholar]
  • 30.Williams CA, Mosley-Williams AD, Overhage JM. Arthritis quality indicators for the Veterans Administration: implications for electronic data collection, storage format, quality assessment, and clinical decision support. AMIA Annu Symp Proc. 2007:806–10. [PMC free article] [PubMed] [Google Scholar]
  • 31.Bates DW, Kuperman GJ, Wang S, Gandhi T, Kittler A, Volk L, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc. 2003;10(6):523–30. doi: 10.1197/jamia.M1370. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Sequist TD, Cook DA, Haas JS, Horner R, Tierney WM. Moving health information technology forward. J Gen Intern Med. 2008;23(4):355–7. doi: 10.1007/s11606-008-0551-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Collier DS, Kay J, Estey G, Surrao D, Chueh HC, Grant RW. A rheumatology-specific informatics-based application with a disease activity calculator. Arthritis Rheum. 2009;61(4):488–94. doi: 10.1002/art.24345. [DOI] [PubMed] [Google Scholar]
  • 34.Collier DS, Grant RW, Estey G, Surrao D, Chueh HC, Kay J. Physician ability to assess rheumatoid arthritis disease activity using an electronic medical record-based disease activity calculator. Arthritis Rheum. 2009;61(4):495–500. doi: 10.1002/art.24335. [DOI] [PubMed] [Google Scholar]
  • 35.Richter JG, Becker A, Koch T, Nixdorf M, Willers R, Monser R, et al. Self-assessments of patients via Tablet PC in routine patient care: comparison with standardised paper questionnaires. Ann Rheum Dis. 2008;67(12):1739–41. doi: 10.1136/ard.2008.090209. [DOI] [PubMed] [Google Scholar]
  • 36.Harrington JT, Newman ED. Redesigning the care of rheumatic diseases at the practice and system levels. Part 1: practice level process improvement (Redesign 101) Clin Exp Rheumatol. 2007;25(6 Suppl 47):55–63. [PubMed] [Google Scholar]
  • 37.Newman ED, Harrington JT. Redesigning the care of rheumatic diseases at the practice and system levels. Part 2: system level process improvement (Redesign 201) Clin Exp Rheumatol. 2007;25(6 Suppl 47):64–8. [PubMed] [Google Scholar]
  • 38.Harrington JT. The uses of disease activity scoring and the physician global assessment of disease activity for managing rheumatoid arthritis in rheumatology practice. J Rheumatol. 2009;36(5):925–9. doi: 10.3899/jrheum.081046. [DOI] [PubMed] [Google Scholar]
  • 39.In.
  • 40.Bowens FM, Frye PA, Jones WA. Health information technology: integration of clinical workflow into meaningful use of electronic health records. Perspect Health Inf Manag. 7:1d. [PMC free article] [PubMed] [Google Scholar]
  • 41.Porter ME. What is value in health care? N Engl J Med. 363(26):2477–81. doi: 10.1056/NEJMp1011024. [DOI] [PubMed] [Google Scholar]
  • 42.Kahn KL, MacLean CH, Liu H, Rubenstein LZ, Wong AL, Harker JO, et al. Application of explicit process of care measurement to rheumatoid arthritis: Moving from evidence to practice. Arthritis Rheum. 2006;55(6):884–91. doi: 10.1002/art.22361. [DOI] [PubMed] [Google Scholar]
  • 43.Kahn KL, Maclean CH, Wong AL, Rubenstein LZ, Liu H, Fitzpatrick DM, et al. Assessment of American College of Rheumatology quality criteria for rheumatoid arthritis in a pre-quality criteria patient cohort. Arthritis Rheum. 2007;57(5):707–15. doi: 10.1002/art.22781. [DOI] [PubMed] [Google Scholar]
  • 44.Adhikesavan LG, Newman ED, Diehl MP, Wood GC, Bili A. American College of Rheumatology quality indicators for rheumatoid arthritis: benchmarking, variability, and opportunities to improve quality of care using the electronic health record. Arthritis Rheum. 2008;59(12):1705–12. doi: 10.1002/art.24054. [DOI] [PubMed] [Google Scholar]
  • 45.Kievit W, van Hulst L, van Riel P, Fraenkel L. Factors that influence rheumatologists’ decisions to escalate care in rheumatoid arthritis: results from a choice-based conjoint analysis. Arthritis Care Res (Hoboken) 62(6):842–7. doi: 10.1002/acr.20123. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Shipton D, Glazier RH, Guan J, Badley EM. Effects of use of specialty services on disease-modifying antirheumatic drug use in the treatment of rheumatoid arthritis in an insured elderly population. Med Care. 2004;42(9):907–13. doi: 10.1097/01.mlr.0000135810.39691.f6. [DOI] [PubMed] [Google Scholar]
  • 47.Lacaille D, Anis AH, Guh DP, Esdaile JM. Gaps in care for rheumatoid arthritis: a population study. Arthritis Rheum. 2005;53(2):241–8. doi: 10.1002/art.21077. [DOI] [PubMed] [Google Scholar]
  • 48.Schmajuk G, Schneeweiss S, Katz JN, Weinblatt ME, Setoguchi S, Avorn J, et al. Treatment of older adult patients diagnosed with rheumatoid arthritis: improved but not optimal. Arthritis Rheum. 2007;57(6):928–34. doi: 10.1002/art.22890. [DOI] [PubMed] [Google Scholar]
  • 49.Ferucci ED, Donnithorne KJ, Koller KR, Swango-Wilson A, Pflaum J, Lanier AP. Performance on rheumatoid arthritis quality indicators in an Alaska Native healthcare system. Qual Saf Health Care. 19(5):387–91. doi: 10.1136/qshc.2008.030940. [DOI] [PubMed] [Google Scholar]
  • 50.Van Herck P, Smedt D, Annemans L, Remmen R, Rosenthal MB, Sermeus W. Systematic review: Effects, design choices, and context of pay-for-performance in health care. BMC Health Serv Res. 10:247. doi: 10.1186/1472-6963-10-247. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Greenberg JO, Dudley JC, Ferris TG. Engaging specialists in performance-incentive programs. N Engl J Med. 362(17):1558–60. doi: 10.1056/NEJMp1000650. [DOI] [PubMed] [Google Scholar]
  • 52.Harrington JT. Quality of care in rheumatic diseases: performance measures and improvement. Curr Opin Rheumatol. 2008;20(2):153–8. doi: 10.1097/BOR.0b013e3282f4b200. [DOI] [PubMed] [Google Scholar]
  • 53.Berwick DM. A primer on leading the improvement of systems. Bmj. 1996;312(7031):619–22. doi: 10.1136/bmj.312.7031.619. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Speroff T, Connor GT. Study designs for PDSA quality improvement research. Qual Manag Health Care. 2004;13(1):17–32. doi: 10.1097/00019514-200401000-00002. [DOI] [PubMed] [Google Scholar]
  • 55.Harrington JT. A view of our future: the case for redesigning rheumatology practice. Arthritis Rheum. 2003;49(5):716–9. doi: 10.1002/art.11184. [DOI] [PubMed] [Google Scholar]
  • 56.Wagner EH. Chronic disease management: what will it take to improve care for chronic illness? Eff Clin Pract. 1998;1(1):2–4. [PubMed] [Google Scholar]
  • 57.Wagner EH. Managed care and chronic illness: health services research needs. Health Serv Res. 1997;32(5):702–14. [PMC free article] [PubMed] [Google Scholar]
  • 58.Smolen JS, Aletaha D, Bijlsma JW, Breedveld FC, Boumpas D, Burmester G, et al. Treating rheumatoid arthritis to target: recommendations of an international task force. Ann Rheum Dis. 69(4):631–7. doi: 10.1136/ard.2009.123919. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Pincus T, Maclean R, Yazici Y, Harrington JT. Quantitative measurement of patient status in the regular care of patients with rheumatic diseases over 25 years as a continuous quality improvement activity, rather than traditional research. Clin Exp Rheumatol. 2007;25(6 Suppl 47):69–81. [PubMed] [Google Scholar]
  • 60.Pronovost P, Needham D, Berenholtz S, Sinopoli D, Chu H, Cosgrove S, et al. An intervention to decrease catheter-related bloodstream infections in the ICU. N Engl J Med. 2006;355(26):2725–32. doi: 10.1056/NEJMoa061115. [DOI] [PubMed] [Google Scholar]
  • 61.United States Department of Health and Human Services: National Strategy for Quality Improvement in Healthcare. In; 2011.

RESOURCES