Using process improvement and systems redesign to improve rheumatology care quality in a safety net clinic

Abstract

Objective

To develop and evaluate interventions to improve quality of care in four priority areas in an urban safety net adult rheumatology clinic serving a racially/ethnically and socioeconomically diverse patient population.

Methods

The Institute for Healthcare Improvement’s Model for Improvement was used to redesign clinical processes to achieve pre-specified benchmarks in the following areas from 2015–2017: 13-valent pneumococcal conjugate vaccine (PCV13) administration among immunocompromised patients; disease activity monitoring with the Clinical Disease Activity Index (CDAI) for patients with rheumatoid arthritis (RA); latent tuberculosis infection (LTBI) screening for new biologic users with RA; and reproductive health counseling among women receiving potentially teratogenic medications. We measured performance for each using standardized metrics, defined as the proportion of eligible patients receiving recommended care.

Results

1,205 patients were seen in the clinic between 2015 and 2017. 71% were women, 88% identified as racial/ethnic minorities and 45% were eligible for at least one of the quality measures. Shewart charts for the PCV13 and CDAI measures showed evidence of improved healthcare delivery over time. Benchmarks were achieved for the CDAI and LTBI measures with 93% and 91% performance, respectively. Performance for the PCV13 and reproductive health counseling measures was 78% and 46%, respectively, but did not meet pre-specified improvement targets.

Conclusion

Through an interprofessional approach, we were able to achieve durable improvements in key rheumatology quality measures largely by enhancing workflow, engaging non-physician providers and managing practice variation.

Introduction

Despite recent trends to improve quality of care in the US, receipt of basic healthcare services among patients with rheumatic conditions remains suboptimal. Examples include subpar vaccination of immunocompromised patients(1, 2), variable use of treat-to-target approaches in rheumatoid arthritis(3, 4) and underutilization of osteoporosis screening and treatment(5). Compounding these deficits are healthcare disparities which place vulnerable populations at greater risk for poor health outcomes(6). Racial and ethnic minorities experience more severe disease in multiple autoimmune conditions, including lupus and ankylosing spondylitis(7–9), exacerbating socioeconomic barriers to health among these populations.

Stakeholders such as the American College of Rheumatology (ACR) and National Quality Forum (NQF) have identified key quality measures to assist rheumatology practices in measuring and improving the quality of care across diverse clinical settings(10), with the potential to reduce disparities and improve healthcare outcomes regardless of racial, ethnic or socioeconomic background(11). Research on the implementation of quality measures among vulnerable populations with rheumatic conditions in the US is limited, and only a few studies have focused attention on the opportunities and challenges afforded by safety net clinical settings(12–14).

In this study, we used the Institute for Healthcare Improvement’s (IHI) Model for Improvement to facilitate process improvement and system redesign across several domains of healthcare in a safety net rheumatology clinic that serves a racially/ethnically and socioeconomically diverse patient population. Four areas were chosen by rheumatologists and clinic staff as high priority clinical processes, including 13-valent pneumococcal conjugate vaccine (PCV13) administration among immunocompromised patients; regular disease activity monitoring with the Clinical Disease Activity Index (CDAI) for patients with rheumatoid arthritis (RA); latent tuberculosis infection (LTBI) screening for new biologic users with RA; and reproductive health counseling among women receiving potentially teratogenic medications. We evaluated the success of our interventions and sought to identify generalizable strategies for implementing QI in safety net settings.

Materials and methods

Study setting and interventions

This study took place in an academic rheumatology clinic at the Zuckerberg San Francisco General Hospital (ZSFG), a safety net hospital in San Francisco affiliated with the University of California, San Francisco (UCSF). A pay-for-performance initiative for Medi-Cal clinics to improve quality of care incentivized the project. The Performance Improvement Program (PIP) allowed individual clinics to select the most relevant clinic-specific quality measures with reasonable benchmarks over a specified time interval. Achievement of these targets resulted in small financial incentives, which was used by the rheumatology division to support faculty and staff salaries or other needs. Faculty rheumatologists and clinic staff were involved in the selection process of the four quality measures shown in Table 1, chosen for their feasibility, validity and relevance to the clinic population(10).

Table 1:

Description of quality measures in the safety net quality improvement program, including pre-specified performance targets.

Quality measure	Denominator	Numerator	Measurement period	Target
PCV13	Patients age ≥18 years on immunosuppressive medications* with ≥2 clinic visits	Patients with documented PCV13 vaccination	02/2015-03/2017	80%
CDAI	Patients age ≥18 years with RA and with ≥2 clinic visits	Patients with ≥1 CDAI score	02/2016-12/2016	75%
LTBI screening	Patients age ≥18 years with RA initiating new biologic DMARD** with ≥2 clinic visits	Patients with documented PPD or IGRA at least 12 months prior to biologic DMARD initiation or history of prior TB treatment	01/2015-12/2017	90%
Reproductive health counseling	Women age 18-45 years on potentially teratogenic medications§ with ≥2 clinic visits	Patients with ≥1 counseling session	03/2016-12/2016	50%

^*Immunosuppressive medications included biologic agents (abatacept, adalimumab, anakinra, certolizumab, etanercept, golimumab, infliximab, rituximab, tocilizumab) and non-biologic medications (tofacitinib, azathioprine, cyclophosphamide, cyclosporine, gold, leflunomide, methotrexate, minocycline, penicillamine, sulfasalazine).

^**Biologic DMARDs included abatacept, adalimumab, anakinra, certolizumab, etanercept, golimumab, infliximab, and tocilizumab.

^§Potentially teratogenic medications included methotrexate, mycophenolate, leflunomide, cyclophosphamide, gold, minocycline, adalimumab, certolizumab, etanercept, golimumab, infliximab, abatacept, rituximab, anakinra, tocilizumab and tofacitinib. PCV13 denotes 13-valent pneumococcal conjugate vaccine; CDAI, Clinical Disease Activity Index; LTBI, latent tuberculosis infection; DMARD, disease-modifying antirheumatic drug; PPD, purified protein derivative; IGRA, interferon-gamma release assay.

Interventions were planned and executed using the IHI’s Model for Improvement with Plan, Do, Study, Act (PDSA) methodology, in which small-scale cycles of change are implemented in a consecutive fashion to improve care(15, 16). All providers working in the clinic were included in the intervention, including attending physicians, rheumatology fellows, nurses and medical assistants. UCSF medical students had a particularly active role in planning, executing and evaluating QI interventions as part of their medical school curriculum. Patients were also involved in the early planning phases of these quality improvement interventions, especially in providing feedback on workflow and developing educational materials. Data from the electronic health record (EHR) was extracted on a quarterly basis to evaluate performance on the quality measures. This investigation was considered exempt from IRB approval because it qualified as a quality improvement project.

Pneumococcal vaccination (PCV13) quality measure

Vaccination with the 13-valent pneumococcal conjugate vaccine (Prevnar, or PCV13) is recommended by the Advisory Committee on Immunization Practices for all adults older than 18 years with immunocompromising conditions(17). Prior to this study, the clinic lacked a standardized protocol to ensure administration of this vaccine.

Patients

We measured the proportion of patients age ≥18 years on immunosuppressive medications with documented PCV13 vaccination from February 2015 to March 2017. Immunosuppressive medications included biologic agents (abatacept, adalimumab, anakinra, certolizumab, etanercept, golimumab, infliximab, rituximab, tocilizumab) and non-biologic medications (tofacitinib, azathioprine, cyclophosphamide, cyclosporine, gold, leflunomide, methotrexate, minocycline, penicillamine, sulfasalazine). Our target for this measure was 80% of eligible clinic patients by March 2017.

Interventions

Several PDSA cycles were designed to improve vaccination rates. First, in February 2015, a multidisciplinary conference was held educating providers and clinic staff on the role of PCV13 vaccination. Second, in April 2015, a coordinated effort led by ancillary staff helped identify patients in need of vaccination on a weekly basis. Medical assistants also identified eligible clinic patients from the EHR using chart review prior to each clinic session, and this information was used to flag all unvaccinated patients. Of note, the decision to order the vaccine was left to physicians (this was intentional given that some patients decline or have contraindications to vaccination).

RA disease activity monitoring (CDAI) quality measure

In order to promote a treat-to-target approach in RA(18), regular disease activity monitoring with a validated tool has been endorsed by the ACR and NQF as a critical quality metric(19). Before this study, there were three separate processes related to disease activity measurement. Providers received an RA-specific paper note with most disease activity data elements (e.g., a homunculus for joint counts). Patients were separately given a paper document to record their global assessments. Lastly, providers could document disease activity in the EHR, although there was no way to document this information in a structured EHR field (i.e., it could be included in the history, physical exam or assessment sections of the clinic note). We chose the Clinical Disease Activity Index (CDAI) given its ease-of-use and inclusion of data elements that can be obtained within a single visit(18).

Patients

We measured the proportion of patients with RA age ≥18 years with at least one CDAI score between February and December 2016. Our target was 75% of eligible patients by December 2016.

Interventions

Several PDSA cycles were deployed. First, in February 2016, the EHR was reconfigured with assistance from information technology (IT) staff to allow for capture of a numerical CDAI score in a structured template. Medical assistants were trained to merge this electronic CDAI template to EHR notes prior to all RA patient encounters. In April 2016, additional one-on-one provider training on the CDAI template was provided. A final PDSA cycle followed in June 2016, when physicians began receiving their individual performance rates on a quarterly basis during pre-clinic conferences.

LTBI screening quality measure

Screening for latent tuberculosis infection (LTBI) was endorsed by the ACR and NQF as a critical quality measure for RA patients newly started on biologic disease-modifying anti-rheumatic drugs (DMARDs)(19). Prior to this study, there was no standardized way of tracking TB screening history at the rheumatology clinic. This was compounded by the fact that the hospital TB clinic, which archives the LTBI treatment history of many San Francisco residents, was not fully integrated with the hospital-wide EHR.

Patients

We measured the proportion of patients with RA age ≥18 years who had documented TB screening or history of prior LTBI therapy prior to initiating new biologic DMARDs between 2015–2017. DMARDs included abatacept, adalimumab, anakinra, certolizumab, etanercept, golimumab, infliximab, and tocilizumab. Our target was 90% of eligible patients by the end of 2017.

Interventions

PDSA cycles were implemented between January and April 2015. First, a patient safety checklist was introduced to standardize workflow for LTBI screening (see Figure 1). This paper document included patient information, intended biologic therapy, assessment of TB status and history of prior LTBI treatment. TB status could be confirmed with a purified protein derivative (PPD) or interferon-gamma release assay (IGRA) result within 12 months of biologic initiation. If patients had a positive screen or a history of prior LTBI, they were referred to the hospital’s TB clinic for evaluation. Second, a structured template for TB history was developed in the EHR with assistance from IT staff, which was used to track adherence to the quality measure.

Figure 1: Patient safety checklist for LTBI screening quality measure.

Patient safety checklist developed for LTBI screening prior to initiation of biologic DMARDs. Hepatitis B screening was also incorporated into the form. Documents were to be completed by nurses and then scanned into the EHR. PPD denotes purified protein derivate; LTBI, latent tuberculosis infection; DMARD, disease-modifying antirheumatic drug.

Reproductive health counseling measure

Many women with rheumatic diseases receive potentially teratogenic medications during their reproductive years. Attention to this important aspect of clinical care has been proposed as a quality measure in SLE(20) and other rheumatologic conditions, given the suboptimal receipt found in prior studies(21, 22). There was no standardized way of documenting contraception counseling in the clinic prior to this study.

Patients

We measured the proportion of women age 18–45 years who had received standardized contraception counseling at least once between March and December 2016 and were taking medications with either high teratogenic potential or with unknown or potential pregnancy risks. Medications included methotrexate, mycophenolate, leflunomide, cyclophosphamide, minocycline, adalimumab, certolizumab, etanercept, golimumab, infliximab, abatacept, rituximab, anakinra, tocilizumab and tofacitinib. Our target was 50% of eligible patients by the end of December 2016.

Interventions

PDSA cycles were deployed over the course of March 2016. First, IT staff added a teratogen counseling template in the EHR which provided a simple yes/no/NA option to track counseling events and offered a standardized template for providers to fill regarding the personalized content of their counseling; use of this template was used to track adherence to the quality measure. Second, handouts in English, Spanish and Chinese were developed with patient feedback to improve education regarding family planning. Third, a paper consent form requiring physician and patient signatures was created to reinforce discussions regarding the teratogenicity of medications (see Figure 2A and 2B). Physicians were trained in the use of the educational materials, paper consent forms and EHR templates. A clinic nurse generated weekly lists of eligible patients, which medical assistants then used to merge the teratogen counseling template into the appropriate clinic note; the paper documents described above were also attached to the physical charts of eligible patients.

Figure 2: Patient education and counseling forms for reproductive health counseling quality measure.

(A) Educational handouts provided to women of reproductive age receiving potentially teratogenic medications. These were made available in English, Spanish and Chinese. (B) Counseling form to be signed by the physician and patient to document discussions regarding medication toxicity and reproductive health. Document was to be scanned into the EHR after completion.

Data analysis

For the analysis, we included eligible patients with at least two rheumatology clinic visits. The outcome was overall performance on the quality measures, defined as the proportion of eligible patients receiving recommended care by the end of the respective measurement period (see Table 1). Baseline performance rates prior to study onset were only available for the PCV13 and LTBI screening measures; baseline documentation and therefore performance data for the CDAI and reproductive health counseling measures were inconsistent and not included in this study. There were enough discrete data points for the PCV13 and CDAI quality measures to construct Shewart charts to analyze performance over time. Given subgroups of varying size, p-charts were constructed, which depict quality measure performance over time in relation to the average of plotted points (also known as the center line), and the expected range of variation in a stable healthcare process (bounded by upper and lower control limits)(23). Monthly and biweekly time intervals were chosen for the PCV13 and CDAI measures, respectively, in part to maximize the ability of charts to detect significant changes in healthcare delivery(24). Raw data was extracted from the EHR and analysis was conducted using SAS version 9.4 and the QI Macros application version 2017.11 for Excel.

Results

Clinic population

During the study period from 2015 to 2017, 1,205 patients with at least two clinic visits were seen in the clinic; 547 (45%) patients were eligible for at least one of the four quality measures (Table 2). The mean age was 56 (±14, SD), and 856 (71%) patients were female. The majority of patients identified as a racial or ethnic minority, with 150 (12%) patients identifying as white. Almost half (47%) of the patients reported a language preference for their encounters other than English.

Table 2:

Sociodemographic characteristics of the rheumatology clinic population with at least two clinic visits 2015-2017.

Characteristic*	Clinic population (n=1205)
Age (mean ± SD)	56 ± 14
Female	856 (71)
Race/ethnicity
Asian	381 (32)
Black, non-Hispanic or Latino	131 (11)
Hispanic or Latino	424 (35)
White	150 (12)
Other	119 (10)
Primary language**
English	635 (53)
Spanish	300 (25)
Cantonese	131 (14)
Other	139 (12)
Eligible for quality measure
PCV13	505 (42)
CDAI	295 (24)
LTBI screening	77 (6)
Reproductive health counseling	57 (5)

^*n (%) unless otherwise noted.

^**Preferred language identified by patients for their clinical encounters. SD denotes standard deviation; PCV13, 13-valent pneumococcal conjugate vaccine; CDAI, Clinical Disease Activity Index; LTBI, latent tuberculosis infection.

PCV13 measure

There were a total of 505 patients seen in the rheumatology clinic who were eligible for PCV13 vaccination with a mean of 159 eligible patients each month. At the beginning of the measurement period in February 2015, only 21 (15%) eligible patients taking immunosuppressive medications had documented vaccination with PCV13. This rose to 74% by 12 months after implementation. By the end of the measurement period, 392 (78%) patients received PCV13 vaccination, which did not meet our target of 80%. Figure 3 depicts a Shewart chart of quality measure performance over time. Presence of greater than eight data points above the upper control limit shows evidence of improvement in performance over time(25).

Figure 3: Shewart chart depicting performance on the 13-valent pneumococcal conjugate vaccine quality measure over time.

Shewart chart depicting the proportion of patients vaccinated with PCV13 by month of intervention. Red vertical lines correspond to PDSA cycle 1 in February 2015 (multidisciplinary conference educating clinic staff on vaccination) and PDSA cycle 2 in April 2015 (identification of eligible patients). The blue horizontal line indicates the center line, while the black lines above and below the center line indicate the upper and lower control limits, respectively. PDSA denotes ‘Plan, Do, Study, Act’; PCV13, 13-valent pneumococcal conjugate vaccine.

PDSA cycles for this measure identified the relevant patient population and enhanced knowledge and awareness among physicians and other clinic staff of PCV13 vaccination indications. We observed that success in this quality measure was largely due to engagement of many members of the interprofessional care team with diverse clinical roles. Clinic nurses reviewed lists of unvaccinated patients on a weekly basis to flag patients in need of pneumococcal vaccination. Medical assistants also integrated review of vaccination history into their routine clinical duties. Physicians responded to these notices by increasing the number of orders for vaccines. These changes to clinic workflow reinforced the education that was given to clinic staff and persist to this day.

CDAI measure

There were 295 eligible RA patients from February to December 2016, who contributed 1003 clinic visits with a mean of 76 eligible patients per month. In the first two weeks of the intervention in February 2016, 7 (19%) patients had documented CDAI scores in the EHR. Performance improved to 74% by the third month following the first PDSA cycle. The number of RA patients with at least one CDAI documented by the end of the measurement period was 273 (93%), exceeding our target goal. The Shewart chart in Figure 4 shows the presence of a data point above the upper control limit and at least eight data points above the center line, both evidence of significant improvement in care delivery(25).

Figure 4: Shewart chart depicting performance on the Clinical Disease Activity Index quality measure over time.

Shewart chart depicting the proportion of patients with CDAI completion in biweekly intervals. Red vertical lines correspond to PDSA cycle 1 in February 2016 (introduction of EHR CDAI template); PDSA cycle 2 in April 2016 (one-on-one provider training); and PDSA cycle 3 in June 2016 (physician feedback). The blue horizontal line indicates the center line, while the black lines above and below the center line indicate the upper and lower control limits, respectively. PDSA denotes ‘Plan, Do, Study, Act’; CDAI, Clinical Disease Activity Index.

We observed sustained success in this quality measure, in large part due to efforts to incorporate disease activity monitoring into routine clinical practice. The EHR structured template enhanced the ability to record this information in a standardized way and to track performance. Medical assistants also prepared EHR charts prior to RA encounters by merging the electronic CDAI template to provider notes. Peer reporting and one-on-one physician education on use of the CDAI template further complemented interventions to improve performance. These interventions are still in use at the present time.

LTBI screening measure

There were 77 patients started on biologic therapies during the study period from 2015 to 2017. Prior to the intervention, only 23 (56%) of the patients initiating biologic therapies had documented TB screening results. By the end of the measurement period, 70 (91%) patients had been screened for LTBI or had a documented history of prior LTBI, which exceeded our target goal.

In the first series of PDSA cycles, we created a patient safety checklist which required physician and nurse endorsement to proceed with biologic initiation (see Figure 1). Importantly, the checklist prompted providers to test patients for or investigate prior history of LTBI. We discovered that many patients flagged as eligible for TB testing had already been treated for LTBI by the TB clinic in the past, however this data was not easily accessible to rheumatology clinic providers. Standardized documentation in the EHR remains challenging given the TB clinic’s separate health record system incompatible with that of the outpatient clinics; currently, this aspect of the quality measure is not consistently pursued. Clinic workflow does continue to employ the patient safety checklist, which is scanned into the EHR, as part of the biologic initiation process.

Reproductive health counseling measure

There were 57 women of reproductive age eligible for this quality measure between March and December 2016. By the end of the first four months of the intervention, 18 (78%) patients had documented counseling. By the end of the measurement period in December 2016, 26 (46%) patients had been counseled at least once in the prior year, thus not meeting our 50% target.

Early success in this quality measure was not sustained through the study period. Weekly lists of eligible women were generated by nurses prior to clinic sessions, and medical assistants included the appropriate EHR template and printed contraception counseling materials with eligible patients’ charts—practices that continue to this day. There are several possible reasons for under-performance in this measure, including burdensome electronic and paper documentation and suboptimal patient and physician education.

Discussion

In this study, we report on our multi-faceted quality improvement program in a safety net rheumatology clinic serving a racially, ethnically and socioeconomically diverse patient population. Using the IHI’s Model for Improvement, we were able to significantly improve processes of care on all quality measures examined while achieving sustained improvements in three of the four areas we addressed.

Introducing practice-specific EHR templates were crucial to our QI interventions, and we found that they were most successful when they enhanced existing structures for clinical care. For instance, prior to implementation of the CDAI quality measure, a printed RA-specific clinic note was already in circulation to help providers collect information for disease activity assessment. The creation of a simple-to-use EHR template complemented this practice by offering a reliable means of gathering CDAI elements and tracking response to therapy over time, replacing the prior practice of reviewing handwritten notes. The CDAI template in the EHR was a simple structured field which only documented numerical information, and could likely be implemented in many clinical settings(26). This exemplifies the importance of understanding healthcare context to ensure that EHR-based interventions are successful(27, 28).

QI also helps manage day-to-day practice variation in busy clinical settings. Two ways to do this are improving the categorization of patients into sub-groups in need of certain services(29, 30) and engaging non-physician clinic team members(27, 29, 31, 32). For instance, the PCV13 quality measure was bolstered by the active identification of eligible patients by two team members: first by nurses who generated lists of unvaccinated patients from the EHR, and second by medical assistants who reviewed vaccination history independently. Although seemingly redundant, these efforts to actively categorize patients across the entire team helped engage clinic staff under unified goals.

A strength of our study was the involvement of diverse stakeholders in the clinic. Providers were crucial to the selection of appropriate quality measures. We engaged patients in providing feedback on clinic workflows and development of educational materials in the early stages of QI planning, especially for the reproductive health counseling intervention. Medical students were intimately involved in the development and implementation of QI measures as part of their medical school curriculum. We believe that the success and durability of most interventions described above was in large part due to this team-based approach. To this day, the clinic uses the workflows established by these quality interventions, despite significant fluctuations in workforce (in particular among trainees).

We observed that QI can exert powerful positive downstream effects even if primary aims are not achieved(33). For instance, in the LTBI screening quality measure, success in creating an electronic template for TB history was limited by an inability to access public health-level data. Even though a structured TB field in the EHR was not as successful as hoped, this study inspired the creation of a patient safety document for biologic DMARD initiation, a document that is still in use today, reflecting the utility of checklists for high-risk interventions(34). Thoughtful analysis of QI data can also shed light on care processes when pre-specified benchmarks are not achieved. In the PCV13 measure, for instance, performance at the end of the measurement period was just shy of our target of 80%, but construction of a Shewart chart showed significant improvement in measure performance over time.

We faced challenges in achieving our target in the reproductive health measure despite robust initial performance. Counseling discussions may not have been prioritized in busy clinic visits, and patients may not have felt empowered to raise the issue of reproductive health during visits. Suboptimal provider and patient education may have played a role. Alternatively, under-performance may simply reflect failure to use the EHR template rather than true low performance, although this was not formally measured.

Inconsistent documentation of baseline quality measure performance in the CDAI and reproductive health counseling measures may have limited our interpretation of data for these measures. The Shewart charts for the PCV13 and CDAI quality measures would also have benefitted from pre-intervention data collection to document a stable healthcare process(24). In addition, balancing measures were not formally measured to ensure our interventions were efficient and not associated with unintended consequences. Patients were not systemically surveyed to examine how these measures affected their experiences of care. Lastly, other safety net clinics may not have access to certain resources we employed in this study, including the PIP incentive program, as studies have shown an association between improvements in care and receipt of financial rewards(35).

In conclusion, the Model for Improvement effectively improved performance on quality measures in prioritized clinical areas in a safety net rheumatology clinic. Through a multidisciplinary approach focusing on systems redesign, we achieved durable improvements in key clinical practices by enhancing workflow, engaging non-physician providers and managing practice variation. QI can achieve sustainable improvements in healthcare among diverse patients with rheumatologic conditions.