Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Oct 1.
Published in final edited form as: Rheumatol Int. 2017 Aug 29;37(10):1603–1610. doi: 10.1007/s00296-017-3804-4

Leveraging the electronic health record to improve quality and safety in rheumatology

Gabriela Schmajuk 1,2, Jinoos Yazdany 1
PMCID: PMC5693630  NIHMSID: NIHMS902805  PMID: 28852846

Abstract

During the last two decades, improving the quality and safety of healthcare has become a focus in rheumatology. Widespread use of electronic health records (EHRs) and the availability of digital data have the potential to drive quality improvement, improve patient outcomes, and prevent adverse events. In the coming years, developing and leveraging tools within the EHR will be the key to making the next big strides in improving the health of patients with rheumatoid arthritis and other rheumatic diseases, including building EHR infrastructure to capture patient outcomes and developing automated methods to retrieve information from free text of clinical notes.


The increasing use of electronic health records (EHRs) and the availability of digital data have created new opportunities to improve the quality and safety of healthcare in rheumatology. Most efforts have focused on rheumatoid arthritis (RA) given the relatively high prevalence and morbidity associated with the disease. Over the last 20 years, we have seen the transformation of treatment for many rheumatic diseases, particularly for RA. Scientific discoveries have paved the way for new biologic and targeted drugs that control disease when conventional treatments lack efficacy. The pain and disability associated with many rheumatic diseases can now be managed or prevented in a great majority of patients. However, care has also become more complex, with primary and specialty care often distributed across multiple providers. There is also a growing focus on incorporating patients’ needs and preferences into clinical decision-making. Providers must work to empower patients with the knowledge to understand their disease, adhere to therapy, and receive evidence-based therapy adjustments and monitoring. Studies show that there is room for improvement in creating safe and patient-centered health care in rheumatology.[1,2,3,4] However, the pieces are in place to continue to improve health care for rheumatic diseases over the next decade, including the health information technology (IT) infrastructure provided by electronic health records (EHRs); the availability of valid, responsive, and feasible patient-reported outcomes (PROs) and disease activity measures for clinicians to use in clinical settings; and evidence-based standards to guide monitoring and safety strategies for patients receiving high-risk immunosuppressant drugs.

While there are many opportunities to advance quality and safety in rheumatology using EHRs, [5] below we outline two areas where work has begun and discuss approaches to developing these areas further.

Improving disease outcomes in RA

The primary goal of treating patients with RA is to maximize long-term health-related quality of life through control of symptoms, prevention of structural damage, normalization of function, and maintained social participation.[6] Efforts to advance quality of care in RA have focused on collecting and improving outcome measures. The routine collection of outcome measures forms a framework for benchmarking performance across providers and practices and allows for the creation of a learning health care system, in which information about outcomes and performance is fed back to patients and providers to continuously improve quality of care and to fuel new discoveries.[7]

Measuring PROs and disease activity measures during the course of clinical care allows clinicians to employ a “treat-to-target” strategy, which has been shown to reduce morbidity and mortality for patients with RA. Several important clinical trials have demonstrated that patients managed with treat-to-target approaches had a higher likelihood of achieving remission and experiencing a reduction in radiographic joint erosions, improved physical function, and better quality of life.[8,9,10,11] In a real-world setting, an observational study involving 1,297 patients showed that achievement of recommended disease targets was associated with improved physical function, health-related quality-of-life, and reduced hospitalizations.[12] Routine measurement of physical function helps determine if a key treatment goal–maintaining functional capacity - has been achieved. Achieving lower disease activity has also been associated with further improvement of physical function.[13] When incorporated into the health care visit, PROs can stimulate conversations between patients and providers that lead to shared decision-making and result in more individualized care.[14]

In an effort to encourage the use of treat-to-target strategies, in 2014, the National Quality Forum (NQF) endorsed two quality measures that address the collection of PROs in RA: (1) an annual recorded measure of physical function and (2) a disease activity measure recorded in at least 50% of encounters.[15] American pay-for-performance programs such as the Medicare Incentive Payment System (MIPS) incentivize performance on quality measures. MIPS quality reporting for rheumatologists can be fulfilled by participating in qualified clinical data registries such as the American College of Rheumatology’s national patient registry (known as the Rheumatology Informatics System for Effectiveness or RISE).[16]

Already, many rheumatology practices are using PROs to guide chronic care management, to monitor symptoms, to engage RA patients in disease tracking and shared decision-making, and for national quality reporting and benchmarking. However, the collection and utilization of outcome measures remains inconsistent, with some practices not collecting them at all, and others collecting them inconsistently or as unstructured data in clinical notes. Data from the American College of Rheumatology’s RISE registry found that among the 178,931 individuals enrolled in the registry with RA, just over 50% have a functional status PRO or RA disease activity score recorded.[17] (Rates of PRO collection outside of RISE practices are reported to be similar or lower.[18,19,20]) A significant number of these PROs are documented as free text within clinical notes or as images in scanned documents. This is problematic because detection and extraction of PRO and disease activity measures from these unstructured fields is labor-intensive and error-prone.[21]

How do we increase the use of RA outcome measures in clinical practice and at the same time increase documentation of these outcomes in the EHR? These two concepts are related in a “virtuous cycle,” for once outcomes are reliably documented and easily tracked, they become much more useful for providers and patients, and are then more likely to be collected.[22] Making PROs and disease activity measures easier to collect, and communicating their trajectories with patients, creates a treat-to-target focus for the clinical visit. Below we discuss two approaches to improving outcome measure documentation – 1. Building EHR infrastructure to capture outcome measures in structured fields; and 2. Developing automated methods to retrieve these pieces of information from the text of clinical notes.

Building EHR infrastructure to capture PROs and disease activity measures in structured fields

One approach to making PROs and disease activity measures easier to collect and track involves developing methods to capture key data elements in EHR structured fields (just as vital signs or lab results are). For example, in order to document disease activity, a homunculus might be built within the EHR where tender and swollen joints can be denoted by providers, generating structured data for joint counts. Patients can input Patient Reported Outcome Measurement Information System (PROMIS®) survey items or patient reported global disease activity assessments via a computer portal in the waiting room or online before their visit. The EHR can calculate disease activity scores by compiling tender and swollen joint count data from the homunculus, patient and physician global assessment scores, and laboratory results (if needed). Scores can be documented in flow sheets within the EHR, which are accessible as structured data and subsequently can be used in a provider-facing dashboard. EHR enhancements could enable real-time trending of PRO scores over time, including the current time point during the clinic encounter. Additional features could include the display treatment targets to indicate whether a patient is in the “target zone” (remission or low disease activity) and illustrate how medication changes relate to PROs over time. Elements of such a tool could also be adapted for display to patients or as part of an after-visit summary, to enhance shared-decision making and patient engagement. Outcome data could also be used to provide clinical decision support or guide population health management. This approach has been successful in a large U.S. rheumatology practice.[23]

Despite these potential benefits, structured capture of PROs and disease activity measures may come at the cost of negative or unintended consequences.[24] For example, providers may find it onerous to click through an EHR-based homunculus when their workflow using a paper-based homunculus was faster or allowed for more eye contact with the patient. One key to minimizing burden on clinicians is to use principles of human-centered design to build the EHR for seamless input of such data, including, for example, patient-specific default settings on the homunculus, or order templates that ease data input.[25,26] The development of new digital infrastructure should aim to make care and its documentation simpler and more standardized, including keeping data input as effortless as possible, while attempting to identify and address unintended downstream effects.

Developing automated methods to retrieve PROs and disease activity measures from the text of clinical notes

Many rheumatologists do not have the health IT or EHR vendor support to record PROs or disease activity measures as structured data, and some providers prefer to document as free text in clinical notes. Information written into clinical notes and not input into a structured field is more challenging to extract from the EHR. Current systems to manually retrieve this information through chart review are extremely time- and resource-intensive.

One potential solution is to use automated methods to transform unstructured information into structured data. Natural language processing (NLP) or machine learning algorithms are tools designed to extract information from the text of clinical notes. NLP approaches attempt to interpret human language by accounting for the hierarchical structure of language to combine words into phrases, phrases into sentences, and sentences into concepts in information extraction algorithms. Machine learning approaches incorporate free text and other data as part of statistical models to make predictions about the presence or absence of an outcome or patient characteristic. Both approaches have shown promise in recent studies to retrieve RA diagnoses and outcome measures from the text of clinical notes.[27,28,29,30] Outside of rheumatology, researchers have developed text data extractors for dementia diagnoses and cancer staging, urinary incontinence-related PROs, identification of out-of-network emergent care encounters, disease phenotyping, and extraction of documentation of advanced directives.[31,32,33,34] This work has shown that important healthcare outcomes are being captured in EHRs as free text and that although challenges exist, NLP and machine learning methods may be increasing feasible options for accurately and efficiently identifying health outcomes.

Although automated methods to extract information from the text of clinical notes would minimize the burdens of quality measurement on providers, significant challenges exist in implementing such approaches. Heterogeneity in clinical text can make automated analysis difficult - clinical text can range from well-structured report summaries to loosely organized or telegraphic text, and all can vary in accuracy based on the individual who input the data into the EHR.[27,35,36] Additional challenges include the handling of misspellings, unusual syntax, ambiguous abbreviations, and other clinic-, healthsystem-, regional-, or EHR-specific variations. Nevertheless, there is growing consensus that automated information extraction methods augment the accuracy of case-detection algorithms compared to structured text alone, findings that can likely be extended to quality measurement examples as well. [37]

Improving medication safety for patients receiving high-risk immunosuppressants

While improving outcomes is a top priority for quality improvement efforts, ensuring patient safety is equally important. Over the past two decades, RA management has become progressively more complex due to the increased number of medications available (including biologic agents and biosimilars) as well as multidisciplinary care that is often fractured across different providers. These developments have made medication safety increasingly important as a focus for quality improvement efforts. Unfortunately, innovations to ensure safe prescribing, monitoring and use of the high-risk medications use in RA have not kept pace with these changes, and reports of preventable adverse events are increasing.[4] Examples include fulminant hepatic failure from hepatitis B in patients taking B-cell depleting therapies without appropriate preventive measures, [38,39] reactivation of latent tuberculosis in patients taking anti-TNF therapies, [40,41] and unintended pregnancies in women using teratogenic medications who may not have received adequate contraceptive counseling.[42,43,44,45]

The full power of data available through EHRs has not yet been harnessed to shed light on ambulatory patient safety in rheumatology. Most studies of patient safety still employ manual chart review, which is resource intensive and costly. Quality measurement most often involves post-hoc review of administrative data. However, patient safety process errors such as (1) failure to institute preventive practices that reduce adverse medication events, (2) missed safety monitoring for patients using high-risk immunosuppressive medications, or (3) absence of systems to manage abnormal results from medication toxicity monitoring are all potentially measurable through the EHR, as patients could be assessed in real-time for receipt of recommended care. Wise use of EHR data could allow us to build tools to prevent errors before they occur. The EHR should be able to offer important and targeted warnings and safety-oriented clinical decision support to providers. In addition, patient safety quality measures can serve as benchmarks across providers to compare performance, and stimulate quality improvement.

There are several examples of areas where clinical decision support can improve patient safety in rheumatic diseases. For instance, screening and prophylaxis for Hepatitis B (and antiviral prophylaxis for patients who screen positive) are recommended prior to initiation of rituximab in order to reduce the incidence of Hepatitis B reactivation.[38,46,47,48,49,50,51] With advanced clinical decision support, an automated program within the EHR could potentially scan the EHR to find any results for Hepatitis B testing as a provider orders rituximab. If the Hepatitis B test were absent or positive, the EHR could offer a trigger warning or pop-up tool to correct the omission in real time. Individualized provider dashboards could report the proportion of patients receiving rituximab who had appropriate Hepatitis B screening or prophylaxis and show the average performance on this measure across the health system or region for comparison. Trials of such clinical decision support tools have shown that they can improve patient safety by, for example, increasing prescription of calcium and vitamin D supplementation in patients receiving glucocorticoids or improving rates of herpes zoster vaccination.[52,53]

Although automatic extraction of safety data within the EHR holds promise, understanding potential safety errors and adverse events is a complex task that requires detailed clinical and operational knowledge. In the rituximab/Hepatitis B example above, it would be critical for the EHR to be able to retrieve all existing laboratory results, whether they were stored as structured data, text in clinical notes, or scanned images or pdfs, depending on the health system; this would avoid a slew of “false positive” alerts (i.e., alerting the provider that Hepatitis B tests were missing when in fact, they were imported as scanned outside laboratory records). A naïve EHR decision support tool would incorrectly prompt providers to order these labs again if labs were not captured in structured fields. Since interoperability of the EHR across health systems, laboratories, and pharmacies remains a challenge, the near-term solution is to create systems that can retrieve data from sources such as clinical notes or scanned documents. Systems need to be designed to avoid the burden of data input by providers while at the same time minimizing workflow disruptions and incorrect or irrelevant clinical decision support alerts, which can lead to alert fatigue.[54]

Other examples of areas where EHR enhancements could address safety issues among patients with rheumatic diseases are listed in Table 1.

Table 1.

Potential EHR-related enhancements to safety measures applicable to patients with rheumatoid arthritis.

Medication Example safety failure Traditional approach to quality measurement EHR-enhanced approach to quality measurement
Rituximab Hepatitis B status not determined; known Hepatitis B positive patients not given prophylaxis prior to rituximab start[49]
  • Retrospective assessment based on orders for rituximab and lab results for Hepatitis B tests

  • Incorporates Hepatitis test results from clinical notes

  • Incorporates information scanned from outside hospital records

  • Real-time clinical decision support provides pended order for lab test or prophylactic antibiotic when patient with a missing or positive Hepatitis B is prescribed rituximab

High-risk immuno-suppressants Pneumocystis pneumonia (PCP) prophylaxis not prescribed prior to drug start[62,63]
  • Retrospective assessment based on orders for immunosuppressants and orders for antibiotics

  • Enables identification of patients receiving “high-risk” drugs such as cyclophosphamide or rituximab

  • Incorporates information from allergies and clinical notes to assist in selection of appropriate prophylactic antibiotic

  • Real-time clinical decision support provides pended order for prophylactic antibiotic when patient receives immunosuppressant

Anti-TNFs, abatacept, tocilizumab, tofacitinib Tuberculin Skin Test (TST) status not determined; known, TST+ patients not given treatment prior to drug start[64]
  • Retrospective assessment based on orders for immunosuppressant, PPD and Quantiferon gold results, appropriate tuberculosis treatment

  • Incorporates PPD results from clinical notes

  • Incorporates information from scanned outside hospital records (regarding prior PPD, chest radiograph results, tuberculosis treatment)

NSAIDs Acid reducing medication not prescribed for high-risk patients[65]
  • Retrospective assessment based on orders for NSAID and acid reducer

  • Incorporates data regarding risk factors from problem list and clinical notes to identify high-risk patients

  • Real-time clinical decision support provides pended order for prophylactic acid reducer when high risk patient receives NSAID

Methotrexate, leflunomide LFT monitoring not performed every 3 months[66]
  • Retrospective assessment based on drug orders and lab results

  • Incorporates information scanned from outside hospital results

  • Flags labs that are meaningfully abnormal or reflect a trend as opposed to “above the upper limit of normal”

  • Real-time triggers when patient has missed labs for > 5 months

Methotrexate Folic acid not prescribed alongside methotrexate[67]
  • Retrospective assessment based on drug orders

  • Clinical decision support provides pended orders for folic acid whenever methotrexate is prescribed

Hydroxychloroquine Dosing above that recommended by the American Academy of Ophthalmology[68]
  • Retrospective assessment based on drug orders and weight, if present

  • Clinical decision support provides suggested dosing based on patient’s most recent weight

Hydroxychloroquine Missed retinal toxicity screening after 5 years of use, or sooner in high-risk patients[68]
  • Retrospective assessment based on drug orders and ophthalmology procedures or results

  • Incorporates data regarding risk factors from problem list and clinical notes to identify high-risk patients

  • Real-time clinical decision support provides pended ophthalmology referral after 5 years of use or sooner for high-risk patients

Cyclophosphamide, leflunomide, or other teratogenic drug Female of child-bearing age missing contraception
  • Retrospective assessment based on drug orders

  • Incorporates information from problem list and medications to identify patients of childbearing age at risk for pregnancy

  • Real-time clinical decision support suggests possible contraceptive options

Additional areas for future work

Quality reporting, performance benchmarking and risk adjustment

One goal of quality measurement is to benchmark performance across health systems or providers. Patients want to be able to examine outcomes across providers to make decisions about where they should receive care; doctors want to benchmark their performance and see how their populations of patients are doing relative to others; the information may also be useful for payers and others to understand who is a high-versus lower-quality provider. But comparing outcomes across health systems or providers usually requires risk-adjustment of those outcomes, since providers with fundamentally different populations of patients often have different aggregate outcomes (e.g. a geriatrician with patients of ages 75–99 will, on average, have worse functional status scores compared to an internist whose panel includes patients aged 18–99). In the case of RA outcomes, consensus is lacking around whether or how to adjust for patient factors that affect outcomes outside of the control of the provider. Although process measures (such as prescription of a DMARD, or collection of a disease activity measure) avoid the issue of patient factors or “case mix” adjustments, health outcomes such as pain, functional status, and disease activity are more important to patients and more meaningful on a societal level. Initial work is underway to develop a risk-adjusted outcome measure for RA.[55]

Unintended consequences of performance measurement and clinical decision support

As we develop meaningful measures around quality and safety, we need to search aggressively for unintended consequences of new quality measures, and work to construct balancing measures to capture these consequences.[56] This applies not only to local quality improvement efforts but also at the health system level. On the care delivery side, the time it takes to collect and accurately report measures may lead to slower provider workflows and increasing wait times to see a rheumatologist. On the health outcomes side, one potential unintended consequence of a “treat-to-target” approach to minimizing disease activity for RA patients is the potential for higher incidence of infection, as more patients receive increasing amounts of immunosuppression. To understand the complex effects of quality measurement, these aspects should be measured and accounted for when defining high quality, high value care.

Pragmatic trials that test the effects of quality improvement interventions and clinical decision support on health outcomes

With the implementation of new quality and safety measures, we need to build an evidence base showing which strategies are most successful in improving patient outcomes, minimizing adverse events, and maximizing patient and provider satisfaction. Traditional randomized controlled trials can be difficult in the experimental evaluation of quality improvement interventions because of cost, time, and difficulty in randomizing different providers or health systems. Quasi-experimental study designs aim to evaluate interventions without using randomization. These types of studies use both pre-intervention and post-intervention measurements to compare outcomes of interest before and after an intervention (e.g., a new EHR-based clinical decision support tool) is introduced, and are useful when evaluating new health system interventions. [57,58,59] For example, quasi-experimental studies have quantified the usefulness and impact of clinical decision support tools to curb antibiotic prescribing in acute respiratory infections (a process improvement)[60] and standardized analgesia protocols on post-operative pain (an outcomes improvement).[61] In the United States, Medicare has begun testing and evaluating health care interventions through demonstration projects funded by an institute developed expressly for this purpose, the Center for Medicare & Medicaid Innovation.

Conclusions

In summary, over the last two decades, quality of care and patient safety have become top priorities in rheumatology. Widespread use of EHRs and the availability of digital data have the potential to drive quality improvement, improve patient outcomes, and prevent adverse events. In the coming years, developing and leveraging tools within the EHR will be the key to making additional advances in improving the health of patients with RA and other rheumatic diseases.

Acknowledgments

Funding: This work is supported by the Agency for Healthcare and Research Quality [R01 HS024412] and the National Institutes of Health [K23 AR063770 (GS)]. Drs. Yazdany and Schmajuk are also supported by the Russell/Engleman Medical Research Center for Arthritis and an independent research grant from Pfizer. Dr. Yazdany is supported by the Robert L. Kroc Chair in Rheumatic and Connective Tissue Diseases (I). The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality or National Institutes of Health.

Footnotes

Compliance with Ethical Standards:

Conflict of Interest: Dr. Schmajuk has received an investigator initiated award from Pfizer. Dr. Yazdany has received an investigator initiated award from Pfizer.

Ethical approval: This article does not contain any studies with human participants performed by any of the authors.

References

  • 1.Dragoi RG, Ndosi M, Sadlonova M, et al. Patient education, disease activity and physical function: can we be more targeted? A cross sectional study among people with rheumatoid arthritis, psoriatic arthritis and hand osteoarthritis. Arthritis research & therapy. 2013 Oct 20;15(5):R156. doi: 10.1186/ar4339. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Kjeken I, Dagfinrud H, Mowinckel P, et al. Rheumatology care: Involvement in medical decisions, received information, satisfaction with care, and unmet health care needs in patients with rheumatoid arthritis and ankylosing spondylitis. Arthritis and rheumatism. 2006 Jun 15;55(3):394–401. doi: 10.1002/art.21985. [DOI] [PubMed] [Google Scholar]
  • 3.Nota I, Drossaert CH, Taal E, et al. Patient participation in decisions about disease modifying anti-rheumatic drugs: a cross-sectional survey. BMC musculoskeletal disorders. 2014 Oct 04;15:333. doi: 10.1186/1471-2474-15-333. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Bourgeois FT, Shannon MW, Valim C, et al. Adverse drug events in the outpatient setting: an 11-year national analysis. Pharmacoepidemiology and drug safety. 2010 Sep;19(9):901–910. doi: 10.1002/pds.1984. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Russo E, Sittig DF, Murphy DR, et al. Challenges in patient safety improvement research in the era of electronic health records. Healthc (Amst) 2016 Dec;4(4):285–290. doi: 10.1016/j.hjdsi.2016.06.005. [DOI] [PubMed] [Google Scholar]
  • 6.Smolen JS, Aletaha D, Bijlsma JWJ, et al. Treating rheumatoid arthritis to target: recommendations of an international task force. Ann Rheum Dis. 2010;69(4):631–637. doi: 10.1136/ard.2009.123919. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Institute of Medicine. The learning health care system in America. Washington (DC): National Academies Press; 2012. [Google Scholar]
  • 8.Schoels M, Knevel R, Aletaha D, et al. Evidence for treating rheumatoid arthritis to target: results of a systematic literature search. Ann Rheum Dis. 2010;69:638–643. doi: 10.1136/ard.2009.123976. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Grigor C, Capell H, Stirling A, et al. Effect of a treatment strategy of tight control for rheumatoid arthritis (the TICORA study): A single-blind randomised controlled trial. Lancet. 2004;364:263–269. doi: 10.1016/S0140-6736(04)16676-2. [DOI] [PubMed] [Google Scholar]
  • 10.Verstappen SMM, Jacobs JWG, van der Veen MJ, et al. Intensive treatment with methotrexate in early rheumatoid arthritis: aiming for remission. Computer Assisted Management in Early Rheumatoid Arthritis (CAMERA, an open-label strategy trial) Ann Rheum Dis. 2007;66:1443–1449. doi: 10.1136/ard.2007.071092. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Allaart CF, Geokoop-Ruiterman YPM, de Vries-Bouwstra JK, et al. Aiming at low disease activity in rheumatoid arthritis with initial combination therapy or initial monotherapy strategies: The BeSt study. Clin Exp Rheumatol. 2006 Nov-Dec;24(6 Suppl 43):S-77–82. [PubMed] [Google Scholar]
  • 12.Alemao E, Joo S, Kawabata H, et al. Effects of Achieving Target Measures in Rheumatoid Arthritis on Functional Status, Quality of Life, and Resource Utilization: Analysis of Clinical Practice Data. Arthritis Care Res (Hoboken) 2016;68(3):308–17. doi: 10.1002/acr.22678. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Curtis JR, Shan Y, Harrold L, et al. Patient perspectives on achieving treat-to-target goals: a critical examination of patient-reported outcomes. Arthritis Care Res (Hoboken) 2013;65(10):1707–12. doi: 10.1002/acr.22048. [DOI] [PubMed] [Google Scholar]
  • 14.Lavallee DC, Chenok KE, Love RM, et al. Incorporating patient-reported outcomes into health care to engage patients and enhance care. Health Aff (Millwood) 2016;35(4):575–82. doi: 10.1377/hlthaff.2015.1362. [DOI] [PubMed] [Google Scholar]
  • 15.National Quality Forum. [Accessed January 30, 2017];NQF-Endorsed Measures for Musculoskeletal Conditions. 2016 http://www.qualityforum.org/Publications/2015/01/NQF-Endorsed_Measures_for_Musculoskeletal_Conditions.aspx.
  • 16.Yazdany J, Robbins M, Schmajuk G, et al. Development of the American College of Rheumatology’s Rheumatoid Arthritis Electronic Clinical Quality Measures. Arthritis care & research Nov. 2016;68(11):1579–1590. doi: 10.1002/acr.22984. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Yazdany J, Bansback N, Clowse MEB, et al. Practice-Level Variation in Quality of Care in the Acr’s Rheumatology Informatics System for Effectiveness (RISE) Registry [abstract] [Accessed January 30, 2017];Arthritis Rheumatol. 2016 68(suppl 10) http://acrabstracts.org/abstract/practice-level-variation-in-quality-of-care-in-the-acrs-rheumatology-informatics-system-for-effectiveness-rise-registry/ [Google Scholar]
  • 18.Desai SP, Liu CC, Tory H, et al. Rheumatoid arthritis quality measures and radiographic progression. Seminars in arthritis and rheumatism. 2014 Aug;44(1):9–13. doi: 10.1016/j.semarthrit.2014.01.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Navarro-Compan V, Smolen JS, et al. Quality indicators in rheumatoid arthritis: results from the METEOR database. Rheumatology (Oxford, England) 2015 Sep;54(9):1630–1639. doi: 10.1093/rheumatology/kev108. [DOI] [PubMed] [Google Scholar]
  • 20.Sapir T, Rusie E, Greene L, et al. Influence of Continuing Medical Education on Rheumatologists’ Performance on National Quality Measures for Rheumatoid Arthritis. Rheumatol Ther. 2015 Dec;2(2):141–151. doi: 10.1007/s40744-015-0018-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Yazdany J, Bansback N, Clowse M, et al. Rheumatology Informatics System for Effectiveness: A National Informatics-Enabled Registry for Quality Improvement. Arthritis care & research. 2016 Dec;68(12):1866–1873. doi: 10.1002/acr.23089. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Walsh SH. The clinician’s perspective on electronic health records and how they can affect patient care. Br Med J. 2004;328:1184–1187. doi: 10.1136/bmj.328.7449.1184. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Newman E, Sharma T, Meadows A, et al. Rheumatoid Arthritis Quality Measures–Automated Display of Care Gaps and Capture of Physician Decision Making at the Clinic Visit [abstract] [Accessed January 25, 2017];Arthritis Rheumatol. 2015 67(suppl 10) http://acrabstracts.org/abstract/rheumatoid-arthritis-quality-measures-automated-display-of-care-gaps-and-capture-of-physician-decision-making-at-the-clinic-visit/ [Google Scholar]
  • 24.Powsner SM, Wyatt JC, Wright P. Opportunities for and challenges of computerisation. Lancet. 1998;352:1617–1622. doi: 10.1016/S0140-6736(98)08309-3. [DOI] [PubMed] [Google Scholar]
  • 25.Brown T. Design Thinking. Harvard Business Review. 2008;862008:84–92. [PubMed] [Google Scholar]
  • 26.McCreary L. Kaiser Permanente’s Innovation on the Front Lines. Harvard Business Review. 2010:7. [PubMed] [Google Scholar]
  • 27.Carroll RJ, Thompson WK, Eyler AE, et al. Portability of an algorithm to identify rheumatoid arthritis in electronic health records. Journal of the American Medical Informatics Association : JAMIA. 2012 Jun;19(e1):e162–169. doi: 10.1136/amiajnl-2011-000583. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Cannon GW, Mehorta S, South BR, et al. A Natural Language Processing System Can Capture Rheumatoid Arthritis Disease Activity Measures in US Veterans Across Multiple Sites [abstract] [Accessed January 22, 2017];Arth Rheum. 2016 68(suppl 10) [Google Scholar]
  • 29.Liao KP, Cai T, Gainer V, et al. Electronic medical records for discovery research in rheumatoid arthritis. Arthritis Care Res (Hoboken) 2010 Aug;62(8):1120–7. doi: 10.1002/acr.20184. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Lin C, Karlson EW, Dligach D, et al. Automatic identification of methotrexate-induced liver toxicity in patients with rheumatoid arthritis from the electronic medical record. J Am Med Inform Assoc. 2015 Apr;22(e1):e151–61. doi: 10.1136/amiajnl-2014-002642. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Hernandez Boussard T, Tamang S, Blayney D, et al. New Paradigms for Patient-Centered Outcomes Research in Electronic Medical Records: An Example of Detecting Urinary Incontinence Following Prostatectomy. EGEMS (Wash DC) 2016;4(3):1231. doi: 10.13063/2327-9214.1231. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Tamang S, Patel MI, Blayney DW, et al. Detecting unplanned care from clinician notes in electronic health records. J Oncol Pract. 2015 May;11(3):e313–9. doi: 10.1200/JOP.2014.002741. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Tamang S, Podchiyska T, Gyuang E, et al. Electronic phenotyping for measuring quality of care. American Medical Informatics Association Joint Summit: Clinical Research Informatics [abstract] 2015 [Google Scholar]
  • 34.Harpaz R, Callahan A, Tamang S, et al. Text mining for adverse drug events: the promise, challenges, and state of the art. Drug Saf. 2014;37(10):777–790. doi: 10.1007/s40264-014-0218-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Meystre SM, et al. Extracting information from textual documents in the electronic health record: a review of recent research. Yearb Med Inform. 2008:128–44. [PubMed] [Google Scholar]
  • 36.Friedman C, Rindflesch TC, Corn M. Natural language processing: State of the art and prospects for significant progress, a workshop sponsored by the National Library of Medicine. Journal of Biomedical Informatics. 2013;46(5):765–773. doi: 10.1016/j.jbi.2013.06.004. [DOI] [PubMed] [Google Scholar]
  • 37.Ford E, Carroll JA, Smith HE, et al. Extracting information from the text of electronic medical records to improve case detection: a systematic review. J Am Med Inform Assoc. 2016 Sep;23(5):1007–15. doi: 10.1093/jamia/ocv180. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Perrillo RP, Martin P, Lok AS. Preventing Hepatitis B Reactivation Due to Immunosuppressive Drug Treatments. JAMA. 2015;313(16):1617–1618. doi: 10.1001/jama.2015.2571. [DOI] [PubMed] [Google Scholar]
  • 39.Yazdany J, Calabrese L. Preventing hepatitis B reactivation in immunosuppressed patients: is it time to revisit the guidelines? Arthritis Care Res (Hoboken) 2010 May;62(5):585–9. doi: 10.1002/acr.20167. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Keane J, Gershon S, Wise RP, et al. Tuberculosis associated with infliximab, a tumor necrosis factor alpha-neutralizing agent. The New England journal of medicine. 2001 Oct 11;345(15):1098–1104. doi: 10.1056/NEJMoa011110. [DOI] [PubMed] [Google Scholar]
  • 41.Carmona L, Hernandez-Garcia C, Vadillo C, et al. Increased risk of tuberculosis in patients with rheumatoid arthritis. The Journal of rheumatology. 2003 Jul;30(7):1436–1439. [PubMed] [Google Scholar]
  • 42.Clowse ME, Magder L, Petri M. Cyclophosphamide for lupus during pregnancy. Lupus. 2005;14(8):593–597. doi: 10.1191/0961203305lu2169oa. [DOI] [PubMed] [Google Scholar]
  • 43.Jackson P, Paquette L, Watiker V, et al. Intrauterine exposure to mycophenolate mofetil and multiple congenital anomalies in a newborn: possible teratogenic effect. American journal of medical genetics Part A. 2009 Jun;149A(6):1231–1236. doi: 10.1002/ajmg.a.32715. [DOI] [PubMed] [Google Scholar]
  • 44.Yazdany J, Trupin L, Kaiser R, et al. Contraceptive counseling and use among women with systemic lupus erythematosus: a gap in health care quality? Arthritis Care Res (Hoboken) 2011 Mar;63(3):358–65. doi: 10.1002/acr.20402. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Schwarz EB, Longo LS, Zhao X, et al. Provision of potentially teratogenic medications to female veterans of childbearing age. Medical care. 2010 Sep;48(9):834–842. doi: 10.1097/MLR.0b013e3181e57946. [DOI] [PubMed] [Google Scholar]
  • 46.Hwang JP, Somerfield MR, Alston-Johnson DE, et al. Hepatitis B Virus Screening for Patients With Cancer Before Therapy: American Society of Clinical Oncology Provisional Clinical Opinion Update. J Clin Oncol. 2015;33(19):2212–2220. doi: 10.1200/JCO.2015.61.3745. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Weinbaum CM, Mast EE, Ward JW. Recommendations for identification and public health management of persons with chronic hepatitis B virus infection. Hepatology. 2009 May;49(5 Suppl):S35–44. doi: 10.1002/hep.22882. [DOI] [PubMed] [Google Scholar]
  • 48.Reddy KR, Beavers KL, Hammond SP, et al. American gastroenterological association institute guideline on the prevention and treatment of hepatitis B virus reactivation during immunosuppressive drug therapy. Gastroenterology. 2015;148:215–219. doi: 10.1053/j.gastro.2014.10.039. [DOI] [PubMed] [Google Scholar]
  • 49.Perrillo RP, Gish R, Falck-Ytter YT. American Gastroenterological Association Institute technical review on prevention and treatment of hepatitis B virus reactivation during immunosuppressive drug therapy. Gastroenterology. 2015;148(1):221–244. doi: 10.1053/j.gastro.2014.10.038. [DOI] [PubMed] [Google Scholar]
  • 50.Huang YH, Hsiao LT, Hong YC, et al. Randomized controlled trial of entecavir prophylaxis for rituximab-associated hepatitis B virus reactivation in patients with lymphoma and resolved hepatitis B. J Clin Oncol. 2013;31:2765–2772. doi: 10.1200/JCO.2012.48.5938. [DOI] [PubMed] [Google Scholar]
  • 51.Buti M, Morillas R, Manzano ML, et al. Tenofovir for the prophylaxis of HBV reactivation in anti-HBc-positive patients with hematologic malignancies treated with rituximab: preliminary results of a randomized study (PREBLIN Study) J Hepatol. 2014;60(1):S421–2. [Google Scholar]
  • 52.Kohler MJ, Amezaga M, Drozd J, et al. Use of a computerized order set to increase prescription of calcium and vitamin D supplementation in patients receiving glucocorticoids. J Gen Intern Med. 2013 Jun;28(6):825–9. doi: 10.1007/s11606-013-2360-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Chaudhry R, Schietel SM, North F, et al. Improving rates of herpes zoster vaccination with a clinical decision support system in a primary care practice. J Eval Clin Pract. 2013 Apr;19(2):263–6. doi: 10.1111/j.1365-2753.2011.01814.x. [DOI] [PubMed] [Google Scholar]
  • 54.van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc. 2006 Mar-Apr;13(2):138–147. doi: 10.1197/jamia.M1809. Epub 2005 Dec 15. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Suter LG, Barber CE, Herrin J, et al. American College of Rheumatology White Paper on Performance Outcome Measures in Rheumatology. Arthritis Care Res (Hoboken) 2016 Oct;68(10):1390–401. doi: 10.1002/acr.22936. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Mannion R, Braithwaite J. Unintended consequences of performance measurement in healthcare: 20 salutary lessons from the English National Health Service. Intern Med J. 2012 May;42(5):569–74. doi: 10.1111/j.1445-5994.2012.02766.x. [DOI] [PubMed] [Google Scholar]
  • 57.Ramsey CR, Matowe L, Grilli R, et al. Interrupted time series designs in health technology assessment: Lessons from two systematic reviews of behaviour change strategies. Int J Technol Assess Health Care. 2003 Fall;19(4):613–23. doi: 10.1017/s0266462303000576. [DOI] [PubMed] [Google Scholar]
  • 58.Harris AD, McGregor JC, Perencevich EN, et al. The use and interpretation of quasi-experimental studies in medical informatics. J Am Med Inform Assoc. 2006 Jan-Feb;13(1):16–23. doi: 10.1197/jamia.M1749. Epub 2005 Oct 12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Lewis RJ. The pragmatic clinical trial in a learning health care system. Clin Trials. 2016 Oct;13(5):484–92. doi: 10.1177/1740774516655097. [DOI] [PubMed] [Google Scholar]
  • 60.Mainous AG, 3rd, Lambourne CA, Nietert PJ. Impact of a clinical decision support system on antibiotic prescribing for acute respiratory infections in primary care: quasi-experimental trial. J Am Med Inform Assoc. 2013 Mar-Apr;20(2):317–24. doi: 10.1136/amiajnl-2011-000701. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Titsworth WL, Abram J, Guin P, et al. A prospective time-series quality improvement trial of a standardized analgesia protocol to reduce postoperative pain among neurosurgery patients. J Neurosurg. 2016 Dec;125(6):1523–1532. doi: 10.3171/2015.10.JNS15698. [DOI] [PubMed] [Google Scholar]
  • 62.Green H, Paul M, Vidal L, et al. Prophylaxis of Pneumocystis pneumonia in immunocompromised non-HIV-infected patients: systematic review and meta-analysis of randomized controlled trials. Mayo Clinic proceedings. 2007 Sep;82(9):1052–1059. doi: 10.4065/82.9.1052. [DOI] [PubMed] [Google Scholar]
  • 63.Jones RB, Tervaert JW, Hauser T, et al. Rituximab versus cyclophosphamide in ANCA-associated renal vasculitis. The New England journal of medicine. 2010 Jul 15;363(3):211–220. doi: 10.1056/NEJMoa0909169. [DOI] [PubMed] [Google Scholar]
  • 64.Singh JA, Furst DE, Bharat A, et al. 2012 update of the 2008 American College of Rheumatology recommendations for the use of disease-modifying antirheumatic drugs and biologic agents in the treatment of rheumatoid arthritis. Arthritis Care Res (Hoboken) 2012 May;64(5):625–39. doi: 10.1002/acr.21641. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65.Bhatt DL, Scheiman J, Abraham NS, et al. ACCF/ACG/AHA 2008 expert consensus document on reducing the gastrointestinal risks of antiplatelet therapy and NSAID use. Am J Gastroenterol. 2008;103(11):2890. doi: 10.1111/j.1572-0241.2008.02216.x. [DOI] [PubMed] [Google Scholar]
  • 66.Saag KG, Teng GG, Patkar NM, et al. American College of Rheumatology 2008 recommendations for the use of nonbiologic and biologic disease-modifying antirheumatic drugs in rheumatoid arthritis. Arthritis and rheumatism. 2008 Jun 15;59(6):762–784. doi: 10.1002/art.23721. [DOI] [PubMed] [Google Scholar]
  • 67.Shea B, Swinden MV, Tanjong Ghogomu E, et al. Folic acid and folinic acid for reducing side effects in patients receiving methotrexate for rheumatoid arthritis. Cochrane Database Syst Rev. 2013 May;(5):CD000951. doi: 10.1002/14651858.CD000951.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 68.Marmor MF, Kellner U, Lai TY, et al. American Academy of Ophthalmology. Recommendations on Screening for Chloroquine and Hydroxychloroquine Retinopathy (2016 Revision) Ophthalmology. 2016;123:1386–1394. doi: 10.1016/j.ophtha.2016.01.058. [DOI] [PubMed] [Google Scholar]

RESOURCES