Measuring Process of Arthritis Care. A Proposed Set of Quality Measures for the Process of Care in Juvenile Idiopathic Arthritis

Abstract

Objectives

The ability to assess quality of care is a necessary component of continuous quality improvement. The assessment typically is accomplished by determination of compliance with a defined set of quality measures (QMs). The objective of this effort was to establish a set of QM for the assessment of the process of care in JIA.

Methods

A 12 member working group (WG) composed of representatives from the ACR, AAP, ABP, and ARHP was assembled to guide the project. Delphi questionnaires were sent to 237 health professionals involved in the care of children with juvenile idiopathic arthritis (JIA). A total of 471 items in 27 domains were identified. The WG met via four live e-meetings during which results from the Delphi’s were distilled to a reduced draft set. Each WG member selected a proposed QM to investigate and present evidence from the literature as to its attributes and appropriateness for inclusion into the set. Nominal group technique was used to come to consensus on a proposed set of QMs.

Results

The proposed set contains 12 QMs within four health care domains. Each QM consists of a statement of (i) the assessment to be completed, (ii) when the first assessment should be completed and a suggested frequency of assessment during follow-up, (iii) recommendations of appropriate tools or methods of assessment, and (iv) initial performance goals.

Conclusions

Implementation of the proposed QM set will improve the process of care, facilitate continuous QI, and eventuate in improved health outcomes of children with JIA.

Pediatric rheumatology centers are engaged in local quality improvement (QI) efforts in their centers to varying degrees. Systematic efforts have been pursued at some centers for years, but for most the efforts are just beginning. Central to this process is the identification of ‘quality measures’ (QMs, also commonly referred to as ‘quality indicators’). QMs must be developed in consideration of the evidence that (i) their performance results in improved health outcomes and evidence and (ii) they are operationally feasible in a variety of clinic infrastructures. QMs focused on the process of care typically contain a statement of the assessment to be completed, method for conducting the assessment, frequency of reassessments during follow-up and initial performance goals. Although some individual pediatric rheumatology centers have identified QMs and are using them to perform QI, there is not a uniform QM set relevant to the process of care of children and adolescents with Juvenile Idiopathic Arthritis (JIA) in widespread use. In this project the focus was on process of care to facilitate development of a standardized method of assessment of QI across centers. Once this method is established, then centers will be able to use these data in the future to develop validated QI measures that focus on patient status and outcomes.

In 2008 the American College of Rheumatology (ACR) provided funding to develop a set of QMs for use by health care professionals (HCPs) involved in the care of patients with JIA. The chief objective of the project described herein was to convene representatives from major North American organizations involved in pediatric rheumatology to work together to develop a standard set of QMs, based upon existing evidence and clinical experience focused on the process of care for patients with JIA. The development of this standard set of QMs was done to (i) facilitate the development of a network of pediatric rheumatology centers working in a coordinated fashion to improve the outcome of children with JIA, (ii) to serve as a basis for QI projects for the ACR and other organizations and, (iii) to be used by the American Board of Pediatrics (ABP), as Part 4 of the requirements for Maintenance of Certification in Pediatric Rheumatology¹.

METHODS

We used a methodology similar to that used by MacLean et al to develop QM for rheumatic diseases in adults². A Quality Measures Working Group (WG) was assembled that was composed of 12 individuals, each selected by their respective organization from the pediatric rheumatology sections of the American College of Rheumatology (ACR) and the American Academy of Pediatrics (AAP), the Association of Rheumatology Health Professionals (ARHP), the American Board of Pediatrics (ABP), the Childhood Arthritis and Rheumatology Research Alliance (CARRA) Consolidated Registry Committee, the Pediatric Rheumatology Improvement Network for Clinical Effectiveness (PRINCES), and two facilitators experienced in group process and consensus formation techniques. The WG’s overall charge was to use consensus techniques to develop a QM set, based upon available evidence and clinical expertise, for the process of care for JIA.

In order to obtain broad-based input regarding health care domains and assessments in those domains, the WG sent an online Delphi-type³ electronic questionnaire (Delphi 1) to various stakeholders. The Delphi packets included an explanation of the rationale for need, the intended purpose of the survey, explanations as to what QMs represent and their intended use. Recipients were instructed to state what they believed to be the top five measures of outcome or care in JIA. A total of 237 individuals received the survey, including 170 pediatric rheumatologists, eight advanced practice nurses (APNs) involved in JIA care, and 59 patients and parents, representing 27 states and Prince Edward Island, Canada. Results of Delphi 1 were reviewed by the WG, which distilled the assessments offered by respondents including grouping them into domains, during a series of teleconferences employing nominal group technique (NGT) consensus formation methodology⁴. Replies that were redundant or represented the same concept, but stated in a different manner, were combined. Domains and assessments in the domains were then retained or eliminated based largely upon feasibility (practicality, ease of use), and face validity (clinical sensibility)^5-6.

A second questionnaire survey (Delphi 2) containing a reduced set of domains and assessments was developed and sent to the same 237 stakeholders, regardless of whether or not they had replied to Delphi 1. Participants were asked to rank order the reduced number of domains and assessments within a domain in terms of their perceived relative importance in assessing quality of care.

Working from these results, a series of four web-based interactive live meetings using iLinc (available at www.iLinc.com) were held among WG members and facilitators during 2009. The purposes of these meetings were to come to consensus on the following questions: (1) should the assessment be retained in the proposed set; (2) do validated methods exist for conducting the assessment and, if so, should one method be recommended over the others; (3) when should the first assessment occur, and how frequently should reassessment be done during follow-up; (4) what proportion of patients with JIA in a given clinical practice should meet the QM initially, referred to as the ‘initial performance goal’.

Prior to convening the series of live meetings, each WG member chose a specific QM to investigate by searching the literature and relying on his/her personal clinical expertise in order to answer the four questions shown above. The intent was to familiarize the WG with existing evidence of the utility and clinical relevance of each QM, thereby facilitating consensus-formation and synthesis of a proposed set of recommendations. (The specific references used to obtain evidence on which to base recommendations made to the group by individual presenters are given as an Appendix to this manuscript.)

Format of the four live meetings was as follows. A facilitator introduced the topics for discussion and deliverables (e.g. questions to be addressed) for the call. This was followed by a presentation of the evidence for each QM by the member who had undertaken the review. The presenter stated his or her suggested answers to the questions above, and provided the evidence that formed the basis for their opinions. Following the presentation, NGT was used to come to consensus about the answers to the questions stated above. In general, meetings lasted approximately 3 hours and two to three QM were discussed during each meeting. Facilitators reviewed digital recordings of each meeting, developed a working document containing QM statements, and circulated it to the entire WG for revision. A finalized set of recommendations was then produced.

RESULTS

Response to Delphi 1: Overall response rate to Delphi 1 was 40% (N = 95); 35% for pediatric rheumatologists, 50% for APNs, and 52% for parents and patients. Among physician-respondents, 85% were currently treating more than 50 patients with JIA per year.

Responses to Delphi 1 resulted in 471 proposed JIA QMs sorted into 23 domains (Table 1). Using conference calls, Delphi and NGT, the WG reviewed the 471 QMs for feasibility, face validity, and redundancy. Item reduction resulted in five domains with a total of 19 quality measures: (1) disease control, 11 QMs, (2) safety monitoring, 4 QMs, (3) education, 2 QMs, (4) access/equity/timeliness, one QM and (5) relationship/patient-family satisfaction with care, one QM.

Table 1.

Summary of identified domains and assessments within domains resulting from Delphi 1 survey on measures of process of care in JIA^*

Disease Control	Medication Monitoring	Treatment	Physical Function	Education	Access	Uveitis	Relationship	Timeliness
69	52	48	32	29	26	26	22	22
Pain	Patient Centeredness	Effective	Psychosocial Functioning	Satisfaction	Physical Growth	Quality of Life	Efficiency	Equity
22	13	12	12	9	8	8	8	8
Communication	Finances/Insurance	Compliance	Transition to Adult Care	Other
7	5	4	2	27

^*Numbers below domains indicate the number of assessments that were suggested for inclusion into the domain.

These five domains and 19 assessments formed the basis for Delphi 2 in which participants were asked to rank order the domains and, for the three domains that contained more than one assessment (disease control, safety monitoring, and education), rank order the assessments within the respective domain.

Response to Delphi 2: Overall response rate was 57% (N = 135): 57% for pediatric rheumatologists, 100% for APNs and 50% for patients and parents. Of these respondents, 69% had participated in the Delphi 1.

Results from Delphi 2 showed the highest ranking assessments in domains with more than one assessment were: (1) disease control: (tied) number of joints with active arthritis and physician global assessment of disease activity; (2) safety monitoring: patients undergo eye screening according to published guidelines; (3) access/equity/timeliness: time to first available appointment with a pediatric rheumatologist.

The WG reviewed the Delphi 2 responses, and used NGT to come to consensus on a working set of QMs. The original five domains and 12 assessments within the domains were maintained in the working set (Table 2). This working set formed the basis for the four live web-based consensus conferences aimed at reviewing the evidence for maintaining the proposed domains and assessments, as well as tools for assessment, frequency of assessment, and initial and final quality performance goals.

Table 2.

Working set of domains and assessments from the Delphi exercises and working group consensus on identifying measures for the process of care in JIA^*

Disease Control Domain Pain management Achieve complete disease control within a period of X months Number of active joints Physician’s global assessment of disease activity Physical functional ability Quality of life

Safety Monitoring All patients undergo eye screening according to published guidelines Regular laboratory screening for toxicity to second line therapeutic agents and biologics All patients are screened for TB prior to the start of biologic therapy

Education Visual numeric scale to assess parent/patient confidence in ability to manage care of children with JIA

Access/Equity/Timeliness Time to first available appointment with pediatric rheumatologist

Relationship Parent/patient satisfaction with care

^*Some items offered by Delphi participants represent outcomes rather than process of care assessments. Because the focus of this effort was process of care, such replies were not considered further for inclusion.

Results of the four live web-based meetings were as follows. During the initial meeting, the WG decided that five assumptions were necessary in order to specify what categories of JIA the QMs would be applicable to, and to provide working definitions of the terminology used in QM statements. These assumptions are shown at the top of Table 3. Consensus was reached during the live meetings on eight of the resulting QM statements. The remaining four QMs were re-circulated to WG members in survey format in an attempt to arrive at consensus. When it became apparent that no consensus among the WG members was possible, outside opinion was sought from five external board-certified pediatric rheumatologists chosen by the WG, and with extensive experience in caring for children with JIA. The proposed set of QMs resulting from the WG meetings and follow-up surveys are shown in Table 3. A total of 12 QM distributed among four health care domains are included in the proposed set.

Table 3.

Assumptions and summary of the final proposed set of quality measures for the process of care in JIA

Assumptions: QMs were developed under the assumption that they would be applicable to all categories of JIA. QMs, their frequency of assessment, and performance goals are minimum recommendations. In the spirit of continuous quality improvement, performance goals should be improved upon continually over time. QM in which a timed frequency of assessment (e.g. every 6 months) is recommended implies that if the patient is seen less often than the recommended assessment schedule, then the assessment will be done at each visit. First visit implies first visit to the pediatric rheumatologist after a diagnosis of JIA has been made. If a physician practices within an institution or clinic that recommends a frequency and/or method of assessment of QM/s, that is comparable (and not less stringent) then the local guidelines should be followed in lieu of those below.

DISCUSSION

Healthcare providers strive to provide good care to their patients by virtue of their training, and professional ethics; however, this does not necessarily translate into quality outcomes for the patients. In 1999, the Institute of Medicine (IOM) reported that an estimated 98,000 patients die from medical errors per year in the United States and mandated reformation in the healthcare system to reduce the error and improve safety for patients⁷. The IOM further recommended that all organizations improve their performance by incorporating care process and outcome measures into their daily work for improvement and accountability. The QI movement grew from this and subsequent efforts of the IOM and is now incorporated into a broad, complex and incompletely coordinated landscape of agencies, institutions, healthcare insurers, healthcare facilities, etc. Furthermore, the importance of developing quality measures derives from the contemporary era of mandated quality of care as set forth in virtually all arenas of medical environment: patient care, education, and accreditation.

The JIA QM Workgroup was organized with the purpose of developing a core set of QM for JIA that would be adopted broadly. Moreover, it was thought to be critical that these QM be developed by those most directly involved in the care of children with JIA-parents, patients and pediatric rheumatology health care providers. The organizations which the WG members were drawn from represent the key professional, scientific and accreditation organizations in pediatric rheumatology. The ABP, in accord with the recommendations of the Task Force on Competence of the American Board of Medical Specialties (ABMS), has mandated that all pediatric rheumatologists seeking ABP recertification demonstrate competency through active participation in quality improvement. Representatives from the ABP were actively involved in the development of these JIA QMs and will incorporate them in Maintenance of Certification training and materials. The JIA QM Workgroup chose to focus on measures of the system or process of care initially to establish regular assessments of the common important measures of care in a standardized fashion. This is a necessary and critical first step in QI efforts. Once centers have adopted these process measures we will be able to move forward as a group with addressing patient outcomes. This work will be informed and grounded in the results obtained from the current proposed core set of QI measures for the process of care in JIA.

The domains developed in this project will assess dimensions of care which the IOM identified as needing improvement. The IOM specified dimensions of quality care include safety, efficiency, efficacy, patient-centered, equity, and timeliness⁸. The domains in this JIA QM set include: 1) Disease Control which reflects effectiveness; 2) Safety Monitoring, which addresses Safety; 3) Assessment of Self-Efficacy and Patient/Parent Satisfaction which are central to Patient Centered Care; and lastly 4) Access which incorporates Timeliness. All of the selected JIA QMs are measurable and were subject to evidence in the literature. Equity in closing racial and ethnic gaps in health status, recognized as a key area needing improvement, is an outcome-focused measure, and therefore not included in the current set of QMs. ‘Initial performance goals’ given in the QM set are based largely on what the QMWG concluded were reasonable and feasible for most practices to achieve. Conversely, no ‘ideal performance goal’ is given because continuous QI implies that ever higher performance goals are to be set, achieved, and then improved upon further.

The JIA QMs developed in this project hopefully will serve as a format for registry development and initiation of serial data acquisition. This will provide a baseline of data for the development of quality improvement projects, both for the individual practitioner or as a collaborative for a group of centers. Many pediatric subspecialty groups, including neonatology, pulmonology, and critical care medicine have had strategies to conduct quality improvement through national and regional collaboratives. Benchmarking can be established as collaborative members are transparent in the data and share best practices. The optimal result of the quality improvement work done in either setting, individual or collaborative, will be the improvement of the processes of care which ultimately affects the outcome of the patient’s disease status. The view of the WG members is that this improvement will be achieved in a much more efficient and timely manner for JIA if the effort is driven by the use of a standardized set of widely adopted QMs. QMs developed in this project provide parameters but leave latitude for choosing the specific tools or instruments to measure the assessment. The aim is to have relatively uniform data collection for optimizing the outcome of quality improvement projects. Broad based efforts in pediatric rheumatology are ongoing at this time to develop a common JIA registry for longitudinal data collection in JIA patients in many pediatric rheumatology centers and to develop a sophisticated QI network. The incorporation of these JIA QMs in both these efforts will enhance the effectiveness of our shared goal of improving the systems of care for children with JIA so as to improve their disease, function, and quality of life outcomes.

Appendix

Disease Control Domain

Quality Measure #1: Assessment of Arthritis-Related Pain

Pain should be assessed in all patients at the first visit and at each subsequent visit that occurs at least 7 days apart. (The QMWG recognized two types of pain: (i) acute pain intensity at a point in time and, (ii) average pain over some period. Here, average pain over a period of 7 days is what is to be assessed.)

• Tools: In patients 7 years or older, pain should be assessed as an average over the last 7 days. In patients less than 7 years of age, the parent or guardian should be the proxy reporter of average pain. Any validated, reliable, age-appropriate tool to measure average pain may be used.

• Initial Quality Performance Goal: ≥80% of patients receive an assessment of pain at the first visit and at each subsequent visit if at least 7 days apart.

Quality Measure #2: Rheumatological Joint Count

A full joint count of all 75 joints should be done on all patients at the first visit and at 6 month intervals. (Reduced or weighted joint counts may be acceptable in the future after these methods have been more fully validated)

• Tools: Included in the QM.

• Initial Quality Performance Goal: ≥80% of initial visits and 6 month interval visits include a joint count.

Quality Measure #3: Physician’s Global Assessment of Disease Activity (PGA)

A PGA should be completed on all patients at the initial visit and at each subsequent visit.

• Tools: Any validated, reliable method may be used to assess the PGA. An example is the 0-10 VAS, divided into 0.5 increments⁹.

• Initial Quality Performance Goal: 80% of initial and subsequent visits include a PGA.

Quality Measure #4: Assessment of Functional Ability

All patients should receive an assessment of functional ability at the initial visit and at a minimum of 6 month intervals thereafter.

• Tools: Any age-appropriate validated and reliable tool for the assessment of functional ability may be used. An example is the Childhood Health Assessment Questionnaire¹⁰.

• Initial Quality Performance Goal: ≥70% of initial and 6 month interval visits will include an assessment of functional ability.

Quality Measure #5: Assessment of Health Related Quality of Life (HRQOL)

All patients should receive an assessment of HRQOL at the initial visit and at 6 month intervals.

• Tools: Any age appropriate, validated, reliable and age-appropriate tool, which is capable of changing over a 6 month interval, may be used to assess HRQOL. Examples include the PedsQL™¹¹ and Childhood Health Questionnaire¹².

• Initial Quality Performance Goal: ≥70% of initial and 6 month interval visits will include an assessment of HRQOL.

Quality Measure #6: Eye Examinations and Documentation of Compliance to Guidelines

Either the AAP¹³ or modified Heiligenhaus¹⁴ guidelines for eye examinations should be followed for patients with any category of JIA. Documentation of compliance to the guidelines should be performed at every visit at least 3 months apart.

• Tools: No single mechanism for obtaining written or verbal documentation of compliance is recommended, but should be one that is convenient for the practice, such as consultation with the ophthalmologist.

• Initial Quality Performance Goal: ≥40% of patients provide documentation of compliance to the guidelines.

Safety Monitoring Domain

Quality Measure #7: TB Screening in JIA Patients Beginning Biologic Therapy

All patients with JIA will undergo TB screening no longer than 3 months prior to the start of any biologic therapy and yearly thereafter as long as the patient remains on biologic therapy.

• Tools: TB screening may be done using the standard TB skin test or, if appropriate, a standard blood test. If, in the clinician’s judgment, the patient is immuno-compromised and the TB skin test is negative or the patient has received BCG so that skin test results are in question, a blood assay test is recommended prior to beginning any biologic therapy.

• Initial Quality Performance Goal: 100% of patients receive TB screening prior to start of any biologic therapy and yearly thereafter.

Quality Measure #8: Laboratory Monitoring

All JIA patients receiving DMARD or biologic therapy will be monitored for toxicity by clinical laboratory methods. The minimal frequency of laboratory monitoring for DMARDs is every 2-4 weeks for the first 3 months of therapy, every 8-12 weeks after 3-6 months, and every 12 weeks after 6 months of therapy. Guidelines for laboratory monitoring of biologic therapy are evolving and recommendations in this quality measure will be added in the future.

• Tools: The minimal laboratory panel for monitoring laboratory toxicity for DMARDS includes CBC, platelets, and transaminases. CrCl should be assessed in new DMARD starts and at regular intervals in patients on cyclosporine or in patients with known renal disease. Pregnancy testing is at the discretion of the managing physician.

• Quality Performance Goal: An initial quality performance goal is 70% of patients receiving DMARDs undergo the recommended laboratory monitoring.

Quality Measure #9: Behavioral Counseling about Toxicity to DMARDS and Biologic Therapy

Patients engaging in risk-taking behaviors which could have health consequences related to their medication or disease should receive counseling at each visit. Patients not engaging in risk-taking behaviors which may have health consequences related to their medication or disease should receive counseling at least annually.

• Tools: At the discretion of the health care professional.

• Initial Quality Performance Goal. ≥70% of patients receiving DMARDs or biologics undergo counseling.

Assessment of Self Efficacy and Patient/Parent Satisfaction

Quality Measure #10: Assessment of Self-Efficacy

(self efficacy is the belief that one is capable of performing in a certain manner to attain certain goals¹⁵)

Patients should be assessed for self-efficacy at periodic intervals. The first assessment of self-efficacy should be ≤ 6 months after the initial visit and every 6 months thereafter.

• Tools: Acceptable tools for assessment of self-efficacy include (i) the General Self-Efficacy Scale¹⁶, (ii) the Children’s Arthritis Self-Efficacy Scale¹⁷, and (iii) the Parent’s Arthritis Self-Efficacy Scale¹⁸. Other validated tools may be used.

• Initial Quality Performance Goal: ≥50% of patients are assessed for self-efficacy ≤ 6 months after the initial visit, and every 6 months thereafter, if feasible.

Quality Measure #11: Assessment of Patients/Parents Satisfaction with Care

Patients and/or parents should be assessed for their level of satisfaction with the quality of care provided to them/their child. The first assessment should be done no longer than 1 year after the initial visit, and annually thereafter. (Because the individual physician is often not able to influence the patient’s satisfaction with the overall health care delivery system or insurance company, this QM focuses upon satisfaction with the health care provided by the physician.)

• Tools: No one specific tool for use in JIA is recommended. Satisfaction may be assessed using any validated tool such as the Consumer Assessment of Healthcare Providers and System questionnaire (CAHPS¹⁹) or other validated and reliable tool.

• Initial Performance Goal: Satisfaction assessments are received from ≥30% of patients according to the recommended schedule. (This implies that the physician has made enough effort to receive satisfaction assessments from ≥30% of the overall JIA population in the practice.)

Access to Care

Quality Measure #12: Time to Third-Next Available

As an initial goal, all patients with signs and/or symptoms of JIA should receive an appointment, based upon the third-next-available²⁰, an average of ≤ 60 calendar days for a new patient visit from the time of referral. (Timing of follow-up visits is dependent upon numerous factors, and no recommendation is made here.)