Abstract
Objective
Antineutrophil cytoplasmic antibody (ANCA)-associated vasculitis (AAV) is a group of linked multisystem life- and organ-threatening diseases. The Outcome Measures in Rheumatology (OMERACT) vasculitis working group has been at the forefront of outcome development in the field and has achieved OMERACT endorsement of a core set of outcomes for AAV. Patients with AAV report as important some manifestations of disease not routinely collected through physician-completed outcome tools; and they rate common manifestations differently from investigators. The core set includes the domain of patient-reported outcomes (PRO). However, PRO currently used in clinical trials of AAV do not fully characterize patients’ perspectives on their burden of disease. The OMERACT vasculitis working group is addressing the unmet needs for PRO in AAV.
Methods
Current activities of the working group include (1) evaluating the feasibility and construct validity of instruments within the PROMIS (Patient-Reported Outcome Measurement Information System) to record components of the disease experience among patients with AAV; (2) creating a disease-specific PRO measure for AAV; and (3) applying The International Classification of Functioning, Disability and Health to examine the scope of outcome measures used in AAV.
Results
The working group has developed a comprehensive research strategy, organized an investigative team, included patient research partners, obtained peer-reviewed funding, and is using a considerable research infrastructure to complete these interrelated projects to develop evidence-based validated outcome instruments that meet the OMERACT filter of truth, discrimination, and feasibility.
Conclusion
The OMERACT vasculitis working group is on schedule to achieve its goals of developing validated PRO for use in clinical trials of AAV. (First Release September 1 2015; J Rheumatol 2015;42:2204–9; doi:10.3899/jrheum.141143)
Key Indexing Terms: VASCULITIS, ANCA, OUTCOMES, PATIENTS
Antineutrophil cytoplasmic antibody (ANCA)-associated vasculitis (AAV) is a group of linked diseases that includes granulomatosis with polyangiitis (Wegener’s), eosinophilic granulomatosis with polyangiitis (Churg-Strauss), and microscopic polyangiitis. These are multisystem life- and organ-threatening diseases that result in substantial morbidity, both from the disease itself and its treatment1. The course of AAV has changed over the past 40 years from a usually acute disease with high short-term mortality to a now usually chronic, relapsing disease marked by alternating periods of active vasculitis and periods of full remission. Treatments characterized by the chronic use of glucocorticoids and immunosuppressive medications complicate the disease course and patient experiences with AAV.
The OMERACT vasculitis working group has been at the forefront of outcome development in the field and includes major international leaders in outcomes research and trial design in vasculitis. The development and subsequent endorsement at OMERACT of the core set of outcomes for AAV was a substantial achievement for the group2. The core set includes the domain of patient-reported outcomes (PRO) with the aim of recording patients’ perspectives on their disease in clinical trials.
The OMERACT vasculitis working group and others have explored the patient perspective in AAV and demonstrated, not surprisingly, that patients report as important, manifestations of disease that are not routinely collected through physician-completed outcome tools; and they rate common manifestations differently from investigators3,4,5. Fatigue, musculoskeletal symptoms, and effect of disease on daily life and function are all of great concern to patients but are rated lower or are not measured at all by traditional outcome tools in vasculitis. The general health-related quality of life (HRQOL) measure, the Medical Outcomes Study Short Form-36 survey (SF-36), is used regularly in clinical AAV research, and is currently included in the AAV core set of measures6. The SF-36, along with other generic measures, has documented significant multidimensional impairments in HRQOL in patients with AAV7,8,9,10,11. While the SF-36 identifies some aspects of AAV, generic PRO are not specific enough to measure the complexity and change of experiences in patients with multisystem diseases such as AAV.
Members of the OMERACT vasculitis working group formed the Vasculitis Clinical Research Consortium–Patient-Centered Outcomes Research Institute Steering Committee (VCRC-PCORI) to oversee coordinated international efforts to address the need for more comprehensive and disease-specific PRO. The group has developed a comprehensive research strategy, organized an investigative team, included patient research partners (PRP; Table 1), obtained peer-reviewed funding for these projects, and is using a considerable research infrastructure to complete a series of interrelated projects aimed at developing evidence-based validated outcome instruments that meet the OMERACT filter of truth, discrimination, and feasibility12,13.
Table 1.
Patient involvement in the OMERACT Vasculitis working group/VCRC-PCORI steering committee, and research projects.
| Patient Involvement | ||
|---|---|---|
| ||
|
| ||
| Use of PROMIS Measures in AAV |
Development of a Disease-specific PRO Instrument for AAV |
Application of the ICF in AAV |
|
| ||
|
|
|
Future step. VCRC: Vasculitis Clinical Research Consortium; PCORI: Patient-Centered Outcomes Research Institute; PROMIS: Patient-reported Outcomes Measurement Information System; AAV: ANCA-associated vasculitis; PRO: patient-reported outcome; ICF: International Classification of Functioning, Disability and Health; ANCA: antineutrophil cytoplasmic antibody.
Three interrelated projects currently designed to address the unmet needs for PRO in vasculitis include (1) evaluation of the feasibility and construct validity of instruments within the Patient-Reported Outcome Measurement Information System (PROMIS) to record components of the disease experience among patients with AAV; (2) creation of a disease-specific PRO measure for AAV; and (3) application of the International Classification of Functioning, Disability and Health (ICF) to examine the scope of outcome measures used for assessment of patients with AAV and to identify areas of disease in need of further study and instrument development.
Generic PRO Measures in AAV
The OMERACT vasculitis working group is conducting a validation study of selected instruments within the PROMIS for use in AAV. PROMIS is an evolving set of generic item banks intended to cover all aspects of self-reported health14. The underlying statistical framework for PROMIS is item response theory that allows computer-adaptive testing (CAT) to be conducted. CAT presents study participants with a series of questions where the choice of progressive questions varies in number and is guided by algorithms driven by the participant’s response to the previous question. CAT increases the precision of the instrument and minimizes floor and ceiling effects. CAT allows incorporation of items operating at the extremes of the domain; such items are often excluded from paper instruments. PROMIS item banks include several versions of paper short forms with fixed numbers of questions derived from extensive data on the use of the item banks. The short forms are both well validated and more precise than many similar-length “generic” health status questionnaires.
With the PROMIS vasculitis project, the VCRC-PCORI steering committee hypothesizes that the developed outcome measures will have excellent statistical properties and high discriminatory power to detect differences between therapeutic agents. High precision and minimal floor and ceiling effects with CAT would maximize information contributed from each enrolled participant. Such improved trial efficiency is especially valuable when conducting trials with small sample sizes or short followup times and can also provide more statistical power to analyze a drug’s efficacy in selected clinical subgroups. Although these are important considerations for the study of any disease, they are especially crucial for research in rare diseases such as the vasculitides. Other features that affected the group’s decision to explore the utility of PROMIS include its comprehensiveness with respect to content and design that facilitates easy translation to other languages, and that PROMIS is free of charge for all uses.
The OMERACT vasculitis working group PROMIS project is well under way, with the research protocol, including choice of specific instruments for study, decided upon by the steering committee composed of clinical investigators, qualitative researchers, and PRP.
As of OMERACT 12 (2014), 10 selected CAT-based PROMIS instruments had been administered to about 300 study participants within the VCRC longitudinal study of AAV (as well as 4 other forms of vasculitis). A preliminary analysis is under way and will be directed toward determining construct validity and assessing performance of PROMIS instruments longitudinally by exploring sensitivity to change and defining values for minimal clinically important changes. Four short-form PROMIS instruments are also being collected among participants in 2 randomized controlled trials (RCT) to help determine the feasibility of PROMIS in a clinical trial setting and provide data on how PROMIS discriminates between treatment arms12,13.
Feedback on the PROMIS project was solicited and obtained from participants at OMERACT 12. Although it was considered unlikely that data from PROMIS instruments could serve as a primary outcome measure in RCT in vasculitis, such data could be highly supportive of a labeling claim of novel therapeutic agents12. In addition, PROMIS instruments could contribute to a composite primary outcome measure or serve as informative secondary outcome measures in RCT in vasculitis. The optimal use of PROMIS in vasculitis remains to be defined and will be dependent on the strength of the data, and acceptance by patients and investigators.
Creation of a Disease-specific PRO for AAV
The vasculitis workshop at OMERACT 11 (2012) highlighted the lack of a disease-specific PRO for patients with AAV6. It was decided that a collaborative approach to developing candidate questionnaire items, involving patients from the United Kingdom, United States, and Canada, was feasible and desirable because of the relative rarity of the disease and the ability to create a tool with content validity and cultural/linguistic equivalence appropriate for use in all 3 countries. The development of a new PRO involves 3 stages: (1) questionnaire item development; (2) item reduction and scale generation; and (3) testing of scale properties, including reliability, validity, and responsiveness. It is important that a PRO be developed using methodologies that comply with the US Food and Drug Administration recommendations to support the instrument’s use in clinical trials and in supporting labeling claims for new medications12.
A multinational collaboration of researchers and PRP conducted the first stage of questionnaire item development. Exploratory semistructured patient interviews were performed in Oxford, UK, Philadelphia, USA, and Ottawa, Canada, with the aim of identifying salient dimensions of quality of life and perceived problems of health status related to having AAV. The overall sample size was determined by the point at which no new themes emerged from interviews (saturation), and was also guided by a purposive sampling framework to ensure that the broadest range of experiences possible was recorded, consistent with disease variability. Researchers within and across research groups independently reviewed de-identified interview transcripts for relevant themes including symptoms related to condition or treatment, and the ways in which symptoms influenced people’s ability to work, activities of daily living, engagement in social activities, and their state of mind. Themes identified from transcripts were then recast as candidate questionnaire items, which were reviewed and amended by our PRP on the steering committee. Drafted items were further refined using feedback from PRP related to item presentation, construction, and response categories. Additional steps in assessing the adequacy of scope and item prioritization will include reviewing results from freelisting and pilesorting activities of participant responses. The last step in item refinement will include piloting the candidate questionnaire items using cognitive interviews, a formal linguistic assessment of item readability and translatability, then surveying several hundred patients, in collaboration with the patient group Vasculitis UK and the VCRC in the United States, to produce the reduced final form of the instrument with appropriate scale structure, measurement properties, and scoring algorithms fully specified. This will be followed by a multicenter prospective validation study to permit assessment of responsiveness and minimal important change.
At OMERACT 12 a breakout group cohosted by a PRP and a researcher, each on the VCRC-PCORI Steering Committee, solicited feedback on the work completed and the next steps. The group voted unanimously that contextual factors, such as access to healthcare or information about the condition, were important and should be measured, but not within the PRO currently being developed and designed primarily for use in RCT. Also discussed was the form that any stated attribution should take within the questionnaire, i.e., “due to your vasculitis” versus “due to your vasculitis or its treatment.” The group agreed that treatment effects should be considered because of the challenges for patients and clinicians in differentiating the effects of disease versus external factors such as toxicities or comorbidities. The breakout group advised that the issues of attribution should be further explored at the cognitive interviewing stage. The breakout group considered the proposed survey for item reduction and scale generation, which would include data from patients in the United Kingdom completing paper questionnaires and patients in the United States completing online surveys. There was unanimous support for combining the data, but it was suggested that subsets of patients in each population should complete the questionnaire using the alternative method to evaluate the equivalence of these approaches. The group plans to build on the feedback received from this breakout group in planning the next stage of validation, designing the larger scale survey, and in providing prompts and cues for the in-depth cognitive interviews of the evolving PRO.
Application of the IFC to AAV
The ICF is a system championed by the World Health Organization as a general health status framework that views health as being the result of a complex interaction among the ICF components of “body functions,” “body structures,” “activities and participation,” and “contextual factors” (personal and environmental)15. An ICF core set is a set of ICF categories (the basic units of ICF classification) that are considered essential to measure in clinical trials for a specific medical condition. The process by which ICF core sets are identified is analogous to the OMERACT process of identifying core domains, including incorporating the existing body of knowledge and input from those involved, especially including patients16. The VCRC-PCORI steering committee initiative is exploring various applications of the ICF in the context of AAV17 with the final aim of developing the ICF core set for AAV.
A systematic review of literature was performed to identify outcome measures that have been used in clinical trials of AAV. The content represented by items in each instrument was extracted. A 3-round Delphi exercise to identify aspects of AAV most important to clinicians is in the final stage of completion. Finally, a series of individual semistructured interviews was performed in collaboration with the other 2 projects to identify aspects of AAV most relevant to patients. Linking the identified outcomes to the ICF classification, according to previously published linking rules18,19, was performed to allow for comparison of aspects of AAV measured in clinical trials to those found to be important to patients and clinicians.
Presenting this work at OMERACT 12 generated productive discussions regarding the methods and future directions of the project. Ongoing input from PRP at all stages of development of ICF core sets for AAV was strongly encouraged, including patient participation at a final consensus meeting where all of the collected data would be combined and the final ICF core set would be identified. In addition, significant attention was devoted to discussing the role of the ICF in the OMERACT process. A number of novel applications of the ICF to the field of vasculitis were proposed for future consideration, most notable of which was incorporation of ICF with PROMIS to generate ICF-based instruments using PROMIS technology and elements.
The ICF is proving to be a useful complement to other processes for the development of AAV PRO outcomes. The development of the ICF core sets for AAV directly involves patients (among other key participants), and the overall process is consistent with the recent OMERACT Filter 2.0 approach17. The ICF core set for AAV could identify important domains not addressed by the current OMERACT core set for AAV that might be appropriate targets for development of new outcome measurement tools and provide input for a future update of the OMERACT core set.
Integration of the 3 Linked Projects
The 3 projects outlined above are highly integrated with one another and benefit substantially from shared resources, common investigators, discussion of overlapping concepts, and development of interoperable procedures and products. The VCRC-PCORI steering committee oversees all 3 projects, and the shared expertise, group memory, and familiarity greatly enhance the work on each individual project. Importantly, there is input of the PRP across all 3 projects (Table 1). Qualitative interviews inform all the projects, and the group is already considering how to combine and link PROMIS with a disease-specific PRO, with understanding emerging from the ICF mapping.
There is now a clear mandate from, and enthusiasm within, the vasculitis research community to embrace PRO as an important source of information that contributes to our understanding of the effect of the disease on our patients. Patient involvement is key to the success of these projects and is seen at every level, as shown in Table 1, including membership on the overarching steering committee, input into research protocols, and individual participation in semistructured interviews and online PROMIS data collection. The continued constructive feedback provided by the OMERACT community, including at the 2014 Workshop, in terms of both the overall approach to PRO in vasculitis and specific technical questions, is invaluable to the process of development of appropriate tools in this disease.
Acknowledgments
Sponsored by the Vasculitis Clinical Research Consortium, with support from the US National Institute of Arthritis and Musculoskeletal and Skin Diseases (U54 AR057319 and U01 AR51874), the National Center for Research Resources (U54 RR019497); and the Office of Rare Diseases Research and the National Center for Advancing Translational Science. Additional support for the work of the OMERACT vasculitis working group was received by a Patient-Centered Outcomes Research Institute Pilot Project Grant, the Medical Research Fund, Oxford, and the Oxfordshire Health Services Research Committee. Dr. Robson and Professor Luqmani are supported in part by the National Institute for Health Research Musculoskeletal Biomedical Research Unit, Oxford, UK. Dr. Milman was supported by a UCB/Canadian Rheumatology Association/Arthritis Society postgraduate rheumatology fellowship award and a research fellowship from the Department of Medicine at the Ottawa Hospital.
Contributor Information
Joanna C. Robson, Clinical Lecturer in Rheumatology, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Science, University of Oxford, Oxford, UK.
Nataliya Milman, Clinical Lecturer in Rheumatology, Division of Rheumatology, University of Ottawa, Ottawa, Ontario, Canada.
Gunnar Tomasson, Department of Public Health Sciences, University of Iceland, Reykjavik, Iceland.
Jill Dawson, University Research Lecturer and Senior Research Scientist, Nuffield Department of Population Health (HSRU), University of Oxford, Oxford, UK.
Peter F. Cronholm, Associate Professor, Department of Family Medicine and Community Health, University of Pennsylvania, Pennsylvania, USA.
Katherine Kellom, Senior Research Coordinator, Department of Family Medicine and Community Health, University of Pennsylvania, Philadelphia, Pennsylvania, USA.
Judy Shea, Professor of Medicine, Department of Medicine; Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania, USA.
Susan Ashdown, Patient Partner, King’s Sutton, Banbury, Northamptonshire, UK.
Maarten Boers, Professor of Clinical Epidemiology, Department of Epidemiology and Biostatistics; and Rheumatology, VU University Medical Center, Amsterdam, The Netherlands.
Annelies Boonen, Professor of Rheumatology, Division of Rheumatology, Maastricht University Medical Center and Caphri Research Institute, University Maastricht, Maastricht, The Netherlands.
George C. Casey, Jr., Patient Partner, Vasculitis Foundation, Denver, Colorado, USA.
John T. Farrar, Associate Professor of Epidemiology, Neurology, and Anesthesia, Department of Biostatistics and Epidemiology, University of Pennsylvania, Philadelphia, Pennsylvania, USA
Don Gebhart, Patient Partner, Columbus, Ohio, USA.
Jeffrey Krischer, Professor, Division of Biostatistics and Informatics, Department of Pediatrics, College of Medicine, University of South Florida, Tampa, Florida, USA.
Georgia Lanier, Patient Partner, Framingham, Massachusetts, USA.
Carol A. McAlear, Project Manager, Vasculitis Clinical Research Consortium, University of Pennsylvania, Pennsylvania, USA.
Jacqueline Peck, Patient Partner, Grove, Oxfordshire, UK.
Antoine G. Sreih, Assistant Professor Medicine, Division of Rheumatology, University of Pennsylvania, Philadelphia, Pennsylvania, USA.
Peter Tugwell, Professor of Medicine and Epidemiology and Community Medicine, University of Ottawa, Ottawa, Ontario, Canada.
Raashid A. Luqmani, Professor of Rheumatology, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Science, Oxford, UK.
Peter A. Merkel, Professor of Medicine and Epidemiology, Division of Rheumatology and Department of Biostatistics and Epidemiology, University of Pennsylvania, Pennsylvania, USA.
References
- 1.Robson J, Doll H, Suppiah R, Flossmann O, Harper L, Hoglund P, et al. Damage in the ANCA-associated vasculitides: long-term data from the European Vasculitis Study Group (EUVAS) therapeutic trials. Ann Rheum Dis. 2015;74:177–84. doi: 10.1136/annrheumdis-2013-203927. [DOI] [PubMed] [Google Scholar]
- 2.Merkel PA, Aydin SZ, Boers M, Direskeneli H, Herlyn K, Seo P, et al. The OMERACT core set of outcome measures for use in clinical trials of ANCA-associated vasculitis. J Rheumatol. 2011;38:1480–6. doi: 10.3899/jrheum.110276. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Herlyn K, Hellmich B, Seo P, Merkel PA. Patient-reported outcome assessment in vasculitis may provide important data and a unique perspective. Arthritis Care Res. 2010;62:1639–45. doi: 10.1002/acr.20276. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Seo P, Jayne D, Luqmani R, Merkel PA. Assessment of damage in vasculitis: expert ratings of damage. Rheumatology. 2009;48:823–7. doi: 10.1093/rheumatology/kep103. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Grayson PC, Amudala NA, McAlear CA, Leduc RL, Shereff D, Richesson R, et al. Illness perceptions and fatigue in systemic vasculitis. Arthritis Care Res. 2013;65:1835–43. doi: 10.1002/acr.22069. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Merkel PA, Aydin SZ, Boers M, Cornell C, Direskeneli H, Gebhart D, et al. Current status of outcome measure development in vasculitis. J Rheumatol. 2014;41:593–8. doi: 10.3899/jrheum.131248. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Basu N, Jones GT, Fluck N, MacDonald AG, Pang D, Dospinescu P, et al. Fatigue: a principal contributor to impaired quality of life in ANCA-associated vasculitis. Rheumatology. 2010;49:1383–90. doi: 10.1093/rheumatology/keq098. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Koutantji M, Harrold E, Lane SE, Pearce S, Watts RA, Scott DG. Investigation of quality of life, mood, pain, disability, and disease status in primary systemic vasculitis. Arthritis Rheum. 2003;49:826–37. doi: 10.1002/art.11471. [DOI] [PubMed] [Google Scholar]
- 9.Carpenter DM, Thorpe CT, Lewis M, Devellis RF, Hogan SL. Health-related quality of life for patients with vasculitis and their spouses. Arthritis Rheum. 2009;61:259–65. doi: 10.1002/art.24235. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Walsh M, Mukhtyar C, Mahr A, Herlyn K, Luqmani R, Merkel PA, et al. Health-related quality of life in patients with newly diagnosed antineutrophil cytoplasmic antibody-associated vasculitis. Arthritis Care Res. 2011;63:1055–61. doi: 10.1002/acr.20471. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Tomasson G, Boers M, Walsh M, LaValley M, Cuthbertson D, Carette S, et al. Assessment of health-related quality of life as an outcome measure in granulomatosis with polyangiitis (Wegener’s) Arthritis Care Res. 2012;64:273–9. doi: 10.1002/acr.20649. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.U.S. Department of Health and Human Services Food and Drug Administration. Guidance for industry patient-reported outcome measures: use in medical product development to support labelling claims. 2009 [Internet; accessed April 2, 2015.] Available from: www.fda.gov/downloads/Drugs/Guidances/UCM193282.pdf.
- 13.de Wit M, Abma T, Koelewijn-van Loon M, Collins S, Kirwan J. Involving patient research partners has a significant impact on outcomes research: a responsive evaluation of the international OMERACT conferences. BMJ Open. 2013;3:e003311. doi: 10.1136/bmjopen-2012-002241. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Cella D, Riley W, Stone A, Rothrock N, Reeve B, Yount S, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. J Clin Epidemiol. 2010;63:1179–94. doi: 10.1016/j.jclinepi.2010.04.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.World Health Organization. International classification of functioning, disability and health (ICF) Geneva: World Health Organization; 2001. [Internet. Accessed April 2, 2015.] Available from: www.who.int/classifications/icf. [Google Scholar]
- 16.Cieza A, Ewert T, Ustun TB, Chatterji S, Kostanjsek N, Stucki G. Development of ICF core sets for patients with chronic conditions. J Rehabil Med. 2004:9–11. doi: 10.1080/16501960410015353. [DOI] [PubMed] [Google Scholar]
- 17.Milman N, Boonen A, Merkel PA, Tugwell P. Mapping of the OMERACT core set for ANCA-associated vasculitis to the International Classification of Function, Disability and Health. Arthritis Care Res. 2015;67:255–63. doi: 10.1002/acr.22414. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Cieza A, Brockow T, Ewert T, Amman E, Kollerits B, Chatterji S, et al. Linking health-status measurements to the International Classification of Functioning, Disability and Health. J Rehabil Med. 2002;34:205–10. doi: 10.1080/165019702760279189. [DOI] [PubMed] [Google Scholar]
- 19.Cieza A, Geyh S, Chatterji S, Kostanjsek N, Ustun B, Stucki G. ICF linking rules: an update based on lessons learned. J Rehabil Med. 2005;37:212–8. doi: 10.1080/16501970510040263. [DOI] [PubMed] [Google Scholar]