Abstract
The concepts of minimal clinically important improvement (MCII) and patient acceptable symptomatic state (PASS) could help in interpreting results of trials involving patient-reported outcomes by translating the response at the group level (change in mean scores) into more clinically meaningful information by addressing the patient level as “therapeutic success (yes/no)”. The aims of the special-interest group (SIG) at OMERACT 8 were to discuss specific issues concerning the MCII and PASS concepts, especially the wording of the external anchor questions used to determine the MCII and PASS estimates and to move towards a consensus for the cut-off values to use as the MCII and PASS in the different outcome criteria. The purpose of this SIG at OMERACT 8 was to inform participants of the MCII and PASS concepts and to agree on MCII and PASS values for pain, patient global assessment and functional impairment.
Keywords: Health Status, Humans, Outcome Assessment (Health Care), Patient Satisfaction, Remission Induction, Rheumatic Diseases, diagnosis, physiopathology, therapy, Severity of Illness Index, Treatment Outcome
Keywords: Minimal clinically important difference, Patient acceptable symptomatic state, Response to therapy, Outcome criteria, Patient's persepctive
Introduction
Most rheumatologic diseases are chronic and symptomatic conditions. Thus, the aim of treatment is to improve patient symptoms, function and well-being, rather than to cure the disease. Consequently, patient-reported outcomes (PROs) are widely used in assessing the result of care in clinical practice, clinical trials, and longitudinal epidemiological studies.(1, 2) Most of the tools evaluating these subjective criteria in rheumatic disorders measure continuous variables (scores). Thus, with the exception of summary indices such as ACR 20/50/70, (3) results of clinical trials for symptoms [eg pain] and signs [eg joint count] are usually expressed as continuous data at the group level (e.g., mean change or effect size), which is difficult to interpret and cannot easily be translated to the level of an individual response. Clinicians need to know how many patients showed a response, what was their level of response, and how many patients are doing well. But what constitutes a clinically relevant therapeutic success with PROs? Translating these continuous criteria (e.g., WOMAC score) to more clinically meaningful information such as “therapeutic success (yes/no)” would be helpful in better understanding the results of trials. However, the cut-off used for the dichotomization must be clinically relevant.
Two concepts can be distinguished in interpreting PRO scores at the individual level: i) the concept of improvement [which can be relative or absolute] and ii) the concept of a state of well-being. For the concept of improvement, the minimal clinically important difference (MCID)(4–6) or minimal clinically important improvement’ (MCII),(7) defined as the smallest change in measurement that signifies an important difference/improvement in a symptom, could be used. In this paper, because of the direction of effect in most trials being improvement and because of now-documented differences in the MCID for deterioration and improvement (8), we focus only on the MCII. For the concept of status of well-being at any one point in time (usually after treatment in a trial setting), a few definitions are being used: first, the patient acceptable symptom state (PASS),(9) second, low disease activity (LDA),(10) now renamed minimal disease activity (MDA), and third, remission. The PASS is defined as a symptom state that the patients consider acceptable. MDA is now defined in rheumatoid arthritis as ‘that state deemed a useful target of treatment by both the physician and the patient given current treatments and knowledge’ and is thus placed between activity of the disease and (close to) remission.(11) The MDA for rheumatoid arthritis is based on the opinion of physician and patient, whereas the PASS is strictly a patient reported outcome. Classifying individuals’ conditions as being below a threshold level of disability or pain allows for the description of the proportion of therapeutic success or failure (patients who have improved or achieved an acceptable state or not) in addition to mean effects at the group level. Such analysis would add useful information and aid in the interpretation of trial and longitudinal results to decide whether a treatment should be used. Accurate methods for such analysis would result in improved decision tools that are dependent on a “responder analysis” at each node in a decision tree.
The question of what constitutes an important improvement or an acceptable state is an issue of increasing interest. At the moment, no consensus exists on the methods that should be used to determine the MCID/MCII. During OMERACT 5, 6 and 7, the anchor-based method, an external judgment of the importance of change as the anchor, was recommended.(12, 13) Moreover, the patient’s perspective was recommended as the perspective to use for the anchor.(14) Furthermore, the importance of standardizing the wording of the anchor question has been raised (15) and needs to be addressed.
The concept of MDA was addressed during OMERACT 6 and 7.(10, 11) These OMERACT modules aimed at seeking a consensus on the definition of MDA in rheumatoid arthritis (RA), based on patients’ profiles. So the MDA state was defined for a prespecified core set of criteria in RA. Similarly, the ASAS working group has developed the ASAS partial remission criteria for ankylosing spondylitis, based on an aggregation of a core set of items. However, the main outcome criterion in clinical trials often concerns a single domain of symptom (e.g., pain, function), and the concept of MDA for these specific domains has not been thoroughly evaluated.
Since OMERACT 7, knowledge in the field of MCID/MCII and MDA/PASS has increased, moving from theory to practice. Particularly, the determination of MCII and PASS estimates from several studies in different diseases allows for investigating some of the properties of these criteria.
During OMERACT 8, the aim of the SIG was to discuss specific issues concerning the MCII and PASS concepts and to move towards a consensus for use of the MCII and PASS: Which external anchor should be used? Should the MCII, the PASS, or both, be recommended? What should the MCII and PASS values be? Should the effect of some covariates be taken into account?
Specific issues addressed
Standardization of the wording of the external anchor question to determine the MCII and PASS
Various questions with different wordings have been used as an external anchor for the MCII and PASS. In addition, the response modalities and the group for which MCII is defined (patients with fair response to therapy, good response to therapy, very good response to therapy, etc.) have been shown to have an impact on the MCII estimates.(15) For the PASS value determination, the issue of the time spent in the state is very important for the wording of the anchor question (Was the patient asked if his symptom state was acceptable if he was to remain in that state for 48 hours, 3 months or for the rest of his life?). Standardization is needed to allow comparison across studies, diseases and/or languages.
The review of these different issues was the basis of a discussion that led to the survey we report below.
Should the MCII and PASS be treatment specific or the same whatever the treatment evaluated?
Determining treatment-specific MCII and PASS values (i.e., a PASS for evaluating NSAID therapy different from a PASS for evaluating anti-TNFα therapy) allows for taking into account the different levels of patients’ expectations for the treatment. Actually, whether patients consider a state (or a change) satisfactory independently of the treatment they receive (i.e., whether the PASS values are related to patients’ expectations of the treatment) is not known. One may hypothesize, for instance, that patients expect stronger effects from a TNFα antagonist than from NSAID therapy and thus would consider a lower level of symptoms as satisfactory with TNFα antagonist therapy. This point should be investigated in a further study. The drawback of using treatment-specific PASS or MCII values is that these values should be regularly updated as treatment options and knowledge and expectations about them evolve.(2, 4)
Relation between MCII and PASS
The relative meaning of the MCII and PASS was unknown. Whether the concept of improvement, remission, or both, should be recommended was discussed and was addressed in a survey following the SIG session, reported below. The results on how the MCII and PASS are interrelated in a study of hip and knee osteoarthritis and acute rotator cuff syndrome(16) was presented. In pain and function scores, the resulting MCII values corresponded with the amount of improvement needed to reach the level defined by the PASS. The MCII appears to be the amount of change in status required which will allow the patient to achieve the PASS. It seems that patients consider that they experienced an important improvement only if this improvement allows them to achieve a satisfactory state, a state in which they feel good. Consequently, it seems that what is important to patients is to feel good (concept of PASS) rather than to feel better (concept of MCII).
Review of the existing values determined for MCII and PASS for different outcome criteria in different diseases
MCID values have been determined in some rheumatic diseases, such as chronic low back pain,(17) and hip or knee osteoarthritis.(18–20)
The MCII has been estimated for pain, patient global assessment and the WOMAC function subscale in hip and knee osteoarthritis patients in France.(7) The definition of MCII and PASS was chosen by a group of experts rheumatologists, who are members of this OMERACT SIG: the external anchor used was patients’ evaluation of their response to therapy on a 5-point Likert scale ‘none = no good at all, ineffective drug; poor = some effect but unsatisfactory; fair = reasonable effect but could be better; good = satisfactory effect with occasional episodes of pain or stiffness; excellent = ideal response, virtually pain free’. The MCII was defined as the 75th percentile of the change in score between the baseline and final visit among patients whose final evaluation of response to therapy was good. The MCII was defined in the group of patients whose evaluation of response to therapy was “good”, because one is always looking for clinically important differences. Patients whose evaluation of response to therapy was “excellent” were not included since our target was the minimal change important in the patient’s perspective. Following the same methodology, the MCII was estimated in acute rotator cuff syndrome in France, for pain and Neer index(16) which is a score of shoulder function(21), and in ankylosing spondylitis in Norway, for pain, night pain, patient’s global assessment, the BASDAI and BASFI.(15)
The PASS has been determined in studies of hip or knee osteoarthritis,(9) acute rotator cuff syndrome(16) and ankylosing spondylitis,(22) defined as the 75th percentile of the score at the final visit for patients who considered their state satisfactory at the end of the study.
At the SIG session, MCII and PASS estimates were presented from a multinational cohort study of similar design in the main outcome criteria in hip and knee osteoarthritis, hand osteoarthritis, rheumatoid arthritis, ankylosing spondylitis and low back pain for disease-specific criteria (e.g., BASDAI in ankylosing spondylitis), and for generic criteria (e.g., pain as measured on a numeral rating scale). This study involved 8 countries (6 languages): France, Spain, United Kingdom, Belgium, the Netherlands, Australia, Lebanon, and Morocco. These results gave a basis for discussion on the best values for the MCII and PASS in different outcome criteria.
The data supported moving towards a consensus on cutoff values for generic criteria (pain, patient’s global assessment of disease activity, and function, all measured with visual analog scale or numerical rating scale) but not for disease-specific criteria (e,g., BASDAI in ankylosing spondylitis), for which more work was needed (table 2).
Table 2.
1) The wording of the external anchor | |||||
1) Should the external anchor be symptom specific (i.e. addressing improvement in a particular symptom) or general? | |||||
Symptom specific | 42% | ||||
General | 50% | ||||
Both | 8% | ||||
Don’t know | 0% | ||||
If you answered ‘symptom specific’, which one do you prefer? | |||||
Examples for the MCII (same issue for the PASS) | |||||
“Has there been a change in your level of functional impairment since you started the study?” | 61% | ||||
“Think only about the difficulty you had in doing daily physical activities due to your (name of rheumatic disease) during the last 48 hours. Compared to when you started the study, how has the level of difficulty been during the last 48 hours?” | 39% | ||||
If you answered ‘general, which one do you prefer? | |||||
Examples for the MCII (same issue for the PASS) | |||||
“Taking into account all the ways your (name of rheumatic disease) is affecting you, your level of pain and your functional impairment, how would you rate your response to the medication you have received for your arthritis for 4 weeks ?” | 38% | ||||
“Think about all the ways your (name of rheumatic disease) has affected you during the last 48 hours. Compared to when you started the study, how have you been during the last 48 hours?” | 62% | ||||
2) What should be the time frame in the external question to estimate the PASS? | |||||
no time frame: “Taking into account all you have to do during your daily life, your level of pain, and your functional impairment, do you consider that your current state is satisfactory?” | 43% | ||||
“Think about all the ways your (name of rheumatic disease) has affected you during the last 48 hours. If you were to remain in the next few months as you were during the last 48 hours, would this be acceptable or unacceptable to you?” | 43% | ||||
“Think about all the ways your (name of rheumatic disease) has affected you during the last 48 hours. If you were to remain for the rest of your life as you were during the last 48 hours, would this be acceptable or unacceptable to you?” | 9% | ||||
3/35 (8.6%) | |||||
Others | 6% | ||||
3) Do you agree that we have enough data to propose the endorsement of estimates of MCII and PASS for the 3 generic criteria (pain, patient global assessment of disease activity and one-item global function)? | |||||
YES | 40% | ||||
NO | 29% | ||||
Don’t know | 31% | ||||
4) If yes, what is the best proposal for these different criteria? | |||||
MCII (abs*) for Pain | MCII (rel**) for Pain | PASS for Pain | |||
−10 | 7% | −10% | 0% | 20 | 7% |
−15 | 27% | −15% | 7% | 30 | 33% |
−20 | 53% | −20% | 40% | 35 | 13% |
−30 | 7% | −30% | 33% | 40 | 40% |
−40 | 7% | −50% | 20% | 50 | 7% |
MCII (abs*) for Patient global assessement | MCII (rel**) for Patient global assessement | PASS for Patient global assessement | |||
−10 | 7% | −10% | 0% | 20 | 19% |
−15 | 20% | −15% | 0% | 30 | 13% |
−20 | 60% | −20% | 40% | 35 | 19% |
−30 | 7% | −30% | 29% | 40 | 44% |
−40 | 7% | −50% | 29% | 50 | 6% |
MCII (abs*) for function | MCII (rel**) for function | PASS for function | |||
−10 | 27% | −10% | 0% | 20 | 13% |
−15 | 33% | −15% | 23% | 30 | 13% |
−20 | 27% | −20% | 54% | 35 | 25% |
−30 | 7% | −30% | 15% | 40 | 44% |
−40 | 7% | −50% | 8% | 50 | 6% |
What is the impact of various parameters on the MCII and PASS estimates?
In the hip or knee osteoarthritis study,(7) the MCII was shown to vary greatly across tertiles of baseline scores and age. This impact of the baseline level of symptom was reduced in part only when relative change instead of absolute change was used. Gender and disease duration did not appear to affect the MCII value. The impact of the baseline value had also been previously demonstrated by Riddle and colleagues(17) in their investigation of low back pain, in which the MCID varied between 3 and 13 depending on the baseline range of scores (from the Roland Morris Back Pain Questionnaire). Patients dealing with the most severe symptoms have to experience a greater absolute change to consider themselves improved.
In the hip or knee osteoarthritis study, the PASS was more constant across tertiles of the baseline score than the MCII, and age, gender and disease duration did not affect the PASS.(9)
An important aspect of any desirable state is the time spent in that state. In the ankylosing spondylitis study, the PASS was shown to be stable over 10 weeks.(22) This key finding supports the use of the PASS values to describe patients achieving and maintaining such a state for a specified period of time. This point should be confirmed in a study with a longer follow-up.
Would modification of the values for the MCII and PASS have an impact on the evaluation of the treatment effect?
One of the objectives of the SIG was to move towards a consensus of the MCII and PASS value to recommend, so the knowing whether there is an impact of the choice of the cut-off value on the evaluation of the treatment effect is important. Would it be relevant to propose rounded values, for instance, for a goal of simplicity? Some members of the group are investigating this point.
Summary of progress made at OMERACT 8
The SIG session was attended by patients, researchers and clinicians.
A point of debate was the place of the MCII and PASS in rheumatology. Some participants raised the important issue of the loss of power, which is the cost of dichotomizing continuous variables.(23–25) The group made clear that the MCII and PASS aim at reporting the proportion of improved patients and the proportion of patients in an acceptable state in addition to the conventional way of reporting the results (e.g., difference in mean change). The aim is to provide complementary and more meaningful information to help with the interpretation of trial results. Participants also suggested that these concepts were interesting tools to enhance physician-patient communication.
Using feedback from the SIG session, we proposed first a survey of session attendees during the OMERACT 8 meeting. A total of 35 people completed the survey (2 patients, 1 clinician, 25 academic researchers and 7 industry researchers). The input of the participants confirmed the relevance of the use of the MCII and PASS in rheumatology. Most respondents considered that OMERACT should recommend the reporting of the percentage of patients in an acceptable state plus or minus the percentage of improved patients and that OMERACT members should consider adding external anchors to determine MCII and PASS estimates in their clinical studies (table 1).
Table 1.
Three questions were voted on: | |||
1) In addition to the conventional way of reporting results of trial using patient-reported outcomes measured with continuous variables (e.g. mean change in pain 0–100 VAS), should OMERACT recommend adding reports of percentage of improved patients (MCII concept) and/or percentage of patients in an acceptable state (PASS concept) | |||
88% YES | 12% NO | ||
2) If yes, are you in favor of | |||
- Reporting the percentage of improved patients at the end of the study (MCII) | 10% | ||
- Reporting the percentage of patients in an acceptable symptomatic state at the end of the study (PASS) | 45% | ||
- Reporting both | 45% | ||
3) OMERACT members should consider adding external anchors (“gold standard” questions) to their clinical studies in order to estimat MCII or PASS values and to contribute to our understanding of how they work in practice. | |||
84% YES | 16% NO |
The next step was an email survey after OMERACT 8 to the SIG participants addressing the wording of the external anchors and the best values of the MCII and PASS in the generic outcome criteria: for a given criteria, same MCII and PASS values whatever the disease (table 2). A total of 36 people completed the email survey (5 patients, 2 clinicians, 26 academic researchers and 3 industry researchers). Advice was divided on whether the external anchor should be symptom specific or general. Most of the participants felt that the external anchor to estimate the PASS should contain no element of time frame or a short time frame (in the next few months or in the next week) rather than “for the rest of your life”. Of the respondents, 40% thought that we have enough data to propose the endorsement of cutoff values for the MCII and PASS (52% among the academic researchers). The most frequently chosen value of MCII for absolute change was −20 for pain and patient global assessment, but advice was divided for function (−10, −15, or −20). The most frequently chosen value of MCII for relative change was −20% for pain, patient global assessment and function. The most frequently chosen value of PASS was 40 for pain (i.e. patients are considered in an acceptable state if their pain score is below 40 on a 0–100 scale), patient global assessment, and function.
In summary, this SIG session at OMERACT 8 allowed productive discussions of the MCII and PASS concepts and validation of the usefulness of these concepts in rheumatology by the OMERACT participants with examples of their application to pain, patient global assessment and function (measured on a VAS or a NRS). More work is needed to move towards a consensus on the wording of the external anchor, and to propose MCII and PASS values for disease-specific criteria, using a data driven approach in different diseases and with different wordings of the anchor question. Including external anchors in clinical studies to determine the MCII and PASS, especially for disease-specific criteria, will be very useful.
Acknowledgments
Source of support: Florence Tubach obtained a EULAR/OMERACT Travel Bursary to attend the OMERACT8 meeting
References
- 1.Fairclough DL. Patient reported outcomes as endpoints in medical research. Stat Methods Med Res. 2004;13(2):115–38. doi: 10.1191/0962280204sm357ra. [DOI] [PubMed] [Google Scholar]
- 2.Willke R, Burke B, Erickson P. Measuring treatment impact: a review of patient-reported outcomes and other efficacy endpoints in approved product labels. Control Clin Trials. 2004;25(6):535–552. doi: 10.1016/j.cct.2004.09.003. [DOI] [PubMed] [Google Scholar]
- 3.Felson DT, Anderson JJ, Boers M, et al. American College of Rheumatology. Preliminary definition of improvement in rheumatoid arthritis. Arthritis Rheum. 1995;38(6):727–35. doi: 10.1002/art.1780380602. [DOI] [PubMed] [Google Scholar]
- 4.Beaton DE, Boers M, Wells GA. Many faces of the minimal clinically important difference (MCID): a literature review and directions for future research. Curr Opin Rheumatol. 2002;14(2):109–14. doi: 10.1097/00002281-200203000-00006. [DOI] [PubMed] [Google Scholar]
- 5.Jaeschke R, Singer J, Guyatt GH. Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials. 1989;10(4):407–15. doi: 10.1016/0197-2456(89)90005-6. [DOI] [PubMed] [Google Scholar]
- 6.Juniper EF, Guyatt GH, Willan A, Griffith LE. Determining a minimal important change in a disease-specific Quality of Life Questionnaire. J Clin Epidemiol. 1994;47(1):81–7. doi: 10.1016/0895-4356(94)90036-1. [DOI] [PubMed] [Google Scholar]
- 7.Tubach F, Ravaud P, Baron G, et al. Evaluation of clinically relevant changes in patient-reported outcomes in knee and hip osteoarthritis: the Minimal Clinically Important Improvement. Ann Rheum Dis. 2005;64(1):29–33. doi: 10.1136/ard.2004.022905. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Schwartz AL, Meek PM, Nail LM, et al. Measurement of fatigue. determining minimally important clinical differences. J Clin Epidemiol. 2002;55(3):239–44. doi: 10.1016/s0895-4356(01)00469-3. [DOI] [PubMed] [Google Scholar]
- 9.Tubach F, Ravaud P, Baron G, et al. Evaluation of clinically relevant states in patient-reported outcomes in knee and hip osteoarthritis: the Patient Acceptable Symptom State. Ann Rheum Dis. 2005;64(1):34–37. doi: 10.1136/ard.2004.023028. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Wells G, Boers M, Shea B, et al. MCID/Low Disease Activity State Workshop: Low Disease Activity State in Rheumatoid Arthritis. J Rheumatol. 2003;30(5):1110–1. [PubMed] [Google Scholar]
- 11.Wells GA, Boers M, Shea B, et al. Minimal disease activity for rheumatoid arthritis: a preliminary definition. J Rheumatol. 2005;32(10):2016–24. [PubMed] [Google Scholar]
- 12.Wells G, Anderson J, Beaton D, et al. Minimal clinically important difference module: summary, recommendations, and research agenda. J Rheumatol. 2001;28(2):452–4. [PubMed] [Google Scholar]
- 13.Wells G, Beaton D, Shea B, et al. Minimal clinically important differences: review of methods. J Rheumatol. 2001;28(2):406–12. [PubMed] [Google Scholar]
- 14.Crosby RD, Kolotkin RL, Williams GR. Defining clinically meaningful change in health-related quality of life. J Clin Epidemiol. 2003;56(5):395–407. doi: 10.1016/s0895-4356(03)00044-1. [DOI] [PubMed] [Google Scholar]
- 15.Pham T, Tubach F, Skomsvoll JF, et al. Minimal Clinically Important Improvement (MCII) determination in patients with Ankylosing Spondylitis (AS) is dependent on the response modalities. Meeting of the American College of Rheumatology; San Diego, USA. 2005. [Google Scholar]
- 16.Tubach F, MD, BF, GB, IL, PR Feeling good rather than feeling better matters more to patients. Arthritis Rheum. 2006;55(4):526–30. doi: 10.1002/art.22110. [DOI] [PubMed] [Google Scholar]
- 17.Riddle DL, Stratford PW, Binkley JM. Sensitivity to change of the Roland-Morris Back Pain Questionnaire: part 2. Phys Ther. 1998;78(11):1197–207. doi: 10.1093/ptj/78.11.1197. [DOI] [PubMed] [Google Scholar]
- 18.Angst F, Aeschlimann A, Michel BA, Stucki G. Minimal clinically important rehabilitation effects in patients with osteoarthritis of the lower extremities. J Rheumatol. 2002;29(1):131–8. [PubMed] [Google Scholar]
- 19.Bellamy N, Carette S, Ford PM, et al. Osteoarthritis antirheumatic drug trials. III. Setting the delta for clinical trials--results of a consensus development (Delphi) exercise. J Rheumatol. 1992;19(3):451–7. [PubMed] [Google Scholar]
- 20.Ehrich EW, Davies GM, Watson DJ, Bolognese JA, Seidenberg BC, Bellamy N. Minimal perceptible clinical improvement with the Western Ontario and McMaster Universities osteoarthritis index questionnaire and global assessments in patients with osteoarthritis. J Rheumatol. 2000;27(11):2635–41. [PubMed] [Google Scholar]
- 21.Neer CS, 2nd, Watson KC, Stanton FJ. Recent experience in total shoulder replacement. J Bone Joint Surg Am. 1982;64(3):319–37. [PubMed] [Google Scholar]
- 22.Tubach F, Pham T, Skomsvoll JF, et al. Stability of the patient acceptable symptomatic state over time in outcome criteria in ankylosing spondylitis. Arthritis Rheum. 2006 Dec 15;55(6):960–3. doi: 10.1002/art.22342. [DOI] [PubMed] [Google Scholar]
- 23.MacCallum R, Zhang S, Preacher K, Rucker D. On the practice of dichotomization of quantitative variables. Psychol Methods. 2002;7(1):19–40. doi: 10.1037/1082-989x.7.1.19. [DOI] [PubMed] [Google Scholar]
- 24.Cohen J. The cost of dichotomization. Applied Psychol Meas. 1983;7:249–53. [Google Scholar]
- 25.Altman DG, Royston P. The cost of dichotomising continuous variables. BMJ. 2006;332(7549):1080. doi: 10.1136/bmj.332.7549.1080. [DOI] [PMC free article] [PubMed] [Google Scholar]