Discrepancies between modified Medical Research Council dyspnea score and COPD assessment test score in patients with COPD

Abstract

Background and objective

According to the Global Initiative for Chronic Obstructive Lung Disease (GOLD) guidelines, either a modified Medical Research Council (mMRC) dyspnea score of ≥2 or a chronic obstructive pulmonary disease (COPD) assessment test (CAT) score of ≥10 is considered to represent COPD patients who are more symptomatic. We aimed to identify the ideal CAT score that exhibits minimal discrepancy with the mMRC score.

Methods

A receiver operating characteristic curve of the CAT score was generated for an mMRC scores of 1 and 2. A concordance analysis was applied to quantify the association between the frequencies of patients categorized into GOLD groups A–D using symptom cutoff points. A κ-coefficient was calculated.

Results

For an mMRC score of 2, a CAT score of 15 showed the maximum value of Youden’s index with a sensitivity and specificity of 0.70 and 0.66, respectively (area under the receiver operating characteristic curve [AUC] 0.74; 95% confidence interval [CI], 0.70–0.77). For an mMRC score of 1, a CAT score of 10 showed the maximum value of Youden’s index with a sensitivity and specificity of 0.77 and 0.65, respectively (AUC 0.77; 95% CI, 0.72–0.83). The κ value for concordance was highest between an mMRC score of 1 and a CAT score of 10 (0.66), followed by an mMRC score of 2 and a CAT score of 15 (0.56), an mMRC score of 2 and a CAT score of 10 (0.47), and an mMRC score of 1 and a CAT score of 15 (0.43).

Conclusion

A CAT score of 10 was most concordant with an mMRC score of 1 when classifying patients with COPD into GOLD groups A–D. However, a discrepancy remains between the CAT and mMRC scoring systems.

Introduction

In the new Global Initiative for Chronic Obstructive Lung Disease (GOLD) guidelines, the patients’ symptoms can be assessed by either the modified Medical Research Council (mMRC) dyspnea score or the chronic obstructive pulmonary disease (COPD) assessment test (CAT) score.¹ MRC was developed to help physicians establish clinical grades of breathlessness for their patients with emphysema.^2,3 A modified version of this scale is used today and is based on five stages of breathlessness due to exertion.^3,4 mMRC was well correlated with dyspnea index and lung function.⁴ The CAT is an eight-item unidimensional measure of health status impairment in COPD.^5,6 It was well correlated with St George’s Respiratory Questionnaire for COPD patients (SGRQ-C)⁵ and provided a surprising insight into the severity of the effects of COPD.⁷ CAT score was developed to be applicable worldwide and validated translations are available in a wide range of languages.⁶

According to the GOLD guidelines, an mMRC score of ≥2 or a CAT score of ≥10 are considered to represent patients with more symptoms.¹ However, the composition of the mMRC and CAT scoring systems is different. The mMRC involves a simple question regarding the patient’s degree of dyspnea. The CAT involves eight questions regarding not only dyspnea but also other components of quality of life (cough, phlegm, chest tightness, breathlessness, activity, confidence, sleep, and energy).⁵ Because of this difference, a discrepancy between the mMRC and CAT scores in the same patients with COPD is expected.

Discrepancy between the mMRC and CAT score is well described in previous studies. Kim et al⁸ showed that classification of COPD by the mMRC or CAT score was not identical. Rieger-Reyes et al⁹ showed more than 25% of patients were reclassified into different categories by these two scores. Moreover, Jones et al³ described that cutoff points of an mMRC score of 1 and a CAT score of 10 were approximately equivalent. However, there are still some questions that remain unanswered. First, what is an ideal cutoff point of CAT which is equivalent to an mMRC score of 1 or 2 (calculated by statistical method)? Second, what will be the distribution of patients according to the new cutoff point of CAT equivalent to an mMRC score of 1 or 2? Third, how will the discrepancy be changed according to four different cutoff points?

To answer these questions, we aimed to examine the discrepancy between the mMRC and CAT scores in patients with COPD. We also aimed to identify the ideal CAT score that exhibits minimal discrepancy with the mMRC score. Finally, we compared the extent of agreement between the mMRC and the CAT scores in different settings.

Methods

This study analyzed data obtained from the Korean COPD Subgroup Study cohort (KOCOSS).¹⁰ Briefly, this cohort was initiated in April 2012 and is currently ongoing. On May 15, 2015, approximately 1,148 patients were enrolled from 46 university-affiliated hospitals from Republic of Korea. Each of the participating hospitals had a bed capacity of 345–2,806 beds. The number of patients enrolled per hospital was 24.9±3.2 (mean ± standard error of the mean [SEM]). In the present study, we analyzed 790 patients with COPD who were enrolled until July 31, 2014. The enrollment criteria for this cohort were an age of ≥40 years and a post-bronchodilator forced expiratory volume in 1 second (FEV₁)/forced vital capacity (FVC) of <0.7. Parameters collected at the time of enrollment included age, sex, smoking history, pulmonary function test results, CAT score, mMRC score, SGRQ-C score, previous history of exacerbation, frequency of emergency room (ER) visits, admission into ICU, duration since COPD diagnosis, duration of COPD treatment, presence of chronic bronchitis symptoms, education, comorbidities (myocardial infarction, congestive heart failure, peripheral vascular disease, diabetes, high blood pressure, osteoporosis, gastroesophageal reflux disease, hyperlipidemia, thyroid disease, and inflammatory bowel disease), previous history of diseases (bronchiectasis, chronic bronchitis, emphysema, COPD, tuberculosis, measles, pertussis, pneumonia, asthma, allergic rhinitis, atopic dermatitis, and drug allergy), area of living, biomass fuel exposure, occupational history, present medication for COPD, weight, height, and results of tests (6 minutes walking distance, chest X-ray, electrocardiography, computed tomography scan, and laboratory test).

This study was approved by the ethics committees of: Gacheon University Gil Medical Center, Hallym University Kangnam Sacred Heart Hospital, Gangnam Severance Hospital, Kyung Hee University Hospital at Gangdong, Hallym University Kangdong Sacred Heart Hospital, Kangbuk Samsung Hospital, Kangwon National University Hospital, Konkuk University Hospital, Konkuk University Chungju Hospital, Kyungpook National University Hospital, Gyeongsang National University Hospital, Korea University Guro Hospital, Korea University Anam Hospital, Seoul Eulji Hospital, Dongguk University Gyeongju Hospital, Dongguk University Ilsan Hospital, Keimyung University Dongsan Medical Center Dong-A University Hospital, Hallym University Dongtan Sacred Heart Hospital, Pusan National University Hospital, Inje University Busan Paik Hospital, The Catholic University of Korea Bucheon St. Mary’s Hospital, Soonchunhyang University Hospital Bucheon, Seoul National University Bundang Hospital, Bundang CHA Hospital, Seoul Metropolitan Government Seoul National University Boramae Medical Center, Sam-sung Medical Center, Soonchunhyang University Hospital Seoul, The Catholic University of Korea Seoul St. Mary’s Hospital, The Catholic University of Korea St. Paul’s Hospital, The Catholic University of Korea St. Vincent’s Hospital, Severance Hospital, Asan Medical Center, Ajou University Hospital, The Catholic University of Korea Yeouido St. Mary’s Hospital, The Catholic University of Korea Uijeongbu St. Mary’s Hospital, Yeungnam University Medical Center, Ulsan University Hospital, Wonkwang University Sanbon Hospital, Wonju Severance Christian Hospital, Ewha Womans University Mokding Hospital, Incheon St. Mary’s Hospital, Inha University Hospital, Chonnam National University Hospital, Chonbuk National University Hospital, Jeju National University Hospital, Soonchunhyang University Hospital Cheonan, Hallym University Chuncheon Sacred Heart Hospital, Hallym University Sacred Heart Hospital, Hanyang University Guri Hospital. All subjects gave written informed consent. We obtained approval from each institution to use the patients’ records for this study and patients’ confidentiality was maintained.

According to the 2015 GOLD guidelines,⁶ patients were classified into either a high- or a low-risk group according to their FEV₁ (%) or previous history of exacerbation. Patients with either an FEV₁ of <50%, or a history of more than two exacerbations, or a history of admission were classified as high-risk patients. Patients were also classified as either less or more symptomatic by using the mMRC or CAT score. According to these scores, patients were classified into group A (less symptomatic and low-risk), B (more symptomatic and low-risk), C (less symptomatic and high-risk), and D (more symptomatic and high-risk). Using the CAT and mMRC scores in our cohort, we classified patients with COPD into GOLD groups A–D and analyzed the discrepancies among these groups according to the CAT and mMRC settings.

Statistical analysis

All the statistical analyses were performed using PASW Statistics, version 17.0 (SPSS, Inc., Chicago, IL, USA). Pearson’s correlation coefficient (R) was used to assess whether a relationship was present between continuous variables. A receiver operating characteristic (ROC) curve of the CAT scores was generated for mMRC scores of 1 and 2. Youden’s index was used to find cutoff point for best combination of sensitivity and specificity. A concordance analysis by a measure of Cohen’s kappa (κ) was applied to quantify the association between the frequencies of patients categorized into GOLD groups A–D using the following symptom cutoff points: CAT score of ≥10 versus mMRC score of ≥1, CAT score of ≥10 versus mMRC score of ≥2, CAT score of ≥15 versus mMRC score of ≥1, and CAT score of ≥15 versus mMRC score of ≥2, respectively. A κ-coefficient was calculated (a value of 1 indicates perfect agreement). P-values of <0.05 were considered to indicate statistical significance. Data are expressed as means ± standard deviation (SD).

Results

Patients

The baseline characteristics of the 790 enrolled patients with COPD are described in Table 1. Their mean age was 71.1±7.8 years, and 90.8% of the patients were male. Most of the patients (96.2%) were current or ex-smokers. The mean FEV₁ (%) was 55.7±16.7%, and the majority of patients were in GOLD stage 2 or 3.

Table 1.

Baseline characteristics of patients (n=790)

Characteristics	Mean ± SD or n (%)
Age (years)	71.1±7.8
Male	720 (90.8)
Smoking
Current	220 (27.8)
Ex	540 (68.4)
Never	30 (3.8)
pBD FVC (L)	3.0±0.8
pBD FVC (%)	81.7±17.6
pBD FEV1 (L)	1.5±0.5
pBD FEV1 (%)	55.7±16.7
GOLD stage 1	39 (4.9)
GOLD stage 2	443 (56.1)
GOLD stage 3	259 (32.8)
GOLD stage 4	49 (6.2)
pBD FEV1/FVC (%)	49.0±12.0
CAT	15.6±7.8
mMRC	1.6±1.0
SGRQ-C	35.0±19.7

Abbreviations: SD, standard deviation; pBD, post-bronchodilator; FVC, forced vital capacity; FEV₁, forced expiratory volume in 1 second; GOLD, Global Initiative for Chronic Obstructive Lung Disease; CAT, COPD assessment test; mMRC, modified Medical Research Council; SGRQ-C, St George’s Respiratory Questionnaire for COPD patients.

Correlation among CAT, mMRC, and SGRQ-C

There was a significant correlation between the CAT and mMRC scores (P<0.01). However, the correlation was not strong (R=0.49). There was wide variation in the CAT scores among patients with the same mMRC score. For example, the minimum and maximum CAT scores were 0 and 34, respectively, in patients with an mMRC score of 0. Similarly, the minimum and maximum CAT scores were 11 and 40, respectively, in patients with an mMRC score of 4. On the other hand, the CAT score was more strongly correlated with the SGRQ-C score (R=0.74, P<0.01; Figure 1). A histogram of CAT scores according to mMRC score also showed wide variation in the CAT score in each mMRC group (Figure 2).

Figure 1.

Correlation between CAT and other scores.

Notes: (A) Correlation between CAT and mMRC scores. Although there was a significant correlation between the CAT and mMRC scores, the correlation was not strong. (B) Correlation between CAT and SGRQ-C scores. The CAT score was well correlated with the SGRQ-C score.

Abbreviations: CAT, COPD assessment test; mMRC, modified Medical Research Council; SGRQ-C, St George’s Respiratory Questionnaire for COPD patients.

Figure 2.

Histogram of CAT scores according to mMRC score.

Note: There was a wide variation in the CAT scores in each mMRC group.

Abbreviations: CAT, COPD assessment test; mMRC, modified Medical Research Council.

Cutoff CAT score for mMRC score

An ROC curve was used to identify the CAT score that was best correlated with the mMRC score. For an mMRC score of 2, a CAT score of 15 showed the maximum value of Youden’s index with a sensitivity and specificity of 0.70 and 0.66, respectively (area under the ROC curve [AUC] 0.74; 95% confidence interval [CI], 0.70–0.77; P<0.01). For an mMRC score of 1, a CAT score of 10 showed the maximum value of Youden’s index with a sensitivity and specificity of 0.77 and 0.65, respectively (AUC 0.77; 95% CI, 0.72–0.83; P<0.01; Figure 3).

Figure 3.

ROC curve of CAT score.

Notes: ROC curve for (A) an mMRC score of 2 and (B) an mMRC score of 1. A CAT score of 15 was compatible with an mMRC score of 2, and a CAT score of 10 was compatible with an mMRC score of 1.

Abbreviations: ROC, receiver operating characteristic; CAT, COPD assessment test; mMRC, modified Medical Research Council; AUC, area under the ROC curve; CI, confidence interval.

Classification of patients with COPD according to CAT or mMRC score

Patients with COPD were classified into GOLD groups A–D under four conditions (mMRC score =1, CAT score =10, mMRC score =2, and CAT score =15). When classified according to an mMRC score of 1, the numbers (%) of patients in GOLD groups A, B, C, and D were 41 (5.2%), 349 (44.2%), 28 (3.5%), and 372 (47.1%), respectively. When classified according to a CAT score of 10, these numbers were 113 (14.3%), 277 (35.1%), 64 (8.1%), and 336 (42.5%), respectively. When classified according to an mMRC score of 2, these numbers of patients were 266 (33.7%), 124 (15.7%), 156 (19.7%), and 244 (30.9%), respectively. When classified according to a CAT score of 15, these numbers of patients were 234 (29.6%), 156 (19.7%), 153 (19.4%), and 247 (31.3%), respectively. The κ value for concordance was highest between an mMRC score of 1 and a CAT score of 10 (0.66), followed by an mMRC score of 2 and a CAT score of 15 (0.56), an mMRC score of 2 and a CAT score of 10 (0.47), and an mMRC score of 1 and a CAT score of 15 (0.43) (Figure 4).

Figure 4.

Classification of patients with COPD according to four cutoff points.

Notes: A, less symptomatic and low-risk group. B, more symptomatic and low-risk group. C, less symptomatic and high-risk group. D, more symptomatic and high-risk group.

Abbreviations: κ, coefficient of agreement; CAT, COPD assessment test; mMRC, modified Medical Research Council.

When comparing the classification of patients between a CAT score of 10 and an mMRC score of 1, many cases were concordant. Most of the discrepant cases were upgraded by the mMRC score compared with the CAT score (to group A by the CAT score and to B by the mMRC score, or to group C by the CAT score and to D by the mMRC score). Ninety patients were classified into group A by a CAT score of 10 and group B by an mMRC score of 1, and 49 were classified into group C by a CAT score of 10 and group D by an mMRC score of 1. A relatively small number of patients were classified into group B by a CAT score of 10 and group A by an mMRC score of 1 (n=18), and group D by a CAT score of 10 and C by an mMRC score of 1 (n=13) (Figure 5).

Figure 5.

Discrepancy between a CAT score of 10 and an mMRC score of 1.

Notes: A, less symptomatic and low-risk group. B, more symptomatic and low-risk group. C, less symptomatic and high-risk group. D, more symptomatic and high-risk group.

Abbreviations: CAT, COPD assessment test; mMRC, modified Medical Research Council.

More cases were discrepant between a CAT score of 15 and an mMRC score of 2 than between a CAT score of 10 and an mMRC score of 1. Moreover, patients were both upgraded and downgraded by the mMRC score. Fifty-one patients were classified into group A by a CAT score of 15 and group B by an mMRC score of 2, and 58 were classified into group C by a CAT score of 15 and D by an mMRC score of 2. However, 83 patients were classified into group B by a CAT score of 15 and group A by an mMRC score of 2, and 61 patients were classified into group D by a CAT score of 15 and C by an mMRC score of 2 (Figure 6).

Figure 6.

Discrepancy between a CAT score of 15 and an mMRC score of 2.

Notes: A, less symptomatic and low-risk group. B, more symptomatic and low-risk group. C, less symptomatic and high-risk group. D, more symptomatic and high-risk group.

Abbreviations: CAT, COPD assessment test; mMRC, modified Medical Research Council.

Discussion

The new GOLD guidelines recommend the use of a CAT score of 10 or an mMRC score of 2 as the symptomatic cutoff point.¹ However, the characteristics of these two scores are different. The CAT score is calculated using an eight-item questionnaire and covers many aspects of quality of life. On the other hand, the mMRC score is determined from only one question regarding the degree of dyspnea. It does not address other important symptoms of COPD, such as coughing, sputum production, chest tightness, and depression. Thus, discrepancy between these two scores is expected.

One of the merits of mMRC is that it is easy to use. Thus, it has been widely used, especially in primary care clinics. Moreover, it is well correlated with disability¹¹ of COPD patients and is an even better predictor of survival than airway obstruction.¹² However, because of its simplicity, the correlation between mMRC and CAT is not strong. In this study, the correlation coefficient between the mMRC and CAT scores was only 0.49. Weak correlation between the two scores was also observed in the previous studies.^8,9,13 It is well known that COPD has multiple symptomatic effects.¹⁴ Thus, measuring only breathlessness is not sufficient to assess multiple symptoms in patients with COPD. In the present study, we found variation in the CAT score despite the same mMRC score. A histogram according to each mMRC clearly demonstrated heterogeneity of the CAT score with the same mMRC score (Figure 2). Whether these different patients can be classified into the same category simply because they have the same mMRC score should be questioned.

The other limitation of mMRC is that the cutoff point of high symptom is not validated. In the GOLD guidelines, the cutoff point of high symptom is mMRC ≥2.^1,6 However, among “less breathless” patients, some of them actually are symptomatic. Jones et al³ showed that even patients with an mMRC score of 0 had modestly elevated SGRQ-C scores and the 12-item Short-Form Health Survey score was 44.5, which was below 50 (normalized score expected of the general population). Also, for patients with an mMRC score of 1, significant levels of impairment in health status were observed. According to the study by Han et al,¹⁵ many patients with an mMRC score of 0 or 1 had an SGRQ-C score greater than 25.

Compared with mMRC, CAT score has several advantages. First, it is much more correlated with SGRQ. In this study, the correlation coefficient between CAT and SGRQ-C scores was 0.74 and a similar result had already been published in a previous study.¹⁶ Second, CAT score was validated in multiple languages. It was originally developed in English; however, the relationship between CAT and SGRQ was the same across four Asian countries.¹³ Third, the cutoff point of high symptom was validated. A CAT score ≥10 has been shown to have a significant impact on the daily lives of patients with COPD.⁷ Patients with a CAT score ≥10 are likely to be breathless on most days, get exhausted easily, and take a long time to do housework.^3,7

Some previous studies^3,8,9 showed that the classification of patients with COPD into GOLD groups A–D can change according to a CAT score of 10 or an mMRC score of 2. Rieger-Reyes et al⁹ showed that more than 25% of patients were reclassified into different categories according to a CAT score of 10 and an mMRC score of 2. Kim et al⁸ also showed that the numbers of patients in the highly symptomatic groups were markedly decreased by an mMRC score of 2 than by a CAT score of 10. Our study showed similar results. The numbers of patients in groups B and D according to a CAT score of 10 were 277 and 336, respectively. These numbers of patients decreased to 124 and 244, respectively, according to an mMRC score of 2. Accordingly, some patients who need long-acting bronchodilators may miss treatment when physicians apply an mMRC score of 2 rather than a CAT score of 10 in clinical practice.

Using an ROC curve, we showed that a CAT score of 15 is compatible with an mMRC score of 2 and that a CAT score of 10 is compatible with an mMRC score of 1. The AUC of a CAT score of 10 for an mMRC score of 1 was higher than that of a CAT score of 15 for an mMRC score of 2. Moreover, the κ between a CAT score of 10 and an mMRC score of 1 was higher than that of all other combinations. This finding is compatible with the previous study by Jones et al³ who found a higher degree of agreement between a CAT score of 10 and an mMRC score of 1 than between a CAT score of 10 and an mMRC score of 2. According to these findings, an mMRC score of 1 seems to be more appropriate than an mMRC score of 2 as the cutoff point for defining highly symptomatic patients.

In this study, we classified our cohort into GOLD groups A–D according to four cutoff points (mMRC score of 1, CAT score of 10, mMRC score of 2, and CAT score of 15). As expected, each cutoff point resulted in a different classification. Overall, the agreement between these classifications was poor. There were many discordant cases. We found that these four cutoff points created a spectrum in that the number of highly symptomatic patients decreased proportionally according to these four cutoff points (in the order of mMRC score of 1, CAT score of 10, CAT score of 15, and mMRC score of 2). Although an mMRC score of 1 and a CAT score of 10 showed the best agreement, there were still many discordant cases. Among the four cutoff points, a CAT score of 10 has been the most reliable point with which to differentiate normal and symptomatic patients.³ The meaning of obtaining a CAT score of 10 was well validated in a previous study.⁷ Thus, according to the present study, the use of a CAT score of 10 and an mMRC score of 2 together in the GOLD classification is not appropriate. An mMRC score of 1 appears to be more optimal than a score of 2; however, there were also many discordant cases even with an mMRC score of 1. Based on these results, we cautiously recommend that the GOLD committee consider discarding the use of the mMRC score as a cutoff point for symptomatic patients.

This study is noble in that we for the first time compared four different cutoff points at the same time. Two previous similar studies^8,9 showed discrepancy only between a CAT score of 10 and an mMRC score of 2. Although Jones et al³ showed discrepancy between three different points, there were no data regarding the ideal cutoff point for CAT that was equivalent to an mMRC score of 2. Moreover, when compared with Jones’ study, we have utilized statistical method (ROC curve) for finding equivalent CAT score. By doing so, we not only found equivalent CAT scores in each mMRC but also provided AUC, sensitivity, and specificity. Interestingly, we have obtained CAT scores of 10 and 15 for mMRC scores of 1 and 2, respectively. According to this result, we could clearly demonstrate that minimal discrepancy exists between a CAT score of 10 and an mMRC score of 1 compared with other combinations (we have provided four different κ values). We also showed that even between a CAT score of 10 and an mMRC score of 1, many patients were still reclassified, which may favor to discard mMRC in GOLD classification.

There were limitations to this study. This was a cross-sectional, not a longitudinal follow-up, study. Although we used data from a COPD cohort, we did not use follow-up data of exacerbations because this cohort is relatively new and the follow-up period was of insufficient duration. We used a simple history of previous exacerbations when classifying low- and high-risk patients. Thus, because of recall bias, some patients may have been misclassified as low-risk despite actually being high-risk. However, this is not directly related to the key results of this study. The main purpose of this study was to show discrepancies between the CAT and mMRC scores (high vs low symptom, not high vs low risk). A history of exacerbation did not substantially affect the main results of this study. Finally, in this study, we only addressed the level of agreement between CAT and mMRC. We did not compare these two scores against a gold standard (eg, survival).

Conclusion

A CAT score of 15 was the best cutoff point for an mMRC score of 2, and a CAT score of 10 was the best cutoff point for an mMRC score of 1. Among four different combinations of CAT and mMRC scores, a CAT score of 10 was most concordant with an mMRC score of 1 when classifying patients with COPD into GOLD groups A–D. However, a discrepancy remains between the CAT and mMRC scores. The results of the present study suggest that the GOLD committee needs to consider whether to lower the mMRC cutoff point to 1 or even to discard mMRC.