Validation of the COPD Assessment Test (CAT) as an Outcome Measure in Bronchiectasis

Abstract

Background

Objective assessment of symptoms in bronchiectasis is important for research and in clinical practice. The COPD Assessment Test (CAT) is a short, simple assessment tool widely used in COPD. The items included in the CAT are not specific to COPD and also reflect the dominant symptoms of bronchiectasis. We therefore performed a study to validate the CAT as an outcome measure in bronchiectasis.

Methods

The CAT was administered to two cohorts of bronchiectasis patients along with other quality of life questionnaires. Patients underwent comprehensive clinical assessment. One cohort had repeated questionnaires collected before-and-after treatment of acute exacerbations. We analyzed convergent validity, repeatability, and responsiveness of the score and calculated the minimum clinically important difference (MCID) using a combination of distribution and anchor-based methods.

Results

In both cohorts there were positive correlations between the CAT and the St. George’s Respiratory Questionnaire (r = 0.90, P < .0001 and r = 0.87, P < .0001). There was an inverse relationship between CAT and Quality of Life – Bronchiectasis Respiratory Symptoms Scale (r = −0.75, P < .0001) and Leicester Cough Questionnaire score (r = −0.77, P < .0001). Patients with more severe disease, based on the bronchiectasis severity index, had significantly higher CAT scores. CAT also correlated with FEV₁ % predicted and 6-min walk distance (6MWD). CAT increased significantly at exacerbation and fell at recovery. The intraclass correlation coefficient for two measurements four-weeks apart while clinically stable was 0.88 (95% CI, 0.73-0.95, P < .0001). An MCID of 4 was most consistent.

Conclusions

CAT is a valid, responsive symptom assessment tool in bronchiectasis. The MCID is estimated as 4 points.

FOR EDITORIAL COMMENT, SEE PAGE 749

Bronchiectasis is a condition which has a significant long-term impact on quality of life (QOL).^1,2 Patients experience daily cough, sputum, fatigue, chest discomfort, rhinosinusitis, and breathlessness along with frequent exacerbations in many cases.^{3, 4, 5} QOL is also impaired by social, psychological, physical, and treatment-related factors such as the burden of treatment from daily chest physiotherapy and medications including oral and nebulized drugs.^{2,3,6, 7, 8}

QOL and symptom assessments are key measurable outcomes in bronchiectasis management. They are among the most important clinical trial end points; therefore, having valid tools to assess QOL is essential for both research and daily clinical practice.^2,9,10 Several different tools have been applied to studying bronchiectasis, including those originally developed for other respiratory diseases such as the St. George’s Respiratory Questionnaire (SGRQ) and those developed specifically for bronchiectasis such as the Quality of Life – Bronchiectasis questionnaire (QOL-B).^2,5,9,11

The use of specific tools is attractive to capture the variety of features that are unique to a certain condition and elucidate the individual patient factors that may require specific attention. There is, however, a high degree of overlap among COPD, bronchiectasis, and asthma, with up to 50% of patients with COPD being reported to have bronchiectasis and up to 50% of bronchiectasis patients reporting a history of asthma.^{12, 13, 14} Disease labels are increasingly being abandoned in favor of a treatable traits concept that acknowledges the heterogeneity of airways disease.^{15, 16, 17} The high degree of similarity in the symptoms of the three major airways diseases may explain why the SGRQ, despite not being designed for use in bronchiectasis, has been shown to be consistently associated with bronchiectasis disease severity measures, and to be responsive to treatments including inhaled antibiotics.^{11,18, 19, 20, 21} In the Ciprofloxacin Dry Powder for Inhalation in Non-cystic Fibrosis Bronchiectasis (Non-CF BE) (RESPIRE) program, treatment with inhaled dry powder ciprofloxacin resulted in a significant improvement in the SGRQ in RESPIRE 1 (adjusted difference −7.59, P = .009 and −5.21, P = .06 in the 14-day on/off and 28-day on/off arms, with a minimum clinically important difference [MCID] of 4), whereas the disease-specific QOL-B questionnaire failed to demonstrate responsiveness (adjusted difference 2.47, P = .3 and 1.18, P = .6 with an MCID of 8).²⁰

There is therefore a strong rationale for considering using validated symptom assessment tools across diseases. The COPD Assessment Tool (CAT) is a short, eight-question, patient-administered questionnaire that was developed for use in COPD. Score ranges from 0 to 40, with higher scores indicating more severe symptoms. It has been shown to be comparable to the SGRQ in COPD.^{22, 23, 24} Symptoms covered are cough, sputum production, chest tightness, exertional dyspnea, activities of daily living, confidence, sleep, and energy, all of which are also key components of disease-specific bronchiectasis tools. The simplicity of the CAT as well as its established performance characteristics in COPD makes it an attractive potential tool for bronchiectasis patients. It is currently being evaluated in several studies because it has been recognized to have validity for other chronic airways diseases; in this context, it has been renamed as the Chronic Airways Assessment Test (https://clinicaltrials.gov/ct2/show/NCT02760329). Pilot studies suggest that the CAT correlates with clinically important outcomes in bronchiectasis.²⁵

This study was therefore designed to validate the CAT questionnaire for use in bronchiectasis and to determine the minimum clinically important difference.

Methods

We performed a prospective study designed to evaluate the convergent validity, responsiveness, and clinical utility of the CAT in patients with bronchiectasis. The study was approved by the local research ethics committee (13/ES/0062), and all patients gave written informed consent to participate.

The CAT was evaluated in two distinct studies, the Tayside Rehabilitation in Bronchiectasis Exacerbations (TRIBE) randomized trial, which was a longitudinal evaluation of the CAT, and a cross-sectional validation cohort in which the CAT was performed at a single time point. These are referred to as the TRIBE cohort and the validation cohort throughout the manuscript.

TRIBE Cohort

Details of the TRIBE trial have been previously published.²⁶ Patients were enrolled from 2014 through 2017. The CAT was a secondary end point in the TRIBE study; validation of the CAT questionnaire was a prespecified substudy. Patients were enrolled in the study if they had high-resolution CT confirmed bronchiectasis and at least one exacerbation in the previous year. Patients were excluded if they were aged <18 years, had a diagnosis of cystic fibrosis, or an exacerbation in the previous 4 weeks. Patients completed the CAT questionnaire at screening as well as undergoing a clinical evaluation including lung function and 6-min walk test according to standard guidelines.²⁶ Baseline data were used to confirm convergent validity of the CAT in a second cohort of patients. Importantly, the TRIBE study specifically excluded patients with any history of COPD (defined as a history of at least 10 pack-years cigarette smoking and an FEV₁/FVC ratio <0.7 along with a clinical diagnosis of COPD). Patients also completed the SGRQ and Leicester Cough Questionnaire (LCQ) at each visit. Of note, this study was initiated before publication of the QOL-B questionnaire; therefore, data on this questionnaire were not available for comparison.

Patients who met eligibility criteria for the TRIBE study were then asked to contact the site when they developed symptoms of an acute exacerbation. Detection of exacerbations was supported by daily diaries. Exacerbations were defined as an increase in respiratory symptoms requiring antibiotic treatment as determined by a clinician. Patients attending for an exacerbation visit then completed the CAT again, followed by a further visit two weeks later after completion of 14 days’ treatment with antibiotics for an acute exacerbation.²⁶

Patients were subsequently randomized to pulmonary rehabilitation or standard care. The CAT was repeated at week 8 and week 12 following the exacerbation (after completion of pulmonary rehabilitation and at the end of the study, respectively).

Cross-Sectional Validation Cohort

In this validation analysis, 83 patients were prospectively enrolled from a specialist tertiary referral center in the UK over a 12-month period. None of the included patients overlapped with those included in TRIBE. Patients were required to be clinically stable for 4 weeks before enrollment and have clinically significant bronchiectasis and confirmation of the diagnosis on a high-resolution CT scan. Patients were excluded if they had a primary diagnosis of COPD, asthma, cystic fibrosis, or other respiratory condition. Patients were evaluated according to British Thoracic Society recommendations, including a comprehensive workup for potential underlying causes.²⁷ The QOL-B and SGRQ were administered alongside the CAT for comparison. This study was cross-sectional with no repeated evaluation of the CAT questionnaire.

Convergent Validity

This represents an assessment of the instrument against other measures that are considered to represent severity of disease because a valid instrument should agree with clinical assessments of severity of disease and disease burden.⁹ The CAT questionnaire was tested for its correlation with other validated questionnaires (QOL-B, SGRQ, LCQ). For convergent validity assessment, the CAT was correlated with these questionnaires, but also with recognized measures of bronchiectasis severity including the bronchiectasis severity index (BSI), exacerbation frequency, FEV₁, and self-reported daily sputum volume.²⁸ In the TRIBE study, CAT was also correlated with the 6MWD. All assessments were performed on the same day as administration of the CAT.

Repeatability

Patients completed the CAT during two visits 1 month apart, if they reported stable symptoms, to determine the repeatability of the measure. Patients were excluded if they reported a change in symptoms or an exacerbation during this 1-month period.

MCID

The MCID can be calculated through distribution-based or anchor-based methods, and there is no agreed optimal method for MCID estimation.²³ For this study, we calculated both distribution-based methods using one-half the baseline SD of the measure and an anchor-based method using three clinically relevant anchors: the mean change in CAT at the onset of exacerbation (a clinically meaningful negative change in patient symptoms), the change from exacerbation to recovery from exacerbation (a clinically meaningful change in patients symptoms in the positive direction), and the change during the course of the TRIBE study anchored to the SGRQ. It is acknowledged that there is no finalized MCID for any QOL tool in bronchiectasis, apart from a MCID of 4 for the SGRQ that has been extensively used in bronchiectasis and so was selected for this study.¹¹

Statistical Analysis

SPSS, version 22.0 (SPSS), and Prism, version 6 (GraphPad), were used for analysis. We present mean with SD for parametric distributions or median with interquartile range for nonparametric distributions as appropriate. Comparisons across more than 2 groups were performed using analysis of variance. Correlations between variables were assessed with linear regression, Pearson r, and Spearman P as appropriate. Repeatability was evaluated using the intraclass correlation coefficient and a Bland-Altman plot. P < .05 was considered statistically significant.

Results

Cohort Description

TRIBE Cohort

Forty-eight patients were enrolled and completed a CAT questionnaire at each visit. The mean age was 67 years (7.5) and there were 31 women (64.6%). The mean BSI score was 6.6 (3.2) and mean FEV₁ % predicted was 78.8% (26.6). Baseline CAT score ranged from 4 to 37. Twenty-four of the 48 patients enrolled had an exacerbation during the 12-month follow-up period and provided additional data at onset and recovery from exacerbation. Characteristics of this patient population are shown in e-Table 1 online.

Cross-Sectional Validation Cohort

Eighty-three patients were included and 80.7% were classified as idiopathic. The mean age was 71 years (9.5) and 45 (54.2%) were women. In keeping with the tertiary referral nature of this population, the patients had more severe disease than the TRIBE cohort, with a mean FEV₁ % predicted of 52% (13.2). The mean exacerbation frequency was two per year (interquartile range, 0-3) and 56.6% were classified as severe using the BSI (e-Table 2). Haemophilus influenzae was the most frequent organism found in 33.7% of the cohort with Pseudomonas aeruginosa found in 18.1%.

Convergent Validity: SGRQ

Both cohorts completed CAT and SGRQ at a clinically stable baseline. The mean CAT score for the TRIBE cohort was 19.3 (7.8) and mean SGRQ was 42.0 (19.7). There was a strong correlation between CAT and SGRQ in TRIBE (r = 0.90, P < .0001) (Fig 1). The mean CAT score for the validation cohort was 21.2 (7.8) and the mean SGRQ score was 52.7 (20.4) (r = 0.87, P < .0001). The CAT score also correlated well with each of the domains within SGRQ: SGRQ symptoms r = 0.68, P < .0001, SGRQ activity r = 0.84, P < .0001, SGRQ impacts (psychosocial) r = 0.83, P < .0001 (e-Fig 1).

Figure 1.

Comparison of CAT score and SGRQ total score in the TRIBE and validation cohorts. CAT = COPD Assessment Test; SGRQ = St. George’s Respiratory Questionnaire; TRIBE = Tayside Rehabilitation in Bronchiectasis Exacerbations.

Convergent Validity: QOL-B and LCQ

QOL-B data were only available for validation cohort and LCQ data were only available in the TRIBE cohort. There was a clear inverse relationship between CAT and QOL-B Respiratory Symptoms Scale (r = −0.75, P < .0001) and LCQ total score (r = −0.77, P < .0001), noting that lower scores on both scales indicate worse symptoms (Fig 2). The CAT was also associated with the individual components of the LCQ score (e-Fig 2).

Figure 2.

Comparison of CAT score and QOL-B in validation cohort and the CAT and LCQ in the TRIBE cohort. CAT = COPD Assessment Test; LCQ = Leicester Cough Questionnaire; QOL-B = Quality of Life – Bronchiectasis; RSS = respiratory symptom score; TRIBE = Tayside Rehabilitation in Bronchiectasis Exacerbations.

Convergent Validity: Other Bronchiectasis Severity Markers

CAT scores were compared with clinical assessments used to assess bronchiectasis severity. BSI score and FEV₁ % predicted were available for both cohorts, 6MWD was available for TRIBE cohort only. Exacerbation frequency was available in both cohorts, but the TRIBE cohort was not evaluated because it was recruited on the basis of exacerbation history at baseline. The mean 6MWD in TRIBE was 420 m. There was a clear relationship between 6MWD and CAT score (r = 0.58, P < .0001) (Fig 3). Patients with more frequent exacerbations in the validation cohort had higher CAT score (P = .0054 comparing across groups using analysis of variance). Mean BSI in TRIBE cohort was 6.6 (3.2), and there was a significant correlation between BSI and CAT (r = 0.34, P = .017). The mean BSI in the validation cohort was 9.4 (4.1) with a significant relationship also evident in this cohort by linear regression (r = 0.63, P < .001). A weak relationship between CAT and FEV₁ % predicted was observed in the TRIBE cohort (r = −0.34, P = .02), which was not replicated in the validation cohort (r = −0.20, P = .3).

Figure 3.

Convergent validity of the CAT score with BSI, 6-min walk distance; and exacerbation frequency. 6MWT = 6-min walk test; BSI = bronchiectasis severity index; CAT = COPD Assessment Test.

Change in CAT at Acute Exacerbation and After Treatment

Data were available for 24 patients experiencing exacerbations during TRIBE study. The CAT was completed at start of treatment and following a 2-week course of antibiotics. The mean change in CAT from stable baseline was 3.57 (95% CI, 0.75-6.4; P = .01) at the onset of an exacerbation, indicating a statistically significant increase in CAT score. Interestingly, some patients showed no change or minimal change in the CAT score at exacerbation. A statistically significant change was also observed following antibiotic treatment with a mean change from exacerbation onset to completion of treatment of −4.83 (95% CI, −1.5 to −6.5; P = .003). Figure 4A shows the dynamics of CAT scores in individual subjects at the onset of exacerbation and following antibiotic treatment. Figure 4B shows the mean and SD of the group changes.

Figure 4.

Change in CAT score at the onset and then recovery from exacerbation. A, Change over time from stable state to exacerbation and then posttreatment (14 days after antibiotic treatment). B, Mean and SD differences between stable state and exacerbation and then exacerbation onset and recovery, representing clinically meaningful changes in patient status. CAT = COPD Assessment Test.

Repeatability and Calculation of Minimum Clinically Important Difference

Test-retest repeatability was only evaluated in the TRIBE cohort in the same 24 patients described previously. The intraclass correlation coefficient for two measurements of the CAT score in individuals 4 weeks apart without changes in clinical status was 0.88 (95% CI, 0.73-0.95; P < .0001) indicating a high degree of repeatability and reliability. The Bland-Altman plot is shown in e-Figure 3.

During recovery from exacerbation over an 8-week period, patients experienced improvements in the CAT, SGRQ, and LCQ in the TRIBE study. Because no difference was observed between those patients randomized to pulmonary rehabilitation or standard care in the original trial, the data were pooled for calculation of the MCID.

The change in CAT correlated with a change in SGRQ (r = 0.68, P = .0004) and the LCQ (r = −0.57, P = .004). For calculation of the MCID, we used a one-half SD as a distribution-based method and the multiple anchor-based methods. The distribution-based methods suggested an MCID of 3 to 4. The anchor-based methods similarly suggested an MCID between 3 and 4 (Table 1). Based on these data, the most reliable MCID was proposed to be 4 points.

Table 1.

Minimum Clinically Important Differences of the CAT in Bronchiectasis

Definition	Result (SD or Mean Change in CAT)	Proposed MCID
Distribution based
½ SD TRIBE cohort	3.89	4
½ SD validation cohort	3.91	4
Anchor based
Exacerbation onset	3.57	4
Exacerbation recovery	-4.83	5
4-point change in SGRQ as anchor	3.43	3
1.3-point change in LCQ as anchor	3.78	4

CAT = COPD Assessment Test; LCQ = Leicester Cough Questionnaire; MCID = minimum clinically important difference; SGRQ = St. George’s Respiratory Questionnaire.

Discussion

Based on these data, we have shown that the CAT is a valid tool to measure symptoms and treatment responses in patients with bronchiectasis. This tool is simple, easy to administer, and consists of only eight items, allowing patients to complete it in a few minutes.^22,23 The CAT measures the severity of respiratory symptoms that are common to all airways diseases including bronchiectasis and COPD. The name Chronic Airways Assessment Test rather than COPD Assessment Test may be more appropriate in view of its broader applicability to several respiratory conditions. The CAT has been shown to appropriately indicate symptoms similarly during pulmonary rehabilitation in both patients with and without COPD in a prospective study of 365 patients in the United Kingdom, whereas the CAT has also been found to have prognostic value in interstitial lung disease.^29,30

Our study builds on prior studies that have evaluated different aspects of the CAT in bronchiectasis patients.^25,31 Lanza et al²⁵ investigated 100 patients from Brazil in a cross-sectional study and found strong relationships between CAT and disease severity, SGRQ, and exercise capacity. Brill et al³¹ studied 22 patients with bronchiectasis and found a significant increase in CAT scores as part of a study to evaluate the dynamics of symptoms around exacerbations. Neither study was specifically designed to validate the CAT using assessment of convergent validity, responsiveness, repeatability, nor calculation of the minimum clinically important difference.

In our study, the CAT score consistently correlated to the multiple questionnaires including the SGRQ, QOL-B, and LCQ. The strength of this correlation suggests that all are measuring similar aspects of the disease with the advantage of the CAT being its greater simplicity and ease of administration. The CAT also correlated well to exacerbation frequency in the cross-sectional validation cohort as well as lung function and 6MWD. Overall, this suggests that the CAT test is a valid tool and provides an immediate assessment of the severity of disease and the degree of disability.

The benefits of using a questionnaire such as the CAT are that it is simpler and faster to administer and can easily be performed in the outpatient setting during consultations or in the waiting room. The questions are clear and easy for patients to understand. Its design makes it more likely to be accepted by patients than the more complex questionnaires with multiple sections.^23,24 The CAT is also available as an online tool that patients can perform independently and has been validated in 90 languages for COPD.

We have demonstrated that the CAT questionnaire indicates a worsening of symptoms at the onset of an exacerbation and improvement following recovery from an exacerbation. Changes in the CAT correlate to changes in the SGRQ and LCQ, all of which suggest that the CAT should be responsive to interventions that have a beneficial effect on symptoms. We were interested to observe that the CAT score did not always increase from the baseline value to the onset of exacerbation. We observed that patients’ symptoms fluctuated over time and this was also observed in the repeatability analysis in which most patients CAT scores were stable but some showed up to a 10-point change due to day-to-day variability in the absence of an exacerbation. A subject could therefore potentially, for example, have a CAT score of 10 at baseline, 2 at a subsequent visit, and then a score of 11 at exacerbation. The change from baseline would be minimal but the change from their other more recent symptoms might be large. Variability in day-to-day symptoms is a phenomenon that has been observed in COPD and other respiratory diseases and is likely to be identified in bronchiectasis. Studies using electronic or other diaries may be more sensitive and useful to evaluate the dynamics of symptom changes around exacerbation.

Identifying better ways of capturing symptomatic treatment benefits is a key research priority in bronchiectasis at present.³² Multiple clinical trials assessing different medications have failed to demonstrate consistent symptom benefits.³³ Possible explanations for this include that inhaled antibiotics are not effective at reducing symptoms or that the current symptom tools are poorly adapted to measuring treatment responses in bronchiectasis patients. In the recent inhaled antibiotic studies, the disease-specific QOL-B tool failed to change in response to liposomal ciprofloxacin treatment in the Study With Ciprofloxacin Dispersion for Inhalation in Non-CF Bronchiectasis (ORBIT) studies despite an exacerbation benefit in the pooled analysis.³³ The SGRQ responded in RESPIRE 1, particularly in the 14-day on/off arm, but the QOL-B showed no similar benefits. Likewise, the QOL-B did not show clear benefits in the AIR-BX studies of aztreonam, although we have recently postulated that this may have been due to inclusion of patients with low bacterial load.³⁴ The SGRQ has shown a degree of responsiveness in studies of macrolides and mannitol.³⁵ Therefore, to date, the SGRQ has been the most responsive tool in this disease, but is limited by complexity. The CAT is therefore attractive because of its close correlation with the SGRQ. Prospective testing of the CAT in clinical trials is, however, needed.

We have proposed a minimum clinically important difference of 4 points based on the changes observed in this study. There is no single accepted method of determining the MCID; therefore, we used multiple methods. The methods used suggested an MCID between 3 and 5 would be considered appropriate. The MCID proposed for COPD is 2 points.²³ In their study evaluating the CAT in >700 patients with COPD across two cohorts, Kon et al²³ found distribution-based analysis suggested an MCID of 3 to 4 points, but the linear regression suggested MCIDs through correlation with the SGRQ score of 2 or 3 points and selected 2 points based on receiver operator characteristic curve analysis. The findings of their analysis are therefore very similar to ours even if the conclusion regarding the MCID is modestly different. The different relationship between CAT and SGRQ in the two studies may represent genuine differences in treatment response between bronchiectasis and COPD or our smaller sample size. Our repeatability analysis in particular was limited by a small sample. Our findings with regard to MCID should be considered preliminary because future studies with larger numbers of patients testing different clinical interventions may identify different patterns of response. Patients with bronchiectasis are heterogeneous and so validation in different patient cohorts would be valuable. In parallel with our study, another validation study of the CAT in bronchiectasis has recently been conducted in Spain. This study by De La Rosa Carrillo³⁶ found that the CAT had excellent internal consistency and repeatability and correlated well with other questionnaires, including the bronchiectasis health questionnaire and SGRQ and QOL-B. The authors proposed an MCID of 3 points based on two measures of distribution of the change in CAT score around exacerbation. Our study included 131 patients in total from the United Kingdom, whereas the De La Rosa Carrillo study included 96 patients from Spain. The two studies used a different design and different methods of analysis and therefore provide complementary information on the utility of the CAT in bronchiectasis.³⁶

Our study is limited by the questionnaires being administered in English and only with patients in the United Kingdom. Nevertheless, the characteristics of the patients are broadly representative of those in larger bronchiectasis patient populations across Europe. We included two patient cohorts in our study with the objective of providing a higher degree of confidence in our findings through cross-validation. Bronchiectasis is a rapidly changing field and this is reflected in our data, in which the QOL-B was only available in one study cohort because it was not developed or validated when the TRIBE study was initiated. Neither study evaluated the Bronchiectasis Health Questionnaire, a shorter disease specific QOL tool similar in design to the CAT, which is awaiting further validation.^2,9

Future studies could focus on assessing the utility of the CAT in a larger bronchiectasis population across multiple centers with a longer period of follow-up, and incorporation into clinical trials to assess how it responds to therapy and correlates to longer-term morbidity, mortality, and disease progression.

Conclusion

This study has validated the CAT questionnaire for use in patients with bronchiectasis. We suggest an MCID of 4 points when used for bronchiectasis. We demonstrate that the CAT is a potentially useful tool for assessing symptoms and QOL in patients with bronchiectasis in clinical practice and in future clinical trials.