Development and Validation of a Short Version of the Kansas City Cardiomyopathy Questionnaire

Supplemental Digital Content is available in the text.

Abstract

Background—

There is a growing demand to collect patients’ experiences of their health status (their symptoms, function, and quality of life) in clinical trials, quality assessment initiatives, and in routine clinical care. In heart failure, the 23-item, disease-specific Kansas City Cardiomyopathy Questionnaire (KCCQ) has been shown to be valid, reliable, sensitive to clinical change, and prognostic of both clinical events and costs. However, its use has been limited, in part, by its length. We sought to develop a shortened version of the instrument that maintains the psychometric properties of the full KCCQ.

Methods and Results—

Using data from 3 clinical studies incorporating 4168 patients, we derived and validated a 12-item KCCQ, the KCCQ-12, to capture symptom frequency, physical and social limitations, and quality of life impairment as a result of heart failure, as well as an overall summary score. The KCCQ-12 scores had high correlations with the original scales (>0.93 for all scales in all clinical settings), high test–retest reliability (>0.76 for all domains), high responsiveness (16–31 point improvements after discharge from hospitalization; standardized response mean =0.61–1.12), and comparable prognostic significance and interpretation of clinically important differences as compared with the full KCCQ.

Conclusions—

The KCCQ-12 is a shorter version of the original 23-item instrument that should be more feasible to implement while preserving the psychometric properties of the full instrument.

WHAT IS KNOWN

• The Kansas City Cardiomyopathy Questionnaire (KCCQ) is an international standard, based on its psychometric properties, for quantifying the disease-specific health status of patients with heart failure.

• At 23 items, the KCCQ may be too long for some applications, such as routine clinical care.

• A shorter version that could preserve its psychometric properties would improve the feasibility of the instrument.

WHAT THE STUDY ADDS

• Using data from 4167 patients with heart failure in different clinical settings, the KCCQ was reduced from 23 to 12 items.

• The KCCQ-12 is highly correlated with the original 23-item scale scores and preserves the validity, reliability, responsiveness, prognostic importance, and interpretability of the original instrument.

• The KCCQ-12 may prove to be a more feasible instrument for quantifying the health status of heart failure patients.

Quantifying patients’ perspectives about the degree to which their heart failure (HF) impacts their health status (their symptoms, function, and quality of life) is becoming an increasingly important outcome in clinical trials, quality assessment, and clinical care.¹ Accordingly, over the past 3 decades, several disease-specific patient-reported outcomes (PROs) for HF have been created.^2–6 Transitioning PROs from outcomes in clinical trials, where the studies pay for additional data collection, to routine clinical care requires that the measures be short and feasible to collect, while also retaining the important psychometric properties of validity, reliability, sensitivity to clinical change, prognostic importance, and interpretability.^7,8 The Kansas City Cardiomyopathy Questionnaire (KCCQ), a commonly used instrument for measuring health status in patients with HF, has excellent psychometric properties,^2,9–13 but is currently 23 items long, taking 5 to 8 minutes for patients to complete. Creating shorter and simpler health status measures is a critical step in supporting their use in clinical care^14,15 and can support inexpensive, serial monitoring of patients’ HF that might help identify those who warrant additional testing with biomarkers^16,17 or physiological measures^18–20 or treatment intensification.

Editorial see p 460

Given the importance of being able to accurately and objectively assess patients’ health status and prognosis using a low-cost, noninvasive strategy, we sought to develop a shorter version of the KCCQ that preserves the psychometric and prognostic properties of the original instrument. Creating a shortened KCCQ meets an important gap in current assessment methods of patients’ health status by enabling a more feasible strategy for serially collecting, quantifying, and monitoring the health of HF patients. Accordingly, using KCCQ data from several existing studies of HF patients that have previously demonstrated the psychometric properties of the instrument, we developed and validated a shortened 12-item KCCQ (KCCQ-12). This report describes the development and validation of the KCCQ-12, including its psychometric properties (validity, reproducibility, and responsiveness) and interpretability.

Methods

The Kansas City Cardiomyopathy Questionnaire

The 23-item KCCQ quantifies 7 domains of patients’ HF-related health status: Physical Limitation (6 items), Symptom Stability (1 item), Symptom Frequency (4 items), Symptom Burden (3 items), Self-Efficacy (2 items), Quality of Life (3 items), and Social Limitations (4 items). Item responses are coded sequentially (1, 2, 3, etc.) from worst to best status. Scores are generated for each domain and scaled from 0 to 100, with 0 denoting the worst and 100 the best possible status. In addition, several summary scores are calculated: a Total Symptom score (average of Symptom Frequency and Symptom Burden), a Clinical Summary score (average of Physical Limitation and Total Symptoms), and an Overall Summary score (average of Physical Limitation, Total Symptoms, Quality of Life, and Social Limitation).² The KCCQ has been shown to be valid, reproducible, and sensitive to clinical change in patients with systolic dysfunction, HF with preserved ejection fraction, and valvular heart disease.^2,13,21,22 Moreover, patients’ KCCQ scores are independently prognostic of survival, HF admissions, and costs.^9–12

Data Sources

To create a shorter KCCQ, we used data from 2 randomized trials and one observational cohort study of HF patients, all of which have been previously used to demonstrate some of the psychometric properties of the full instrument: (1) the Efficacy of Vasopressin Antagonism in Heart Failure Outcome Study With Tolvaptan (EVEREST) study, a randomized trial of patients hospitalized with HF;²³ (2) the Eplerenone Post-AMI Heart Failure Efficacy and Survival (EPHESUS) study, a randomized trial of patients with HF complicating acute myocardial infarction;²⁴ and (3) the KCCQ Interpretability Study (KCCQINT), a 14-center North American cohort study of patients presenting at outpatient HF clinics.¹³ Descriptions of the studies and KCCQ samples used in the various analyses are provided in Figure. Using these studies, we derived and validated the short KCCQ within 3 distinct clinical settings: (1) stable HF, (2) outpatient HF clinic visits, and (3) acute HF recovery (1 week after hospitalization for decompensated HF). Within each setting, data were split randomly into 2 50% samples, one for derivation (ie, item selection) and one for validation of the final short-version scores. All studies underwent IRB approval before their conduct, and each patient signed informed consent to participate.

Figure.

Data sources and study samples. EPHESUS indicates Eplerenone Post-Acute Myocardial Infarction Heart Failure Efficacy and Survival; EVEREST, Efficacy of Vasopressin Antagonism in Heart Failure Outcome Study With Tolvaptan; and KCCQ, Kansas City Cardiomyopathy Questionnaire.

Derivation of the Short KCCQ

In deriving the short KCCQ, we first restricted consideration to domains that directly measure patients’ current health status: physical limitation, symptom frequency, quality of life, and social limitation. We did not include the single-item symptom change scale because this could not be reduced further, is not incorporated into any of the summary scores, and could always be added to the short version of the KCCQ in clinical settings where a more responsive assessment of recent changes in patients’ symptoms was desired. We excluded the self-efficacy scale because it measures a distinctly different concept than the ways in which HF impacts patients’ health status. We also excluded the 3 Symptom Burden items, which ask patients how bothersome their symptoms (edema, fatigue, dyspnea) are because they were each highly correlated with their corresponding Symptom Frequency item (r=0.78–0.87) and had less response variability.

For the Symptom Frequency domain, we chose to retain all 4 items to fully represent the spectrum of HF symptoms. For each of the 3 remaining domains, we sought to select items that would maximize comparability between the short and full versions of their domain score. To accomplish this, we examined multiple versions constructed by selecting different possible subsets of items. For the physical limitation domain, which covers low, moderate, and high intensity activities (2 items each), we sought to preserve the range of activities represented by selecting one item from each level of exertional demand, resulting in 8 possible subsets. For the quality of life and social limitation domains, we considered all possible subsets of items (omitting the “intimate relationships” item from the latter domain as a result of high nonresponse rates), yielding 6 and 7 subsets, respectively. For each subset of items, we then calculated a short-version domain score following the KCCQ scoring methodology and examined how closely the short score tracked with the corresponding full-version score. We used Lin’s concordance correlation coefficient, which measures the agreement between 2 variables; concordance values range from −1 (perfect negative agreement) to 1 (perfect positive agreement), with 0 denoting no agreement.²⁵ Subsets whose scores had the highest concordance with the full-version score were preferred. Analyses were conducted for each of the 3 clinical settings described above (stable HF, outpatient HF clinic visit, acute HF recovery). Item response variability, nonresponse rates, and clinical judgment were also considered in selecting items.

Once the final set of items was identified, scores for each of the 4 domains were calculated using methodology analogous to that of the full KCCQ, so that scores ranged from 0 to 100 for each domain. In addition, an overall summary KCCQ score was derived as the average of the 4 domain scores, as in the full KCCQ.

Validation

Across the 3 clinical settings, we conducted a series of analyses in the independent, validation samples to evaluate construct validity, predictive validity, reproducibility, and responsiveness and to calculate minimal clinically important differences for the short KCCQ scores. Parallel analyses were conducted for the full KCCQ for comparison. Table 1 summarizes the specific analyses performed and the cohorts used for each analysis.

Table 1.

Analyses, Settings, and Cohorts Used for Deriving and Validating the KCCQ-12

Construct Validity

To evaluate construct validity, we first compared each short KCCQ domain scores with their respective score from the full KCCQ. Means and standard deviations of scores, as well as mean and standard deviation of differences and concordance coefficients, as described earlier, were calculated and reported. In addition, we calculated mean KCCQ Overall Summary scores by New York Heart Association class I to IV and estimated the association using Kendall’s τ-b rank correlation coefficient.

Predictive Validity

Predictive validity refers to the association of scores with subsequent clinical outcomes. For these analyses, the KCCQ scores were used to predict the outcome of 6-month death or cardiovascular hospitalization among stable HF patients (EPHESUS Month 6 data) and among patients recovering from acute HF hospitalization (EVEREST Week 1 data). Cumulative 6-month incidence was calculated using Kaplan–Meier methods within predefined score categories of 0–<25 (poor), 25–<50 (fair), 50–<75 (good), and 75 to 100 (excellent), as used in previous studies.^9–12 Discrimination was assessed by c-statistics.²⁶

Reproducibility

Reproducibility, or test–retest reliability, assesses the change in scores for patients whose clinical status has not changed. We assessed the reproducibility of the short KCCQ by comparing baseline and 6-week change among HF clinic patients (KCCQINT) who had no clinical change based on both patient and physician global health assessments and who had no intervening clinical events.¹³ We calculated the mean and standard deviation of 6-week change scores as well as intraclass correlations. The intraclass correlation denotes the proportion of variability in scores because of between-patient (versus within-patient) differences; intraclass correlations >0.4, 0.6, and 0.8 indicate moderate, substantial, and excellent reproducibility.²⁷

Responsiveness

Responsiveness is the converse of reproducibility and assesses the sensitivity of the measure to clinical change.^28,29 The responsiveness of the short KCCQ to clinical change was quantified by the change from baseline to 1 week after hospitalization for acute HF (EVEREST). We calculated the mean and standard deviation of change as well as the standardized response mean, defined as the mean change divided by the standard deviation of change. Standardized response means above 0.5 and 0.8 indicate moderate and strong responsiveness, respectively.²⁷

Minimal Clinically Important Difference

A critically important issue in the use of health status measures is to define what magnitude of change is clinically important.^30,31 In the KCCQ Interpretability Study, physicians were asked to complete a global assessment of patients’ clinical change from baseline, using a 15-point scale ranging from −7 (“a great deal more limited”) to +7 (“a great deal less limited”) at the follow-up visit.^13,31 In fact, this study was designed explicitly to determine the average change in KCCQ scores across different magnitudes of clinical change, and the clinicians were focused on carefully assessing, blinded to patients’ KCCQ scores, the changes in patients’ HF status. We used these assessments to determine the minimal clinically important difference for the short KCCQ Summary Score. ROC analyses were conducted predicting any significant improvement (global assessment rating ≥2) and any significant deterioration (≤-2) on the basis of the 6-week change in KCCQ scores. For each end point, the optimal KCCQ cut point was chosen at the point maximizing Youden’s Index, which weights sensitivity and specificity equally.³² Cut point confidence intervals were derived using bootstrap methods. Given prior reports demonstrating that mean group differences in KCCQ scores ≥5 points are clinically important,¹³ we also report the sensitivity and specificity of intraindividual changes in patients’ scores of these magnitude being clinically important. These analyses were conducted on the entire KCCQ Interpretability Study population to fully use all available data.

Results

Across the 3 studies represented in these analyses, KCCQ data were available on 4168 patients. Descriptions of the studies and the psychometric assessments performed within each study are listed in Table 1.

Derivation

Item selection was conducted within independent derivation samples (50% of the available data, selected randomly) representing each of the 3 clinical settings: stable HF recovering from an acute myocardial infarction (EPHESUS 6-month assessment, N=673), HF outpatient clinic visit (KCCQ Interpretability Study baseline assessment, N=273), and recovery from acute HF hospitalization (EVEREST Week 1 assessment, N=853). Item response means, standard deviations, and missing rates within each of these settings are outlined in Table 2. In general, stable HF patients had few limitations, minimal symptoms, and good quality of life; in comparison, HF clinic patients had slightly more symptoms and worse quality of life, and patients recovering from HF hospitalization had the worst health status across the 3 domains. Nonresponse rates were low, in general, although higher for physical limitations hurrying or jogging and for all social limitation items (Table 2).

Table 2.

Item Descriptive Statistics: Derivation Samples

From the 6-item physical limitation domain, concordance with the full score was excellent for all possible 3-item subsets, including one item from each activity level (Appendix I in the Data Supplement). We selected items 1b (limitation with showering or bathing), 1c (limitation with gardening, vacuuming, and carrying groceries), and 1f (limitation hurrying or jogging) as representing a good balance of clinical relevance, item variability, and response rates. From the symptom frequency scale, we retained all 4 items to preserve the varied manifestations of HF symptoms. From the 3-item quality of life scale, concordance was excellent for all 2-item subsets and superior to single-item subsets. We selected items 12 (limitation of enjoyment of life) and 13 (feelings about spending the rest of your life with symptoms as they are now) for clinical relevance and item variability. Finally, from the social limitation scale, we omitted item 15d (limitation in intimate relationships) because of high nonresponse rates. We retained all 3 remaining items because that yielded the highest concordance with the full social limitation score among all possible subsets. In summary, we identified 12 items (3 physical limitation, 4 symptom frequency, and 2 quality of life and 3 social limitation items) to retain in the final short version of the KCCQ (Appendix II in the Data Supplement).

Validation

Construct Validity

Agreement between the KCCQ-12 and full KCCQ scores was excellent in all clinical settings, with concordances of 0.97 for physical limitation scores, 0.93 to 0.96 for quality of life scores, 0.98 for social limitation scores, and 0.98 to 0.99 for summary scores (Appendix III in the Data Supplement). Concordance was perfect for the symptom frequency domain because the same items are used in both instruments to generate that domain score. As with the full KCCQ, missing data occurred primarily for physical limitation and social limitation scores and was slightly more frequent in the KCCQ-12, but ≤10% in all clinical settings. The KCCQ-12 summary score also demonstrated a strong association with New York Heart Association class, comparable to that of the full KCCQ, with mean±SD summary scores ranging from 29±29 for Class IV to 86±15 for Class I patients with stable CAD (r=−0.43) and from 26±17 for Class IV to 70±22 for Class I patients after hospitalization for HF (r=−0.35; Table 3).

Table 3.

Association Between KCCQ Summary Scores and NYHA Class

Predictive Validity

All KCCQ-12 scores demonstrated a graded inverse relationship with the outcome of 6-month death or cardiovascular (CV) hospitalization, comparable to those of the full KCCQ (Table 4). Predictive power was consistently strongest for the overall summary score, with cumulative incidence among stable HF patients ranging from 7% for those with scores 75 to 100 to 33% for those with scores <25 (c-index=0.64) and among patients recovering from HF hospitalization ranging from 24% for those with scores 75 to 100 to 62% for those with scores <25 (c-index=0.63).

Table 4.

6-Month Death or Cardiovascular Hospitalization by KCCQ Score

Reproducibility

Among 79 clinically stable patients assessed 6 weeks after a HF clinic visit, the KCCQ-12 revealed minimal changes in scores between assessments and showed high intraclass correlations of ≥0.76 for all domains (Table 5). The overall summary score had the highest reproducibility with an intraclass correlation of 0.92.

Table 5.

6-Week Change in Stable Patients (N=79)

Responsiveness

The KCCQ-12 showed substantial responsiveness to clinical change. One week after hospitalization for acute HF, mean KCCQ-12 scores increased by >16 points for all domains, with the greatest increase in the symptom frequency score (31.0 points; Table 6). Standardized response means were good to excellent, ranging from 0.65 for physical limitation to 1.12 for the summary score.

Table 6.

1-Week Change After Acute Heart Failure Hospitalization (N=853)

Minimal Clinically Important Difference

Among patients at 6 weeks after a HF clinic visit, change in the KCCQ-12 summary score was strongly associated with physicians’ assessment of clinically significant improvement (c-index =0.67) and deterioration (c-index =0.77), mirroring that of the full KCCQ (c-indices of 0.68 and 0.77, respectively). The optimal cut point based on Youden’s index was 4.7 points for predicting improvement and −3.1 points for predicting deterioration, both agreeing favorably with analogous cut points for the full KCCQ (Table 7). A 5-point improvement or deterioration in scores for individual patients was associated with a 58% and 67% sensitivity and a 70% and 75% specificity, respectively, that a physician would have assessed the patient to have a clinically important improvement or deterioration in their HF status. These results were comparable to the full KCCQ and support a minimal clinically important difference of ≈3 to 5 points for the KCCQ-12 summary score.

Table 7.

Minimal Clinically Important Differences in KCCQ Overall Summary Score by 6-Week Physician Global Assessment (N=547)

Discussion

As the US healthcare system struggles to provide more cost-effective, patient-centered care, there is a growing need to efficiently monitor the health status of its patients.³³ For example, the Centers for Medicare and Medicaid Services has begun soliciting proposals for PROs measures to become part of its quality assessment programs (http://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/MMS/CallForMeasures.html). A critical challenge in using PROs for quality assessment is that they must be feasible to collect and incorporate into routine clinical care so that there is minimal burden on practitioners for reporting patients’ health status to payers. Moreover, creating a shortened version of the KCCQ provides a more feasible means to routinely assess patients’ health status to improve clinical care and population management. In this study, we have been able to substantially reduce the KCCQ from 23 to 12 items, while retaining the validity, reproducibility, responsiveness, predictive validity, and interpretability of the original instrument. Moreover, the high concordance (0.93–1.0) between the domain scores of the 2 versions—established in several distinct clinical circumstances—suggests that the scores can be used interchangeably.

The availability of a shortened disease-specific health status measure for patients with HF has the potential to improve care. First, quantifying patients’ health status at each outpatient visit has been endorsed as a performance measure of healthcare quality.³⁴ Although this measure can be met with either the New York Heart Association classification or 1 of 3 PROs, including the KCCQ, the inter-rater reliability of the New York Heart Association is poor,^35–37 with a concordance of 54% as compared with the intraclass correlation coefficient of 92% for the KCCQ-12 overall summary score. Having more reliable estimates of patients’ health status through self-reported PROs becomes, in essence, a standardized history that can provide a better estimate from which to assess whether patients’ conditions have changed over time. In addition, incorporating a short PRO into the clinical examination can theoretically improve the efficiency of a provider’s visit by enabling providers to know, at a glance, whether the patient is doing better or worse as compared with the prior visit. We recently cited this as a potential benefit of the routine use of PROs and highlighted the need for a feasible version of PROs to support their adoption in clinical care.¹⁵ In addition, the foundation of population health and disease management programs is to assess the health status of many patients and to direct resources to those with the greatest potential to benefit. Were health systems to assess the health status of their HF patients with the KCCQ-12, they could readily identify those with the worst symptoms and highest risk for mortality and hospitalization^10–12,22 and those expected to cost the most,⁹ so that they could preferentially direct more care to these patients to improve their outcomes. Testing these applications of PROs in clinical practice is a high research priority as healthcare moves from a volume-based to a value-based reimbursement model.

Although there are several disease-specific PROs available for patients with HF, the original KCCQ had the greatest amount of psychometric data supporting its use, including standards for defining clinically important differences in scores,¹³ and separately quantified a broader range of clinically important domains, including symptoms, physical limitations, social limitations, and quality of life, than other measures. Accordingly, several recent reviews of PROs in HF have recommended the KCCQ or Minnesota Living with Heart Failure questionnaires as the preferred instruments.^5,6 Creating a shorter version of the KCCQ that maintains all of the psychometric properties of the original instrument should further enhance its value and encourage other instrument developers to explore reducing the length of their measures.

The development of the KCCQ-12, a shortened version of the original 23-item KCCQ, should be interpreted in the context of the following potential limitations. First, although we demonstrated excellent psychometric performance of the KCCQ-12, all of the limitations of the original instrument likely apply to the shortened version. For example, patients who do not perform activities because of conditions other than their HF will have missing scores for the physical limitation domain. Moreover, although the shortened version should minimize missing data, any informatively missing health status assessments need to be handled carefully, through appropriate statistical methods, to avoid introducing bias. In addition, despite data showing comparable performance of the KCCQ in patients with valvular heart disease or HF because of preserved ejection fraction,^21,22 all of these analyses were performed in patients with reduced ejection fraction, and replication of these analyses in patients with other etiologies of HF may be important.

In conclusion, we have validated a 12-item version of the KCCQ and shown it to have comparable psychometric properties as compared with the original, 23-item version. It preserves the high test–retest reliability, responsiveness, prognostic ability, and interpretability thresholds of the original KCCQ, while reducing the response burden by half. It thus has the potential to be more implementable in routine clinical care and can open up important opportunities to improve the quality of HF care through quality assessment efforts, tailored population health interventions, more practical clinical trials, and better doctor–patient communication. Testing these applications are important opportunities for future research and, if successful, can support more patient-centered care.³³

Disclosures

Dr Spertus discloses that he owns the copyright to the KCCQ, SAQ, and PAQ; has grant support from Lilly, Abbott Vascular, and Genentech; provides scientific consulting services to Amgen, Novartis, Janssen, and Regeneron; and has an equity interest in Health Outcomes Science.

Supplementary Material

hcq-8-469-s001.pdf ^{(18.4KB, pdf)}

hcq-8-469-s002.docx ^{(68.9KB, docx)}