Validation and responsiveness of the Persian version of HeartQoL questionnaire in cardiac rehabilitation after coronary artery bypass grafting: An observational study

Abstract

BACKGROUND

Decision making and the quality of care provided for chronic diseases have been shown to improve through patient participation. The HeartQoL questionnaire is a core health-related quality of life (HRQOL) tool specifically designed for individuals with ischemic heart disease (IHD) who have undergone interventions such as cardiac rehabilitation (CR).

METHODS

In this observational and multicenter study, 150 patients were recruited. The participants completed the HeartQoL, MacNew Heart Disease Questionnaire, and Short Form Health Survey (SF-36) on entering CR for validity assessment. The HeartQoL along with a Global Rating of Change (GRoC) scale (for responsiveness measurement) were completed by 100 participants 3 months later.

RESULTS

The mean age of all participants in validity assessment was 61.87 ± 8.13 years. Cronbach’s alphas of the total scales ranged from 0.70 to 0.81 and of the subscales from 0.70 to 0.82. The Pearson correlation coefficient was used to determine construct validity; similar constructs were confirmed with correlation coefficients ranging from 0.50 to 0.69 and dissimilar constructs with correlation coefficients ranging from 0.28 to 0.29 (P < 0.010). The assessment of the responsiveness of the questionnaire indicated that the area under curve (AUC) was greater than 0.70 (range: 0.74 to 0.91) and the optimal cut-off point was 0.65.

CONCLUSION

The Persian version of the HeartQoL questionnaire demonstrated satisfactory psychometric properties in the sample of participants admitted to CR after coronary artery bypass grafting (CABG). The present study results showed that the HRQOL can be used by clinicians and researchers in conjunction with other outcome measures to gain additional information about symptoms relevant to HRQOL in patients referred to CR and to evaluate change over time.

Introduction

In line with increasing survival rates, today, medical centers pay more attention to decreasing the prevalence of morbidity after cardiac events. Coronary artery disease (CAD) is one of the leading causes of disability in the world.1,2 The prevalence of coronary risk factors in the Iranian population is the same as some Middle Eastern countries, but is higher than Western countries.3 As an achievement in health care in recent years, patients’ attitudes toward their situation are now considered to be as valid as that of clinicians in clinical practice.4,5 Patient participation has been shown to improve decision making and the care provided for chronic diseases.6

The use of patient-centered outcome measures like the health-related quality of life (HRQOL) in research studies and clinical practice has been recommended by health organizations such as the European Medicines Agency (EMA)7, the US Food and Drug Administration (USFDA)8, and by the National Heart, Lung, and Blood Institute9 and the American Heart Association10 in patients with cardiovascular disease (CVD). While generic HRQOL questionnaires are designed to allow broad comparisons of health status, disease-specific HRQOL questionnaires are appropriate outcome measures for both therapeutic intervention trials and clinical care.11 Some generic measures and several disease-specific HRQOL questionnaires have been used to evaluate the HRQOL of patients with CAD. The reliability, validity, and responsiveness of a health status questionnaire must be assessed and approved for its use in research or clinical practice. Validity implies the validity of a single score, and responsiveness is interpreted as validity of the change score.12

Patients who suffer from different types of ischemic heart disease (IHD) are referred to cardiac rehabilitation (CR). The physical work capacity and quality of life (QOL) of patients with CVDs are improved through CR, and the 6-minute walk test (6MWT), which measures physical work capacity, has been reported as a reliable tool for serial comparisons of CR programs.13 Core QOL questionnaires that can be used in individuals with the 3 major IHD diagnoses [angina, myocardial infarction (MI), and heart failure], are needed in CR centers.

The HeartQoL questionnaire is a core HRQOL questionnaire specifically designed for patients with IHD who have undergone interventions such as CR, and is commonly used in more than one IHD diagnosis.14 The HeartQoL questionnaire has also been validated in patients who have undergone heart valve surgery, which in some cases was combined with coronary artery bypass grafting (CABG),15 individuals with atrial fibrillation (AF)16 and those with an implantable cardioverter defibrillator (ICD).17 In a previous study, HeartQoL was administered as an outcome measure assessment instrument for CR after CABG.18 Therefore, it seems that the questionnaire can be used in a wide range of CVDs as a core heart disease-specific HRQOL questionnaire. Despite the prevalence of patients referred to CR centers after CABG in Iran, the Persian version of the HeartQoL questionnaire has not been validated in this group of patients. The aim of the present study was the validation of the Persian version of the HeartQoL questionnaire in CR after CABG.

Materials and Methods

This observational and multicenter study was conducted on 150 patients who had undergone CABG and were admitted to the 3 CR units of Imam Khomeini Hospital in Ahvaz, and Tehran Heart Center and Shariati Hospital in Tehran, Iran. They were selected using convenience non-probability sampling. The study inclusion criteria consisted of age of above 18 years, ability to write and read in Persian, lack of any serious psychiatric disorders, and undergone CABG in the past 1 to 2 months. The study exclusion criterion was the unwillingness to cooperate. Before distributing the questionnaires among the participants, the study aim and procedure were explained to them and signed consent forms were obtained from them. Demographic data [age, sex, and body mass index (BMI)] and clinical information [ejection fraction (EF), diabetes, high blood pressure, and hypercholesterolemia as risk factors] were recorded. The study (IR.AJUMS.REC.1397.762) was approved by the Medical Ethics Committee of Ahvaz Jundishapur University of Medical Sciences, Iran.

Patient-Reported Outcome Assessment: All participants completed the Persian version of the HeartQoL, MacNew Heart Disease Questionnaire, and Short Form Health Survey (SF-36) on entering CR. Measurement properties were evaluated based on the Consensus-based Standards for the selection of health Measurements Instruments (COSMIN)12 and accoding to that, construct validity (convergent and divergent hypothesis testing and discriminant validity), reliability (internal consistency, test-retest reliability, and measurement error) and responsiveness were assessed. Floor and ceiling effect were also assessed for interpretability.

Instruments

The HeartQoL Questionnaire: The HeartQoL Questionnaire is a core HRQOL designed in 2014 for assessing IHD. The original questionnaire has been reported as reliable, valid, and responsive to change in patients with angina, MI, or heart failure.19 The HeartQoL comprises 14 items19 scored on a 4-point scale ranging from 0 to 3 (‘not bothered’ to ‘bothered a lot’). Of the items, 10 (items 1-8 and 13-14) are related to physical wellbeing and 4 (items 9-12) to emotional wellbeing. Higher scores indicate better HRQOL. The global scale (all items), and the physical (10 items) and emotional (4 items) subscale scores can be calculated as the mean of the scored items.14 The Persian version of the HeartQoL was translated and culturally adapted by Ranjandish et al.20

MacNew Heart Disease Questionnaire: The MacNew Heart Disease Questionnaire is a self-administered patient-reported HRQOL and IHD-specific instrument, which has been validated in patients with angina, MI, and heart failure.21 It contains 27 items with a global HRQOL scale and the 3 physical, emotional, and social subscales; the scores of the global scale and each subscale range from 1 to 7 with higher scores indicating better HRQOL.22 The MacNew Heart Disease Questionnaire is designed for assessing patients’ attitudes toward CAD effects on daily functioning. This questionnaire has been applied in the area of CR to assess the psychological aspects underlying the psychophysical recovery phase following surgical revascularization in patients with CAD. The Persian version of the MacNew was translated and culturally adapted by Asadi-Lari et al.23

The Short Form Health Survey: The SF-36 is a valid generic health survey. It consists of 36 items related to general health in the 2 summary scales of physical component summary (PCS) and mental component summary (MCS). Moreover, the SF-36 consists of the 8 subscales of physical functioning, role-physical, bodily pain, general health, vitality, social functioning, role-emotional, and mental health. The sum of the raw scores are converted to a 0-100 scale.24 The SF-36 was translated into Persian and adapted to the Iranian culture by Montazeri et al.25

Global Rating of Change: The Global Rating of Change (GRoC) score is a single-question tool about placing an ‘X’ in the box which best represents the change in health status since a previous time-point and was used as an external criterion to determine whether participants had improved, worsened, or not changed. An open question leaves the patient to decide what construct(s) he or she considers important in determining health status.26 In this study, a 9-point Likert scale ranging from ‘very much worse’ to ‘very much better’ was used to score the item.

Floor and Ceiling Effects: Floor and ceiling effects occurred when participants scored close to the lowest (score = 0) and highest score (score = 3), respectively. At percentages below 15%, floor and ceiling effects were considered absent.27

Reliability

The reliability of the HeartQoL was first evaluated by examining its internal consistency (Cronbach’s α); values between 0.7 and 0.95 were considered appropriate.12 Test-retest reliability was assessed by approximately 10% of patients (n = 15) who were retested 7-10 days after the first time under the same conditions of measurement. An intraclass correlation coefficient (ICC) value of ≥ 0.70 was considered as the criterion value.28 Moreover, measurement error was calculated using the standard error of measurement (SEM). The smallest detectable change (SDC) was estimated using the equation SDC = 1.96 × √2 × SEM.

Validity

Construct validity was evaluated by formulating a priori hypotheses for expected correlations of the HeartQoL with MacNew and SF-36 as reference instruments. Convergent and divergent validity were evaluated using the Pearson correlation coefficient. Discriminant validity was determined through the ‘known group’ approach, by examining whether the questionnaire could discriminate between participants with different EF groups (EF < 50% and EF ≥ 50%) as an index of cardiac function, the possible effect of this index on HRQOL has been previously indicated.29 Comparison analysis was performed between the groups divided based on the score of the questionnaire using independent t-test to determine the discriminant validity.

Responsiveness

Longitudinal validity is a measure of responsiveness. It is assessed using the correlation between the change score of the questionnaire and the change score of the reference instrument.30 To assess responsiveness in this study, 100 participants completed the HeartQoL and a GRoC scale 3 months after entering CR. The intervention consisted of 24 sessions of CR. The receiver operating characteristic (ROC), as an anchor-based method, was used with 95% confidence interval (CI) to assess the responsiveness of the questionnaire.

SPSS software (version 19; SPSS Inc., Chicago, IL., USA) was used for data analysis. All P-values of less than 0.050 were considered as statistically significant. The reliability of the questionnaire was evaluated using internal consistency (Cronbach’s α) and test-retest reliability (ICC). Responsiveness was evaluated using the ROC with a 95% CI. An area under curve (AUC) ≥ 0.7 was considered adequate.27

Correlation analysis was conducted between the change scores of the HeartQoL and raw scores of GRoC using the gamma correlation coefficient to assess the responsiveness of the questionnaire. Correlation coefficients of 0.50-0.75 and greater than 0.75 were considered moderate to good relationship and good to excellent relationship, respectively.31

Results

Participants Characteristics: Baseline demographic and clinical information of 150 patients after CABG were obtained on entering CR (Table 1). The mean age of all participants was 61.87 ± 8.13 years, 80.7% of the participants (n = 121) were men and 19.3% (n = 29) were women.

Table 1.

Demographic and clinical characteristics of all patients (n = 150)

Characteristic		Mean ± SD
Age (year)		61.87 ± 8.13
BMI (kg/m2)	Men (n = 121)	26.42 ± 3.66
	Women (n = 29)	27.26 ± 4.10
		n (%)
Ejection Fraction	< 50	63 (42.0)
	≥ 50	87 (58.0)
Diabetes		67 (44.7)
Hypertension		57 (38.0)
Hypercholesterolemia		50 (33.3)

SD: Standard deviation; BMI: Body mass index

Floor and Ceiling Effects: There were no ceiling or floor effects for total and subscale scores (total ceiling effect = 5.4%, total floor effect = 1.6%).

Reliability

The Cronbach’s alphas of the total scale and subscales ranged from 0.70 to 0.81 and 0.70 to 0.82, respectively; thus, the internal consistency of the Persian version of the HeartQoL exceeded the criterion (Table 2). The test-retest reliability of the total and subscale scores (range: 0.89 to 0.92) also exceeded the criterion (Table 2).

Table 2.

Reliability (internal consistency and test-retest reliability) of the HeartQoL among patients after coronary artery bypass grafting on entering cardiac rehabilitation

Variable	Internal consistency (Cronbach’s alpha)	ICC2,1 (95% CI)	SEM	SDC
	(n = 150)	(n = 15)
HeartQoL Physical subscale	0.76	0.92 (0.76-0.97)	0.14	0.38
HeartQoL Emotional subscale	0.76	0.89 (0.68-0.96)	0.20	0.55
Total HeartQoL score	0.77	0.92 (0.77-0.97)	0.15	0.41

ICC: Intraclass correlation coefficient; CI: Confidence interval; SEM: Standard error of measurement; SDC: Smallest detectable change

Validity

Convergent and Divergent Validity: Through the assessment of construct validity, the priori convergent hypotheses for moderate to strong correlations of HeartQoL with MacNew and SF-36 constructs and weak correlations between dissimilar constructs were found to be statistically significant (P < 0.010), and thus, were approved. Similar constructs were confirmed with an r value range of 0.50-0.69 (Table 3). The correlation value (r) between the physical subscale of the HeartQol and the emotional subscale of the MacNew was 0.28, and between the physical subscale of the HeartQol and the mental component summary of the SF-36 was 0.29 (Table 3). An r < 30 and 50 < r < 70 is interpreted as weak and moderate correlation, respectively.32

Table 3.

Construct validity of the HeartQoL among patients after coronary artery bypass grafting on entering cardiac rehabilitation (n = 150)

Variable	HeartQoL Physical subscale	HeartQoL Emotional subscale	Total HeartQoL score
MacNew Physical subscale	0.67*	0.29*	0.69*
MacNew Emotional subscale	0.28*	0.68*	0.55*
MacNew Social subscale	0.50*	0.29*	0.50*
Total MacNew score	0.55*	0.61*	0.67*
SF-36 PCS	0.61*	0.29*	0.62*
SF-36 MCS	0.29*	0.55*	0.56*

PCS: Physical component summary; MCS: Mental component summary

^*P < 0.010

Discriminative Validity: Differences in the total, physical, and emotional scores show that HRQOL was poorer in CAD patients who had reduced EF (EF < 50%) (Table 4).

Table 4.

Discriminative validity of the HeartQoL among patients after coronary artery bypass grafting on entering cardiac rehabilitation

Variable	Ejection Fraction		P
	< 50 (n = 63)	≥ 50 (n = 87)
Total scale	1.71 ± 0.34	2.36 ± 0.32	< 0.001
Physical subscale	1.65 ± 0.40	2.26 ± 0.38	< 0.001
Emotional subscale	1.86 ± 0.60	2.43 ± 0.41	< 0.001

Responsiveness

The change score was obtained by subtracting the initial score from the follow-up score (Table 5). Therefore, a positive score illustrated improvement and a negative change score indicated deterioration. The AUC and minimal clinically important difference (MCID) were obtained from the ROC (Table 6). MCID is the smallest meaningful change to the patient31 that can differentiate, with the highest sensitivity and specificity, between improved and unimproved patients. There is no global agreement on the optimal cut-off point on an anchor. As the participants had undergone two treatment procedures (surgery and CR), the health status change to “very much better” or “much better” was considered as a good outcome (external criterion variable = 1), all others were considered as poor outcome (external criterion variable = 0). The dichotomized GRoC was “good” for 61% of participants (Table 5).

Table 5.

Mean scores and standard deviation of pre-intervention, post-intervention, and change scores of the HeartQoL among patients after coronary artery bypass grafting in cardiac rehabilitation (n = 100)

Score	Pre-intervention	Post-intervention	Change score
Total (n = 100)	1.94 ± 0.36	2.59 ± 0.36	0.65 ± 0.05
Good outcome (n = 61)	1.88 ± 0.37	2.65 ± 0.23	0.77 ± 0.04
Poor outcome (n = 39)	2.04 ± 0.32	2.24 ± 0.40	0.20 ± 0.05

Data are reported as mean ± standard deviation (SD)

Table 6.

Responsiveness of the HeartQoL among patients after coronary artery bypass grafting in cardiac rehabilitation (n = 100)

Questionnaire	Gamma coefficient (P)	AUC (95% CI)	Optimal cut-off value	Sensitivity (95% CI)	Specificity (95% CI)
HeartQoL	0.66 (P < 0.001)	0.82 (0.74-0.91)	0.65	0.63 (0.49-0.76)	0.87 (0.74-0.94)

AUC: Area under curve; CI: Confidence interval

To verify the strength of a questionnaire in following changes, it is important to know that SDC, which is a measure of variation in a scale due to measurement error, should be smaller than minimal important change (MIC) to allow the differentiation of important changes from measurement error in individual patients.33

Assessing responsiveness indicated that the AUC was greater than 0.70 (range: 0.74-0.91) and the optimal cut-off point was 0.65 (Figure 1). Gamma correlation coefficient was considered good between change scores of the HeartQoL and raw scores of GRoC (0.66; P < 0.001) (Table 6).

Figure 1.

Receiver operating characteristics curve of the HeartQoL among patients after coronary artery bypass grafting in cardiac rehabilitation (n = 100)

Discussion

The present study results illustrated the reliability, validity, and responsiveness of the Persian version of the HeartQoL questionnaire in the assessment of HRQOL among patients who have undergone CABG in CR units. Internal consistency reliability was ≥ 0.76, and test-retest reliability was significant and ≥ 0.89 in the global scale and subscales.

All correlations of the corresponding HeartQoL with MacNew and SF-36 constructs were both moderate and significant ranging from 0.50 to 0.69 and significantly lower correlations were observed between dissimilar constructs. Discriminative validity was demonstrated with different EF groups (P < 0.050). The assessment of responsiveness indicated that the AUC was greater than 0.70 (range: 0.74 to 0.91) and the optimal cut-off point was 0.65. The gamma correlation coefficient was considered as good between change scores of the HeartQoL and raw scores of GRoC (0.66).

No floor or ceiling effects were observed in this study, which was consistent with the psychometric properties of the main questionnaire.19 The internal consistency reliability of the total HeartQoL and its subscales was sufficient, which is in agreement with the original HeartQoL validation study19 and indicates that the items of the Persian version of the HeartQoL measure the same concept. The test-retest reliability was found to have satisfactory replicability as demonstrated by a high ICC in the total scale and its subscales, which was consistent with previous studies that indicated an ICC > 0.8615-17, and the study by Lee et al. who reported ICC > 0.78.34 In the current study, the SDC of the total scale and its subscales was 0.41 and 0.38-0.55, and in the post heart valve surgery population, it was 0.6 and 0.5-0.7,15 respectively.

In the study by Lee et al., the SDC of the total scale and its subscales was 0.55 and 0.67-0.7,34 and in the study by Zangger et al., it was 0.56 and 0.62-0.76, respectively.17 The validation of an instrument refers to the degree to which it can measure what it purposes to measure. In this study, construct validity was evaluated using the correlations of the HeartQoL with MacNew and SF-36 as reference instruments. In previous studies, the SF-36 has also been used as a reference questionnaire for construct validity in a post heart valve surgery population,15 AF population,16 and in patients with an ICD.17

The correlation of the HeartQoL with SF-36 across similar constructs was reported as moderate-strong; thus, its convergent validity was approved in a post heart valve surgery population (84 > r > 0.68), AF population (81> r > 0.78), and in patients with an ICD (82 > r > 0.72). In the present study, a moderate correlation was observed between the HeartQoL and SF-36 (62 > r > 55). In this study, discriminative validity was demonstrated with different EF groups (P < 0.050); however, discriminative validity in a post heart valve surgery population,15 AF population,16 and in patients with an ICD17 was confirmed using SF-36 health transition and some other items.

This study seems to be the first one to evaluate the responsiveness of the HeartQoL using the ROC as an anchor-based method. In the study by Oldridge et al.,35 change in the HeartQoL score was analyzed using paired t-test and effect size (ES) statistics. The 3-month improvement in mean HeartQoL scores was assessed in patients with angina or MI, who had undergone percutaneous intervention (PCI) or had been referred to CR. They found that the scores were significantly higher after 3 months compared to before. As a measure of the magnitude of the response to treatment, ES statistics were small in PCI (< 0.44) for all patients, and moderate (0.63 < ES < 0.74) in CR with the exception of the emotional subscale in patients with MI. Responsiveness shows the instrument’s ability to detect change overtime.

Gronset et al. conducted a study on conventional open-heart valve replacement alone or combined with CABG and percutaneous valve replacement or repair in Denmark.15 They found that some items of the HeartQoL questionnaire are not in agreement with the sternal precautions and specific activity restrictions after median sternotomy.15 The issue is the point of resemblance with this study as for CABG there is sternal precautions after the surgery. The low Cronbach’s alpha and moderate Pearson correlation in the present study in comparison with other studies may be on account of activity restriction 1-2 months after CABG. The HeartQoL has been previously administered in a randomized clinical trial conducted in Denmark for assessing CR outcomes after CABG.18

The limitation of the present study was that it was performed on Persian-speaking patients with IHD after CABG who attended CR, and despite prevalence of this type of patients in these centers, the study results cannot be generalized to all patients in CR. Moreover, further research is needed to validate the HeartQoL in patients with other types of IHD referred to CR.

The Persian version of the HeartQoL demonstrated satisfactory psychometric properties for use in individuals referred to CR after CABG, so it seems it can be recommended for used in related clinical practice and research trials.

The HeartQoL is a core HRQOL instrument specific to IHD following interventions such as CR. The HeartQoL has also been validated in a wide range of CVDs. The content of the questionnaire has been derived from well-established, condition-specific HRQOL instruments,14 so it is sufficiently generalized. Furthermore, the completion of the questionnaire is not too long to cause problems such as dishonest responses due to fatigue.

Conclusion

The Persian version of the HeartQoL demonstrated satisfactory psychometric properties in individuals admitted to CR after CABG. The present study results illustrated that the HeartQoL can be used by clinicians and researchers as a core HRQOL assessing tool with other outcome measures, such as the 6MWT, to gain additional information about HRQOL in patients referred to CR and to evaluate change over time.