Evidence regarding the validity, reliability, and responsiveness of health related quality of life instruments among people with ischaemic heart disease

Generic instruments

Nottingham Health Profile (NHP)

Major floor (minimum possible score on the scale) and ceiling (maximum possible score) effects have been reported for the NHP.¹ All scales on part one of the NHP were found to produce a ceiling effect (indicating no problems with health status) for at least 25% (50% in the case of the energy and social isolation scales) of a sample of mostly female patients six months after myocardial infarction (MI).² Ceiling effects have also been reported for between 16.6% and 62.5% of a sample of mostly male patients beginning a cardiac rehabilitation programme on all scales of part 1.³
An examination of the discriminative validity of part 1 of the NHP has shown a significant difference on all six scales between the New York Heart Association (NYHA) classes I, II, and III for people with angina (79% male; mean age 60 years).⁴ Later studies (with 73% of samples male; mean age 64-65 years) found that the Emotion and Sleep scales did not discriminate between the NYHA classes, unlike the other four scales in part one.^{1 5 6} The same insignificant results were reported when the Canadian Cardiovascular Society (CCS) classification system was used with a slightly younger all male group.⁷ Two of these studies also reported that for part 2 of the NHP, only the Home Maintenance scale was able to discriminate between NYHA classes I to III.^{1 6} There is a poor consensus about the ability of the NHP to discriminate between people with heart disease and samples of healthy people. Part 1 of the NHP has been found to discriminate between healthy people and people with stable coronary artery disease, angioplasty, or coronary artery bypass graft surgery (CABG) (68% male; mean age 59.2 years).^{8 9} However, using a sample with a similar male:female ratio as the latter studies, only the Mobility scale of part one of the NHP was able to discriminate between healthy people and people who underwent CABG or heart valve operations, and even this relation did not hold for those younger than 55 years.¹⁰ Furthermore, only the Energy and Pain scales in part 1 and a total score for part 2 have indicated a significant difference between healthy people and those who had experienced MI within the previous six months (60% female; mean age 73 years).²

All six scales of part 1 portray a significant improvement from before to after cardiac surgery in a predominantly male group^11-13 and a significant correlation was found between the NHP and ratings of chest pain for a sample of patients two years after cardiac surgery.¹⁴ Moreover, when used with people who had experienced MI, the six scales of part 1 showed a significant difference between those able to complete more than 12 minutes on a treadmill test and those who could not; those who had a history of angina within the previous month and those who had no recent history of angina¹⁵; those with frequent angina and those with infrequent angina; and finally, those with severe dyspnoea and those with mild or no dyspnoea.¹⁶

Reliability
Some researchers have used Cronbach's alpha to provide an indication of the internal consistency of part one of the NHP.^{3 9} In these studies, the sample was a mixture of people with stable coronary artery disease and those who had angioplasty or CABG and in both studies the alpha values were similar, ranging from 0.58 to 0.84. The Emotion scale had the highest level of internal consistency in both studies, and the Pain, Energy, and Isolation scales submitted the lowest alpha values. Cronbach's alpha ranges from 0 to 1 - the closer to 1, the stronger the internal consistency. Usually, an alpha value of about 0.7 or above is taken as evidence of acceptable internal consistency.
The test–retest reliability of both parts of the NHP was examined among a group of people six months to two years after MI.² The coefficients for the six scales in part 1 ranged from 0.65 to 0.86. A total score for part 2 had a coefficient of 0.88. Test–retest reliability is a correlation coefficient; usually values of about 0.6 or above are taken as evidence of an acceptable level of reliability.

Responsiveness
The responsiveness to change of the NHP is an area that has received little attention. However, effect sizes were calculated after the NHP was administered to people at six weeks and again at six months after MI. The effect sizes of all scales on part 1 were found to be very small (0.01 to 0.17), indicating poor responsiveness to a clinical change which other generic instruments were able to detect.¹⁷
 
 

Short Form 36 (SF-36)

The Role Physical and Role Emotional scales appear to demonstrate minor floor and ceiling effects. Follow up studies of predominantly male patients (mean age 57 to 67 years) with angina or after MI,^{18 19} undergoing cardiac surgery,²⁰ or taking part in cardiac rehabilitation,³ have estimated that between 15.2% and 62.5% of samples scored 0 (minimum possible score) on these two scales and between 16.4% and 57% scored 100 (maximum possible score) on these scales before beginning the rehabilitation programme or before undergoing surgery. Additionally, the same studies found that the Social Functioning scale and, to a lesser extent, the Bodily Pain scale demonstrated a ceiling effect. It is not unusual for people with ischaemic heart disease to attain high scores, though the large proportion of extremely high scorers suggests that some of the SF-36 scales have weak sensitivity. Furthermore, the ceiling effect indicates that ischaemic heart disease patients have no problems in these areas. This is further evidence for the suggestion that these SF-36 scales are not sensitive to specific problems experienced by ischaemic heart disease patients.

All of the SF-36 scales, except Mental Health in the case of women, indicated a significant improvement from before to after participating in cardiac rehabilitation (mean age of sample 61.4 years)²¹; and all of the SF-36 scales, except Mental Health and Role Physical, detected a significant deterioration in the clinical condition of people with coronary artery disease (84% male; mean age 60 years).²² The General Health scale failed to show a significant change between the preoperative and postoperative state of mostly male cardiac surgery patients (mean age 60 years).²⁰

The Mental Health and Energy scales, respectively were the only ones unable to differentiate significantly between people with coronary artery disease who received a health promotion intervention and those who received usual services (58% male; mean age 66 years).²³ In the case of people with angina, studies (predominantly male; mean age 65 to 67 years)^{19 24} have revealed that the SF-36 scales were unable to distinguish between clinical classes of angina (as determined by the CCS's classification system) and all scales have a low correlation with patients' (96% male; mean age 68 years) single item ratings of current health.²⁵ However, all of the SF-36 scales, except Role Emotional, were found to differentiate significantly between ischaemic heart disease patients who were experiencing angina and those who had a history of angina but were not currently experiencing this symptom.¹⁹ Moreover, preoperative scores on the SF-36 scales for male cardiac surgery patients were reported to be lower than norms for the general population²⁶ but approximated to the norms postoperatively.²⁰

Reliability
Estimates of internal consistency (using Cronbach's alpha) for the eight scales of the SF-36 range from 0.76 to 0.93 for people with angina,^{19 24} from 0.73 to 0.9 for patients undergoing cardiac surgery,^{20 26} and from 0.72 to 0.85 for cardiac rehabilitation participants, the largest section of whom had an MI while others had an angina attack or had undergone cardiac surgery.²⁷

Test–retest reliability after a delay of two weeks ranged from 0.67 to 0.84 for all the SF-36 scales except Bodily Pain, Social Functioning, Role Physical, and Role Emotional for angina patients²⁴ and ranged from 0.7 to 0.84 for all scales except Role Emotional for a group of cardiac rehabilitation participants.²⁷

Responsiveness
Effect sizes and standardised response means (SRM) for all eight scales of the SF-36, based on preoperative and postoperative scores, are available for people who underwent angioplasty procedures.²⁰ Six of the eight scales showed moderate to high responsiveness to change – values ranged from 0.52 to 0.86 for the SRM and from 0.53 to 1.12 for the effect sizes, with each scale submitting a similar value on both indices. However, the Mental Health scale indicated a low responsiveness to change (SRM = 0.33; effect size = 0.32) and the responsiveness of the General Health scale was very poor (SRM = 0.05; effect size = 0.06).

SRMs have also been reported after the administration of the SF-36 at two points in time, three months apart, to a group of patients who were mostly diagnosed with stable coronary artery disease.²² The pattern of the resulting SRMs is similar to that reported for angioplasty patients.²⁰ The SRM values ranged from 0.5 to 1.1 except for the Mental Health scale (SRM = 0.4) and the General Health scale (SRM = 0.1).
 
 

Sickness Impact Profile (SIP)

The SIP does not suffer from floor and ceiling effects to any large degree.¹ The total SIP score has indicated a significant improvement in female patients over the six months after hospital discharge after cardiac surgery²⁸ and for a mixed sex sample over six weeks after hospital discharge after cardiac surgery.²⁹ The Ambulation, Body Care, Social Interaction, Communication, and Alertness Behaviour scales of the SIP, the physical and psychosocial dimensions, and the total score of the SIP have been shown to discriminate to a significant level between the NYHA classes I and III among men, but there is little evidence to suggest that the other scales can discriminate between these classes of angina.^{1 6} A significant difference of scores on the physical dimension between those who took part in a cardiac recovery programme and those who did not was found for an overwhelmingly male sample (mean age 64 years) after CABG.³⁰ Finally, only the Ambulation, Recreation, and Emotion scales, and the total SIP score indicated a significant difference between a mainly female group of people after MI and a group of healthy people with similar age and sex characteristics.²

Reliability
The internal consistencies of the three scales which make up the physical dimension for a mostly male sample (mean age 64 years) after CABG were examined, and estimates ranged between 0.55 and 0.84.³⁰ The internal consistency of the total SIP score for a sample of women after CABG (mean age 62 years) was 0.94.²⁸ The same estimate was reported for a sample of male patients after MI.³¹

For post-MI patients, test–retest reliability estimates of the total SIP score have been reported at 0.92 for a male sample³¹ and 0.86 for a mainly female sample.² This latter study also included an estimate of the test–retest reliability of the physical (0.80) and psychosocial (0.82) dimensions of the SIP.

Responsiveness
A high effect size (0.73) for the Mobility scale and moderate effect sizes for the Body Care (0.3), Home Management (0.28), Eating (0.27), Emotion (0.27), and Work (0.23) scales of the SIP have been reported when administered at six weeks and six months after MI to a group of mostly men (mean age 62 years).¹⁷ Effect sizes for the remaining scales ranged from 0.01 to 0.15. The effect size for the total score was 0.05. This contrasts with the magnitude of the effect size (1.02) for the total SIP score when used with a group of women at six weeks and six months after cardiac surgery.²⁸
 
 

Disease specific instruments

Quality of Life after Myocardial Infarction (QLMI)

The QLMI-2 appears to have good construct validity in so far as different factor analytic studies produced confirmation of a three factor or domain structure, which explained 66.5% of the variance.^{32 33} A study of the discriminative validity of the QLMI-2 found a significant difference on all three domains between patients with or without a previous MI; and between those readmitted to hospital within six months of an MI and those who avoided readmission.³³

Reliability
Test–retest reliability between 8 and 12 months after MI ranged from 0.75 to 0.87 for the five domains of the QLMI. Test–retest reliability is a correlation coefficient; usually values of about 0.6 or above are taken as evidence of an acceptable level of reliability. Internal consistency estimates (using Cronbach's alpha) for these domains ranged from 0.5 to 0.78.³⁴ However, when reduced to the three domains of the QLMI-2, internal consistency improved (alpha = 0.93 tp 0.95).³³ Cronbach's alpha ranges from 0 to 1 – the closer to 1 the stronger the internal consistency. Usually, an alpha value of about 0.7 or above is taken as evidence of acceptable internal consistency.

Responsiveness
Standardised response mean values for the QLMI ranged from 0.57 to 1.43 for four of the five domains.³⁴ The SRM value for the symptoms domain was 0.27, indicating low sensitivity. The responsiveness of the QLMI-2 has not been studied (or at least reported).
 

Seattle Angina Questionnaire (SAQ)

Criterion related validity, by means of correlations with performance on a treadmill test, was found for the physical limitation domain of the SAQ (mean age 61 years; 95% male).³⁵ Correlational designs, using samples with a similar average age and sex ratio to the latter study have also found high criterion related validity between the anginal stability domain and patients' assessment of change over three months; the treatment satisfaction domain and a patient satisfaction questionnaire; and the disease perception domain and the general health scale of the SF-36.³⁵ The last result was repeated and a moderate correlation between the SAQ physical dimension and the SF-36 physical scale was also reported for an all male sample (mean age 65 years).²⁴

Scores on the SAQ anginal stability domain discriminated significantly between people with stable and unstable angina (mean age 59 years; 83% male).³⁵ In addition, scores on all of the SAQ domains, except treatment satisfaction, differed significantly between patients in classes I to III of the CCS classification system.²⁴ Furthermore, scores on all scales of the SAQ changed in a manner consistent with the direction of change reported by patients (that is, SAQ scores increased for patients who claimed to experience an improvement in their condition and SAQ scores decreased for patients who claimed to experience a deterioration in their condition) – this was reported not to be the case for the SF-36 scales.²²

Reliability
An all male sample (mean age 65 years) with stable angina recorded alpha estimates of internal consistency ranging from 0.66 to 0.89; and two week test–retest reliability coefficients ranging from 0.58 to 0.80 for all SAQ domains except anginal stability (test–retest coefficient = 0.33).²⁴ Three month test–retest reliability coefficients for a sample of people with stable coronary artery disease (mean age 61 years; 95% male) were estimated to range between 0.76 to 0.83 for all SAQ domains, except anginal stability (0.24).³⁵

Responsiveness
A responsiveness statistic (mean change in score among patients who improve divided by the standard deviation of scores in stable patients) was calculated for all scales of the SAQ and the results ranged from 1.2 to 2.3 except for the treatment satisfaction domain (0.1). These responsiveness indices were much higher than those found for the SF-36.²²
 

Quality of Life Index – Cardiac Version (QLI)

The QLI total score and the health and functioning scale have indicated a significant improvement in health related quality of life (HRQoL) after angioplasty for a mostly male sample (83% male; mean age 59 years).³⁶ Significant improvements on the total QLI score and the health and functioning and socioeconomic scales were recorded also for a group of male patients after taking part in a three month angina management programme (mean age 65 years).²⁴ For the same sample, the QLI health and functioning scale correlated moderately with the general health and physical functioning scales of the SF-36; and the QLI psychosocial scale correlated moderately with the mental health, social functioning, and general health scales of the SF-36. However, no significant differences were found between the CCS classifications I to III on any scales of the QLI, including the total score.²⁴

Reliability
Internal consistency estimates of the total QLI score, when used with patients undergoing angioplasty, ranged from 0.86 to 0.96.^{36 37} Test–retest reliability for the total QLI when used with this group of patients has been estimated at 0.87.³⁷ For people with angina, internal consistency estimates range from 0.70 to 0.93 and test–retest reliability estimates range from 0.68 to 0.79.²⁴
 

Angina Pectoris Quality of Life Questionnaire (APQLQ)

All scales except life satisfaction were found to correlate highly and significantly with a rating of angina severity; and the somatic symptoms and physical activities scales showed moderate and significant negative correlations with time spent on a treadmill test (mean age of sample 59.5 years; 84% male).³⁸ The APQLQ scales also displayed concurrent validity with relevant SF-36 scales for a mostly male sample (mean age 67 years). The physical activities scale correlated highly (0.76) with the SF-36 physical mobility scale; the emotional distress scale correlated highly (0.84) with the SF-36 mental health scale; and the somatic symptoms scale correlated highly (0.76) with the SF-36 vitality scale.³⁹ In terms of discriminant validity, all of the APQLQ scales and the total APQLQ score indicated a significant difference between symptomatic and asymptomatic patients with coronary artery disease.^{19 39} Furthermore, the physical activities scale of the APQLQ detected a significant improvement after CABG surgery for a sample of patients with a mean age of 62.5 years.⁴⁰ This improvement was maintained at three months, one year, and two years after the surgery.⁴¹ None of the other APQLQ scales were administered in the latter two studies.

Reliability
Internal consistency estimates for the four scales of the APQLQ range from 0.82 to 0.90.³⁹ The total APQLQ has a reported internal consistency estimate of 0.95.¹⁹
 

Summary Index

The Summary Index discriminated between men with stable angina (mean age 61.7 years) who had two or fewer angina related symptoms and those with more than two symptoms. However, there was no significant difference on the Summary Index between patients with either a high or a low angina attack rate.⁴² The Summary Index total score showed a significant improvement in HRQoL for patients with angina (all male; mean age 60-61 years) who received transdermal nitroglycerin or long acting oral nitrate medication.⁴³

Reliability
Test–retest (four week interval) reliability coefficients for the separate categories of the Summary Index ranged from 0.69 to 0.84. The reliability coefficient for the total Summary Index score was 0.84. Internal consistency estimates for the Summary Index categories were reported to range from 0.91 to 0.98.⁴²

Responsiveness
Standardised response means ranged from 0.18 (for the Alertness category) to 0.52 (for the Physical Exertion category). Responsiveness estimates for the total Summary Index score ranged from 0.25 to 0.44.⁴²

1. Bulpitt CJ. Quality of life as an outcome measure. Postgrad Med J 1997;73:613-16.
2. Visser MC, Koudstaal PJ, Erdman RA, et al. Measuring quality of life in patients with myocardial infarction or stroke: a feasibility study of four questionnaires in the Netherlands. J Epidemiol Commun Health 1995;49:513-17.
3. Dempster M, Bradley J, Wallace E, et al. Measuring quality of life in cardiac rehabilitation: comparing the Short Form 36 and the Nottingham Health Profile. Coronary Health Care 1997;1:211-17.
4. O'Brien BJ, Buxton MJ, Patterson DL. Relationship between functional status and health-related quality-of-life after myocardial infarction. Med Care 1993;31:950-5.
5. Ekeberg O, Klemsdal TO, Kjeldsen SE. Quality of life on enalapril after acute myocardial infarction. Eur Heart J 1994;15:1135-9.
6. Visser MC, Fletcher AE, Parr G, et al. A comparison of three quality of life instruments in subjects with angina pectoris: the Sickness Impact Profile, the Nottingham Health Profile, and the Quality of Well Being Scale. J Clin Epidemiol 1994;47:157-63.
7. Permanyer-Miralda G, Alonso J, Antó JM, et al. Comparison of perceived health status and conventional functional evaluation in stable patients with coronary artery disease. J Clin Epidemiol 1991;44:779-86.
8. Lukkarinen H, Hentinen M. Assessment of quality of life with the Nottingham Health Profile among patients with coronary heart disease. J Adv Nurs 1997;26:73-84.
9. Lukkarinen H, Hentinen M. Assessment of quality of life with the Nottingham Health Profile among women with coronary artery disease. Heart Lung 1998;27:189-99.
10. Trouillet JL, Scheimberg A, Vuagnat A, et al. Long-term outcome and quality of life of patients requiring multidisciplinary intensive care unit admission after cardiac operations. J Thorac Cardiovasc Surg 1996;112:926-34.
11. Caine N, Harrison SCW, Sharples LD, et al. Prospective study of quality of life before and after coronary artery bypass grafting. BMJ 1991;302:511-16.
12. Klersy C, Collarini L, Morellini MC, et al. Heart surgery and quality of life: a prospective study on ischemic patients. Eur J Cardiothorac Surg 1997;12:602-9.
13. Chocron S, Etievent JP, Viel JF, et al. Prospective study of quality of life before and after open heart operations. Ann Thorac Surg 1996;61:153-7.
14. Sjöland H, Wiklund I, Caidahl K, et al. Relationship between quality of life and exercise test findings after coronary artery bypass surgery. Int J Cardiol 1995;51:221-32.
15. Skinner JS, Albers CJ, Hall RJ, et al. Comparison of Nottingham Health Profile (NHP) scores with exercise duration and measures of ischaemia during treadmill exercise testing in patients with coronary artery disease. Eur Heart J 1995;16:1561-5.
16. Wiklund I, Herlitz J, Hjalmarson A. Quality of life five years after myocardial infarction. Eur Heart J 1989;10:464-72.
17. Taylor R, Kirby B, Burdon D, et al. The assessment of recovery in patients after myocardial infarction using three generic quality-of-life measures. J Cardiopulm Rehabil 1998;18:139-44.
18. Ware JE, Snow KK, Kosinski MK, et al. SF-36 Health Survey Manual and Interpretation Guide. Boston: The Health Institute, New England Medical Center; 1993.
19. Marquis P, Fayol C, Joire JE. Clinical validation of a quality of life questionnaire in angina pectoris patients. Eur Heart J 1995;16:1554-60.
20. Krumholz HM, McHorney CA, Clark L, et al. Changes in health after elective percutaneous coronary revascularization - A comparison of generic and specific measures. Med Care 1996;34:754-9.
21. Lavie CJ, Milani RV. Effects of cardiac rehabilitation and exercise training on exercise capacity, coronary risk factors, behavioural characteristics and quality of life in women. Am J Cardiol 1995;75:340-3.
22. Spertus, JA, Winder JA, Dewhurst TA, et al. Monitoring the quality of life in patients with coronary artery disease. Am J Cardiol 1994;74:1240-4.
23. Campbell NC, Thain J, Deans HG, et al. Secondary prevention clinics for coronary heart disease: randomised trial of effect on health. BMJ 1998;316:1434-7.
24. Dougherty CM, Dewhurst T, Nichol WP, et al. Comparison of three quality of life instruments in stable angina pectoris: Seattle Angina Questionnaire, Short Form Health Survey (SF-36), and Quality of Life Index-Cardiac Version III. J Clin Epidemiol 1998;51:569-75.
25. Chen AY, Daley J, Thibault GE. Angina patients' ratings of current health and health without angina: associations with severity of angina and comorbidity. Med Decis Making 1996;16:169-77.
26. McCarthy MJJ, Shroyer AL, Sethi GK, et al. Self-report measures for assessing treatment outcomes in cardiac surgery patients. Med Care 1995;33:OS76-85
27. Jette DU, Downing J. Health status of individuals entering a cardiac rehabilitation program as measured by the Medical Outcomes Study 36 item short form survey. Phys Ther 1994;74:521-7.
28. Redeker NS, Mason DJ, Wykpisz E, et al. Women's patterns of activity over 6 months after coronary artery bypass surgery. Heart Lung 1995;24:502-11.
29. Artinian NT, Duggan C, Miller P. Age differences in patient recovery patterns following coronary artery bypass surgery. Am J Crit Care 1993;2:453-61.
30. Moore SM. The effects of a discharge information intervention on recovery outcomes following coronary artery bypass surgery. Int J Nurs Stud 1996;33:181-9.
31. Munro BH, Creamer AM, Haggerty MR, et al. Effect of relaxation therapy on post-myocardial infarction patients' rehabilitation. Nurs Res 1988;37:231-5.
32. Lim LL-Y, Valenti LA, Knapp JC, et al. A self administered quality of life questionnaire after acute myocardial infarction. J Clin Epidemiol 1993;46:1249-56.
33. Valenti L, Lim L, Heller RF, et al. An improved questionnaire for assessing quality of life after acute myocardial infarction. Qual Life Res 1996;5:151-61.
34. Hillers TK, Guyatt GH, Oldridge N, et al. Quality of life after myocardial infarction. J Clin Epidemiol 1994;47:1287-96.
35. Spertus JA, Winder JA, Dewhurst TA, et al. Development and evaluation of the Seattle Angina Questionnaire - a new functional status measure for coronary artery disease. J Am Coll Cardiol 1995;25:333-41.
36. Bliley AV, Ferrans CE. Quality of life after coronary angioplasty. Heart Lung 1993;22:193-9.
37. Faris J, Stotts N. The effect of percutaneous transluminal coronary angioplasty on quality of life. Progr Cardiovasc Nurs 1990;5:132-40.
38. Wiklund I, Comerford MB, Dimenäs E. The relationship between exercise tolerance and quality of life in angina pectoris. Clin Cardiol 1991;14:204-8.
39. Marquis P, Fayol C, Joire JE, et al. Psychometric properties of a specific quality of life questionnaire in angina pectoris patients. Qual Life Res 1995;4:540-6.
40. Sjoland H, Wiklund I, Caidahl K, et al. Improvement in quality of life and exercise capacity after coronary bypass surgery. Arch Intern Med 1996;156:265-71.
41. Sjoland H, Hartford M, Caidahl K, et al. Improvement in various estimates of quality of life after coronary artery bypass grafting in patients with and without a history of hypertension. J Hypertens 1997;15:1033-9.
42. Wilson A, Wiklund I, Lahti T, et al. A summary index for the assessment of quality of life in angina pectoris. J Clin Epidemiol 1991;44:981-8.
43. Nissinen A, Wiklund I, Lahti T, et al. Antianginal therapy and quality of life - a comparison of the effects of transdermal nitroglycerin and long-acting oral nitrates. J Clin Epidemiol 1991;44:989-97.