Abstract
Objective: To describe the development and scientific validation of a new patient based measure, the coronary revascularisation outcome questionnaire (CROQ), to evaluate health outcomes and quality of life before and after coronary artery bypass grafting and percutaneous transluminal coronary angioplasty.
Design and setting: Psychometric validation study conducted with patients from three hospitals in the UK.
Patients: Two independent field tests were conducted by postal survey of 714 patients before and 1329 patients after coronary revascularisation to evaluate the measurement properties of the CROQ.
Methods: Qualitative methods including patient interviews were used to develop questionnaire content. A full psychometric evaluation was performed on the survey data.
Results: Psychometric tests with the application of stringent criteria confirmed the acceptability (low missing data, good response rates), scaling assumptions (good item convergent and discriminant validity), reliability (good internal consistency and reproducibility), validity (good content and construct validity), and responsiveness of the CROQ.
Conclusions: The CROQ is a practical and scientifically sound patient based measure of outcome developed using psychometric methods. It captures aspects of recovery not addressed in other cardiac questionnaires and has been shown to be a highly responsive instrument that will be useful in evaluating outcomes in clinical trials.
Keywords: bypass, angioplasty, quality of life, questionnaire
Most clinical trials evaluating the relative effectiveness of coronary revascularisation procedures have focused primarily on differences in mortality and morbidity. However, measures of morbidity are often poorly related to subjective accounts of health and well being1 and do not capture all aspects of outcome that are important to patients. To improve the evaluation of treatments, generic measures of health related quality of life such as the short form 36 (SF-36)2 are now increasingly being used alongside clinical measures.3,4 Generic measures are useful for comparing diseases but measure the health status of the patient in general and do not address the condition under evaluation. Disease specific measures are more responsive in detecting treatment effects.5 However, few clinical trials have used disease specific measures to evaluate outcomes in different coronary revascularisation procedures.
Several coronary heart disease specific questionnaires have been developed, but the majority have not been validated against rigorous scientific standards (S Schroter, PhD thesis, London University, 2001). Psychometric methods provide well established scientific techniques for measuring subjective judgements on numerical scales and for evaluating the scientific rigour of measurement scales (that is, reliability, validity, and responsiveness). There are now rigorous criteria for evaluating the scientific robustness of patient reported health outcome measures.6 Although several disease specific measures have been developed to meet these criteria to evaluate outcomes in angina7,8,9,10 and myocardial infarction,11 no validated questionnaires have been developed to evaluate outcomes specific to coronary revascularisation. There is also substantial evidence that there are unique concerns specific to the experience of coronary revascularisation12,13 such as adverse effects, physical and psychological recovery from the interventions, and satisfaction with the procedures that are not covered by existing coronary heart disease specific measures.
We describe the development and scientific validation of the coronary revascularisation outcome questionnaire (CROQ), a new patient based measure to evaluate health outcomes and health related quality of life after coronary revascularisation (coronary artery bypass grafting (CABG) and percutaneous transluminal coronary angioplasty (PTCA)).
METHODS
Questionnaire development
We used four sources of information to develop the content of the CROQ: firstly, the literature of health outcomes in coronary heart disease; secondly, existing patient based measures of outcome in coronary heart disease; thirdly, the expert opinions of key health care professionals involved in cardiac patient care (cardiac surgeons, cardiologists, cardiac specialist nurses, pain control nurses, and cardiac liaison nurses) about problems commonly reported by patients undergoing coronary revascularisation; and fourthly, qualitative in-depth interviews with 10 patients who had undergone CABG and 10 who had undergone PTCA to develop questionnaire items based on the words used by patients to describe their experience of coronary revascularisation.
On the basis of this information, we developed a conceptual model to guide the development of the preliminary versions of the CROQ that had four core content domains (symptoms, physical functioning, psychosocial functioning, and cognitive functioning) and two additional content domains (adverse effects and satisfaction) in the post-revascularisation versions. We then generated questionnaire items for all content domains through discussions with an expert group of methodologists with expertise in health outcome assessment and questionnaire design. We borrowed some items from existing questionnaires2,7,11 and made all items specific to the patient’s heart condition.
We pre-tested preliminary versions of the CROQ by face to face interviews with 11 CABG and eight PTCA patients to evaluate content validity, clarity and appropriateness of wording, item sequence, questionnaire format, and instructions. Minor modifications were made to the pre-test questionnaires to produce field test versions of the CROQ.
Field testing
Patients
After we obtained approval from ethics committees, we recruited patients from the Royal Brompton & Harefield Trust Hospitals in London and the Wythenshawe Hospital in Manchester, UK. Patients were sent consent forms, information sheets, and questionnaires by post at two assessment points: before, and three months after coronary revascularisation. The assessment point of three months was selected because it is generally considered that by this time the majority of patients who have had CABG or PTCA will have recovered from the procedures and only a minority will still be experiencing adverse effects from the procedures.
Pre-revascularisation samples
All patients who expected to undergo isolated CABG or PTCA at the three hospitals during the study period were eligible to participate. Patients who were scheduled for elective surgery were recruited by postal survey to their home address after they were given a date for CABG or PTCA. Patients admitted to hospital by emergency were excluded from the pre-revascularisation sample.
Post-revascularisation samples
All patients who underwent isolated CABG or PTCA in the study period were sent the post-revascularisation version of the CROQ three months after revascularisation, even if they had not completed a baseline questionnaire (including patients who were given very short notice of their procedure and emergency cases). This was done to maximise the sample size for the psychometric analyses and to ensure that samples were representative of all patients undergoing CABG and PTCA. A subset of CABG and PTCA patients were sent post-revascularisation versions of the CROQ on two occasions within a two week interval to evaluate test-retest reliability.
Procedures
The psychometric evaluation of the CROQ was carried out in two independent field tests by postal survey. Standard techniques were used to ensure a high response rate, including personalised letters, standardised instructions, stamped addressed return envelopes, and follow up reminder letters.14 The purpose of the first field test was to produce a shorter version of the CROQ by eliminating items with poor measurement properties and to carry out a preliminary psychometric evaluation of item reduced versions of the questionnaires. The purpose of the second field test was to evaluate the psychometric properties of the shortened questionnaires in independent samples.
Methods for the two field tests and psychometric evaluations were identical, except that subsets of patients in the second field test were randomly assigned to receive a booklet containing only the CROQ, or the CROQ and SF-36,2 or the CROQ and Seattle angina questionnaire (SAQ)7 to evaluate construct validity. Data on angina and dyspnoea severity, measured by the Canadian Cardiovascular Society (CCS) and New York Heart Association (NYHA) classifications, respectively, were obtained from hospital records for a subsample of patients before CABG.
Statistical analyses
Psychometric methods6,15,16 were used to produce item reduced versions of the questionnaires: item-total correlations, item redundancy, missing data, maximum and aggregate adjacent endorsement frequencies, item responsiveness, and item test-retest reliability.17 The most robust items were retained. Preliminary scales were created on the basis of both the a priori conceptual model and empirical criteria (factor analysis and Cronbach’s α). Scale properties were evaluated to confirm that items in the same scale measured the same construct and that items in different scales measured different constructs.15 Items were eliminated until all pre-specified criteria were satisfied by an approach developed in our previous work.16 After confirming that the item reduced scales satisfied tests of scaling assumptions, we evaluated acceptability, reliability, validity, and responsiveness with psychometric tests and criteria (table 1).15–21 Pre- and post-revascularisation versions of the CROQ were analysed separately with the use of CABG and PTCA samples.
Table 1.
Psychometric tests and criteria*
| Psychometric property | Definition/test | Criteria for acceptability |
| Acceptability | Quality of data; assessed by completeness of data and score distributions | • Missing data for scales <10% |
| • Even distribution of endorsement frequencies across response categories; low floor/ceiling effects before revascularisation (percentage scoring lowest/highest scale score) | ||
| Reliability | ||
| Internal consistency | Extent to which items in a scale measure the same construct (such as homogeneity of the scale); assessed by Cronbach’s α18 and item-total correlations | • Cronbach’s α for scales >0.7019 |
| • Item-total correlations >0.206 | ||
| Test-retest reliability | Stability of an instrument; assessed by administering it to respondents on two occasions and examining the agreement between test and retest scores | • Intraclass correlation coefficients >0.7020 |
| Tests of scaling assumptions | Evidence that an item belongs in its own scale and not another scale (item convergent and discriminant validity) | • Scaling success/failure (item does/does not correlate significantly higher with own scale than other scales) and probable scaling success/failure (item does/does not correlate more highly, but not significantly, with own scale than other scales)15 |
| Validity | ||
| Content validity | Extent to which content of a scale is representative of the conceptual domain it is intended to cover; assessed qualitatively during questionnaire development through interviews and pretesting with patients, expert opinion, and literature review | • Evidence from interviews and pretesting with patients, expert opinion, and literature review that items are representative of impact of CABG/PTCA |
| Construct validity (within-scale analyses) | Evidence that each scale measures a single construct and that items can be combined to form summary scores; assessed on the basis of evidence of good internal consistency, factor analysis, and correlations between scale scores | • Internal consistency (Cronbach’s α >0.70) |
| • Principal axis factor analysis (factor loadings ⩾30) | ||
| • Moderate intercorrelations between scale scores | ||
| Construct validity (analyses against external criteria) | ||
| Convergent and discriminant validity | Evidence that scales are correlated with other measures of the same or similar constructs and not correlated with other measures of different constructs; assessed on the basis of correlations between CROQ, SF-36, and SAQ scores | • Magnitude and direction of correlations expected to vary according to the similarity of constructs being measured by each instrument |
| Known group differences | Evidence that scales differentiate known groups; assessed by comparing CROQ-CABG symptoms scores for patients who differ on disease severity as measured by CCS and NYHA | • CROQ scores should decrease (poorer outcome) with increasing severity of angina (CCS scores) and dyspnoea (NYHA classification) at pre-revascularisation assessment |
| Responsiveness | Ability of scales to detect clinically important change over time; assessed by comparing change in CROQ scores from before to after revascularisation (t tests and effect sizes) | • CROQ scores should show significant change from before to three months after revascularisation |
| • Effect sizes defined as small (0.20), moderate (0.50), or large (0.80 or higher)21 | ||
*Adapted from Lamping et al.16
CABG, coronary artery bypass grafting; CCS, Canadian Cardiovascular Society; CROQ, coronary outcome revascularisation questionnaire; NYHA, New York Heart Association; PTCA, percutaneous transluminal coronary angioplasty; SAQ, Seattle angina questionnaire; SF-36, short form 36.
RESULTS
First field test (item reduction)
In the first field test, 146 CABG and 128 PTCA patients completed the CROQ before revascularisation, and 289 CABG and 280 PTCA patients completed it three months after revascularisation (table 2).22
Table 2.
Respondent characteristics (pre- and post-revascularisation samples): first and second field test
| Characteristic | First field test | Second field test | ||||||
| Before revascularisation | After revascularisation | Before revascularisation | After revascularisation | |||||
| CABG (n = 146) | PTCA (n = 128) | CABG (n = 289) | PTCA (n = 280) | CABG (n = 281) | PTCA (n = 159) | CABG (n = 415) | PTCA (n = 345) | |
| Men | 108 (74%) | 86 (67%) | 216 (75%) | 192 (69%) | 238 (85%) | 120 (75%) | 343 (83%) | 251 (73%) |
| Age (years) | ||||||||
| Mean (SD) | 63.3 (8.7) | 62.1 (9.7) | 63.7 (9.0) | 62.3 (9.8) | 63.6 (9.2) | 60.6 (9.7) | 65.0 (8.9) | 62.3 (10.2) |
| Range | 34–82 | 36–88 | 35–82 | 35–88 | 35–85 | 38–89 | 37–94 | 36–84 |
| Ethnicity | ||||||||
| White | 137 (94) | 118 (92) | 267 (92) | 250 (89) | 261 (93) | 144 (91) | 369 (89) | 303 (88) |
| Asian (India/Pakistan) | 3 (2) | 7 (6) | 11 (4) | 20 (7) | 15 (5) | 5 (3) | 28 (7) | 22 (6) |
| Other | 6 (4) | 3 (2) | 11 (4) | 10 (4) | 5 (2) | 8 (5) | 12 (3) | 13 (4) |
| Social class22 | ||||||||
| I | NA | NA | NA | NA | 25 (9) | 7 (4) | 35 (8) | 5 (4) |
| II | NA | NA | NA | NA | 77 (27) | 42 (26) | 107 (26) | 31 (29) |
| III N | NA | NA | NA | NA | 30 (11) | 22 (14) | 43 (10) | 19 (19) |
| III M | NA | NA | NA | NA | 93 (33) | 52 (33) | 131 (32) | 32 (30) |
| IV | NA | NA | NA | NA | 29 (10) | 24 (15) | 48 (12) | 12 (11) |
| V | NA | NA | NA | NA | 13 (5) | 0 (0) | 20 (5) | 0 (0) |
Numbers do not sum to 100% because of missing data.
NA, not applicable.
Item reduction analyses produced shortened, final versions of the CROQ each taking about 10 minutes to complete (Appendices A, B, C, D: to view appendices go to http://www.heartjnl.com/supplemental). All four versions (CROQ-CABG_Pre, CROQ-PTCA_Pre, CROQ-CABG_Post, CROQ-PTCA_Post) contain 32 core evaluative items and one descriptive item that is not included in scale scores. The post-revascularisation versions of the CROQ (CROQ-CABG_Post 52 items, CROQ-PTCA_Post 47 items) contain these 33 core items plus additional evaluative items about adverse effects and satisfaction with outcome and two descriptive items.
Table 3 summarises items in the final versions. The CROQ is scored to produce six scale scores: symptoms (seven items), physical functioning (eight items), psychosocial functioning (14 items), cognitive functioning (three items), satisfaction (six items), and adverse effects (11 or six items). Items in each scale are summed and then transformed to a 0–100 scale by the same method as that used in the SF-36,2 with 100 representing the best possible outcome. Preliminary evaluation of the psychometric properties of the item reduced CROQ in the first field test showed that all versions satisfied the tests and criteria described in table 1 (data not presented).
Table 3.
CROQ items and scales
| Scale | Abbreviated questionnaire item content |
| Items common to pre- and post-revascularisation versions | |
| Symptoms (7 items) | • During the past four weeks, how much were you bothered by each of the following problems related to your heart condition? |
| – Chest pain due to angina | |
| – Discomfort in chest due to angina | |
| – Shortness of breath | |
| – Angina pain that radiates to other parts of body | |
| – Palpitations (strong or irregular heart beat) | |
| • During the past four weeks, on average, how many times have you taken nitros (glyceryl trinitrate tablets or spray) for your chest pain, chest tightness or angina? | |
| • During the past four weeks, how much trouble has your heart condition caused you? | |
| Physical functioning (8 items) | • The following questions ask about activities that you might do during a typical day. During the past four weeks, has your heart condition limited you in your usual daily activities? Please indicate whether your heart condition limits you a lot, limits you a little, or does not limit you at all in the activities listed below: |
| – Moderate activities | |
| – Lifting or carrying groceries | |
| – Climbing several flights of stairs | |
| – Climbing one flight of stairs | |
| – Bending, kneeling, or stooping | |
| – Walking half a mile | |
| – Walking one hundred yards | |
| – Bathing or dressing yourself | |
| Psychosocial functioning (14 items) | • The next questions ask about the impact of your heart condition on your family and friends and the extent to which it has interfered with your social activities. During the past four weeks, how often have you experienced the following as a result of your heart condition: |
| – Family or friends being overprotective | |
| – Feeling like you are a burden on others | |
| – Feeling restricted in your social activities | |
| – Feeling worried about going too far from home | |
| • The next questions ask about your feelings about your heart condition. During the past four weeks, how often have you felt: | |
| – Worried about your heart condition | |
| – Worried about doing too much or overdoing it | |
| – Worried that you might have a heart attack or die suddenly | |
| – Frightened by the pain or discomfort of your heart condition | |
| – Uncertain about the future | |
| – Depressed | |
| – Frustrated or impatient | |
| – Heart condition interfered with enjoyment of life | |
| – Difficult to keep a positive outlook about your health | |
| – Difficult to plan ahead (for vacations, social events, etc) | |
| Cognitive functioning (3 items) | • The next questions ask about problems related to your heart condition. During the past four weeks, how much of the time did you: |
| – Have difficulty reasoning and solving problems | |
| – Forget, for example, things that happened recently | |
| – Have difficulty doing activities involving concentration and thinking | |
| Not scored (1 item) | • During the past four weeks, have you had chest pain, chest tightness, or angina at rest or on exertion? |
| Additional post-revascularisation items | |
| Satisfaction (6 items) | • How satisfied are you with the: |
| – Results of your heart operation | |
| – Information about your heart operation | |
| – Information about how you might feel while recovering | |
| • Overall, how would you describe your heart condition now compared with before you had your heart operation? | |
| • Has your recovery from your heart operation so far been faster/slower than expected? | |
| • Are the results from your heart operation better/worse than expected? | |
| Adverse effects | • The next questions ask about problems you might have had since your heart operation. During the past four weeks, how much were you bothered by the following problems? |
| (CROQ-CABG, 11 items) | • CROQ-CABG_Post only |
| – Pain in chest wound | |
| – Infection in chest wound | |
| – Tenderness around chest wound | |
| – Numbness or tingling around chest wound | |
| – Bruising on chest | |
| – Pain in leg or arm wound | |
| – Any other pain in leg or arm due to operation | |
| – Infection in leg or arm wound | |
| – Numbness or tingling in leg or arm due to operation | |
| – Bruising on leg or arm where a vein was removed | |
| – Swollen feet or ankles | |
| (CROQ-PTCA, 6 items) | • CROQ-PTCA_Post only |
| – Pain in groin wound | |
| – Tenderness around groin wound | |
| – Numbness or tingling in groin area | |
| – Bruising around groin wound or thigh | |
| – Problems in groin where the catheter was inserted | |
| – Concern over the appearance of bruises | |
| Not scored (2 items) | • During the past four weeks, how often have you felt worried your symptoms might return? |
| • Since your heart operation, have you been readmitted to hospital for an overnight stay for any reason to do with your heart condition or heart operation? | |
Second field test (evaluation of psychometric properties)
This section describes results from the final psychometric evaluation of all four versions. Because of space constraints, it focuses mainly on the three month post-revascularisation versions. The post-revascularisation versions of the CROQ can also be used in the longer term—for example, one year after revascularisation. A full psychometric validation of data at nine months after revascularisation showed that the measurement properties of the instrument are retained (data not presented).
Patients
In the second field test 696 of 916 CABG and 504 of 734 PTCA patients responded (table 2). The CABG sample consisted of 281 of 407 patients before revascularisation (mean (SD) age 63.6 (9.2) years, 85% men) and 415 of 509 after revascularisation (65.0 (8.9) years, 83% men; 76% response rate). The PTCA sample comprised 159 of 270 patients before revascularisation (mean (SD) age 60.6 (9.7) years, 75% men) and 345 of 464 after revascularisation (62.3 (10.2) years, 73% men; 69% response rate). Subsamples of 198 CABG and 107 PTCA patients completed pre- and post-revascularisation questionnaires (responsiveness samples), and 50 CABG and 48 PTCA patients formed the test-retest reliability samples.
Psychometric evaluation
Acceptability
All versions had good acceptability (table 4) with a low proportion of missing data. Although the CROQ had some ceiling effects, this was expected for post-revascularisation scores, as high scores reflect patients’ return to optimal functioning after an effective clinical intervention.
Table 4.
Acceptability and reliability of the CROQ (post-revascularisation)
| CROQ scale | Score (range 0–100) mean (SD) | Acceptability | Reliability | ||
| Missing data | Floor/ceiling effects* | Internal consistency (Cronbach’s α) | Test-retest (ICC) | ||
| CROQ-CABG_Post (n = 415) | |||||
| Symptoms (7 items) | 87.57 (14.9) | 1% | 0%/21% | 0.85 | 0.90 |
| Physical functioning (8 items) | 80.27 (22.7) | 2% | 1%/28% | 0.90 | 0.93 |
| Psychosocial functioning (14 items) | 78.14 (21.0) | 1% | 0%/5% | 0.95 | 0.92 |
| Cognitive functioning (3 items) | 78.27 (22.6) | 1% | 1%/28% | 0.89 | 0.80 |
| Adverse effects (11 items) | 80.36 (16.9) | 1% | 0%/4% | 0.84 | 0.83 |
| Satisfaction (6 items) | 83.12 (18.0) | 1% | 0%/29% | 0.81 | 0.90 |
| CROQ-PTCA_Post (N = 345) | |||||
| Symptoms (7 items) | 77.02 (22.1) | 1% | 1%/13% | 0.91 | 0.84 |
| Physical functioning (8 items) | 71.22 (28.1) | 5% | 1%/24% | 0.93 | 0.91 |
| Psychosocial functioning (14 items) | 69.24 (24.9) | 4% | 0%/7% | 0.96 | 0.93 |
| Cognitive functioning (3 items) | 75.91 (27.6) | 3% | 2%/30% | 0.92 | 0.86 |
| Adverse effects (6 items) | 93.54 (14.1) | 4% | 1%/62% | 0.87 | 0.86 |
| Satisfaction (6 items) | 76.77 (22.0) | 1% | 1%/21% | 0.83 | 0.91 |
*Lowest/highest scale scores.
ICC, intraclass correlation coefficient.
Reliability (internal consistency and test-retest)
Cronbach’s α coefficients18 for all scales exceeded the criterion of 0.7019 (table 4). Item-total correlations within scales were similar and exceeded the criterion of 0.20,6 indicating that each item was contributing equally to the scale. Intraclass correlation coefficients exceeded the criterion of 0.7020 for all scales, indicating good test-retest reliability.
Tests of scaling assumptions
Item convergent and discriminant validity correlations15 confirmed the scale structure. The majority of items were scaling successes, a few were probable scaling successes, very few were probable scaling failures, and none were definite scaling failures.
Content validity
Content validity was evaluated during the development of the CROQ. Evidence from interviews and pretesting with patients, expert opinion, and a review of the literature supports the content validity of the CROQ.
Construct validity (within-scale analyses)
Evidence of high internal consistency supports the construct validity of the CROQ. High α coefficients (table 4) and moderately high item-total correlations for scales indicate that a single construct is being measured and that the items can be combined to form scales. Results of principal axis factor analysis and intercorrelations between CROQ scores provide further support for construct validity (data not presented).
Construct validity (analysis against external criteria)
Correlations between the CROQ and the SF-36 and SAQ scores provide further support for construct validity (table 5). Evidence for convergent validity is shown by moderate to high correlations between similar domains of the three measures and, for discriminant validity, low correlations between domains measuring different constructs on the CROQ, SF-36, and SAQ. For example, correlations between the CROQ and the SF-36 physical component summary score (PCS) were higher for symptoms and physical functioning than for psychosocial functioning and cognitive functioning. CROQ symptom and satisfaction scores were highly correlated with SAQ anginal frequency and satisfaction scores, respectively.
Table 5.
Construct validity of the CROQ (post-revascularisation): correlations with other measures
| CROQ scale | SF-36 | SAQ | |||
| PCS | MCS | Exertional capacity | Anginal frequency | Treatment satisfaction | |
| CROQ-CABG_Post | |||||
| Symptoms | 0.60* | 0.36 | 0.59 | 0.74* | 0.62 |
| Physical functioning | 0.75* | 0.36 | 0.67* | 0.47 | 0.35 |
| Psychosocial functioning | 0.59 | 0.64* | 0.76 | 0.52 | 0.55 |
| Cognitive functioning | 0.44 | 0.46* | 0.73 | 0.40 | 0.41 |
| Adverse effects | 0.51 | 0.46 | 0.44 | 0.51 | 0.52 |
| Satisfaction | 0.51 | 0.37 | 0.45 | 0.46 | 0.65* |
| CROQ-PTCA_Post | |||||
| Symptoms | 0.68* | 0.32 | 0.69 | 0.86* | 0.70 |
| Physical functioning | 0.75* | 0.37 | 0.90* | 0.70 | 0.56 |
| Psychosocial functioning | 0.49 | 0.73* | 0.77 | 0.62 | 0.58 |
| Cognitive functioning | 0.36 | 0.49* | 0.65 | 0.50 | 0.38 |
| Adverse effects | 0.25 | 0.21 | 0.44 | 0.45 | 0.35 |
| Satisfaction | 0.29 | 0.38 | 0.53 | 0.59 | 0.72* |
*Correlations between scales that purport to measure similar aspects of health related quality of life.
MCS, mental component summary score; PCS, physical component summary score.
The ability of the CROQ to differentiate between known groups is supported by results from analyses of CROQ-CABG pre-revascularisation scores. CROQ-CABG_Pre symptoms scores showed the expected gradient according to CCS angina severity (mean (SD) score CCS class I 80.1 (19.4), n = 3; CCS II 47.9 (22.2), n = 35; CCS III 42.1 (22.5), n = 53; CCS IV 34.0 (15.4), n = 16; one way analysis of variance p = 0.005) and NYHA dyspnoea severity (NYHA I 56.7 (21.2), n = 17; NYHA II 40.6 (21.4), n = 39; NYHA III 39.2 (22.1), n = 52; NYHA IV 28.9 (22.3), n = 16; one way analysis of variance p = 0.033).
Responsiveness
Table 6 presents mean CROQ scores for subsamples assessed before and after revascularisation. There was significant change in all scales in the CABG and PTCA samples (p < 0.05). There were large effect sizes21 in three of the four scales (symptoms, physical functioning, and psychosocial functioning) in the CABG sample and one scale (symptoms) in the PTCA sample. Effect sizes were moderate for cognitive functioning in the CABG sample and for physical functioning and psychosocial functioning scales in the PTCA sample.
Table 6.
Responsiveness of the CROQ (before to three months after revascularisation)
| CROQ scale | Mean (SD) score | Effect size† | ||
| Before | At 3 months | Change* | ||
| CROQ-CABG (n = 198)‡ | ||||
| Symptoms | 48.98 (24.2) | 88.29 (13.9) | 39.31 (25.3) | 2.83 |
| Physical functioning | 50.48 (26.9) | 82.46 (21.8) | 31.98 (29.4) | 1.47 |
| Psychosocial functioning | 49.59 (24.3) | 79.65 (19.7) | 30.05 (23.1) | 1.53 |
| Cognitive functioning | 62.57 (29.2) | 77.94 (22.8) | 15.36 (25.7) | 0.67 |
| CROQ-PTCA (n = 107)‡ | ||||
| Symptoms | 51.98 (23.4) | 75.07 (21.9) | 23.10 (25.2) | 0.99 |
| Physical functioning | 53.39 (27.2) | 71.42 (26.0) | 18.03 (28.3) | 0.66 |
| Psychosocial functioning | 54.32 (25.1) | 71.06 (24.3) | 16.74 (21.5) | 0.67 |
| Cognitive functioning | 68.46 (29.5) | 75.45 (25.7) | 6.99 (23.5) | 0.24 |
*All change scores are significant (p<0.05).
†Calculated as mean change between pre- and three months post-revascularisation scores divided by the standard deviation of scores before revascularisation.
‡Subsample of patients who completed the CROQ both before and after coronary revascularisation.
Note: It is not appropriate to compare CROQ scores for CABG and PTCA groups in this table, as this dataset has not been adjusted in terms of case mix severity, etc.
Although the intended use of the CROQ is to compare CABG and PTCA procedures, it is not appropriate to make these comparisons with the data presented in this paper. The data presented here were collected for psychometric testing purposes only and have not been adjusted for case mix severity, age, sex, social status, or ethnicity.
DISCUSSION
The CROQ is a practical and scientifically validated patient based measure of outcome for coronary revascularisation that is acceptable to patients and satisfies rigorous psychometric criteria for reliability, validity, and responsiveness. As the only validated instrument developed specifically for use before and after CABG and PTCA, and which is quick and easy to administer, the CROQ provides a rigorous method for improving the evaluation of outcomes in clinical trials and clinical audit. We achieved high response rates to our postal surveys, suggesting that self administration by post is a convenient and feasible method of collecting outcome data.
In the absence of a more appropriate instrument, the SAQ has been widely used in assessing outcomes after CABG and PTCA. The CROQ provides more appropriate content, as it contains items directly addressing the impact of these procedures based on problems that patients reported to be important. The SAQ was originally developed for use with patients given medical treatment and thus has a more limited focus on angina and satisfaction with treatment. The evaluation of construct validity (table 5) showed that the CROQ is on a par with the SAQ in measuring symptoms and satisfaction, but has the advantage of also measuring psychosocial functioning, cognitive functioning, and adverse effects, with little additional patient burden.
Although generic measures such as the SF-36 have been used to measure change in health status after revascularisation, it is widely acknowledged that disease specific measures are more responsive to treatment effects. Given its demonstrated high responsiveness, the CROQ is a promising new tool for use in clinical trials. It may detect important differences between procedures that have previously not been detected by less sensitive generic measures.
This paper reports the development and initial validation of the CROQ in a British sample. Validation of measures is an iterative process. Future research should be undertaken to confirm the psychometric properties of the CROQ and generalisability of findings to other English speaking patient populations, and population norms need to be generated. The CROQ has recently been used in two clinical trials.23,24 Data from these trials will provide information about the relation between the CROQ and clinical variables and the ability of the CROQ to predict clinical outcomes. The availability of different language versions will enable international comparisons of patient based outcomes in clinical trials; an Italian version is being validated.25
With increasing resource allocation to coronary revascularisation, it is essential to have scientifically robust tools with appropriate content to evaluate effectiveness. Our conclusions about the strong psychometric properties of the CROQ are based on the results of rigorous two stage field testing in large samples of patients. The CROQ exceeded criteria for all psychometric tests.
Supplementary Material
Acknowledgments
We thank the NHS Executive/North Thames Research and Development Programme for funding this research; the patients who took part in the study; the cardiologists, cardiac surgeons, and nurses at participating hospitals; B Bridgewater (Wythenshawe Hospital); and D Denison, N Black, A Fletcher, and B Reeves for their contributions as members of S Schroter’s PhD Committee.
Abbreviations
CABG, coronary artery bypass grafting
CCS, Canadian Cardiovascular Society
CROQ, coronary revascularisation outcome questionnaire
NYHA, New York Heart Association
PTCA, percutaneous transluminal coronary angioplasty
SAQ, Seattle angina questionnaire
SF-36, short form 36
Funded by: The NHS Executive/North Thames Research and Development Programme (through S Schroter’s PhD fellowship).
Portions of this paper were presented at the 5th, 7th, and 8th annual meetings of the International Society for Quality of Life Research (1998–2001).
Institutions where work was done: London School of Hygiene & Tropical Medicine, Royal Brompton & Harefield Trust Hospitals, and Wythenshawe Hospital.
Copies of the CROQ and the SPSS program for scoring the CROQ can be obtained from Dr S Schroter.
REFERENCES
- 1.Guyatt GH, Thompson PJ, Berman LB, et al. How should we measure function in patients with chronic heart and lung disease? J Chronic Dis 1985;38:517–24. [DOI] [PubMed] [Google Scholar]
- 2.Ware JE, Snow KK, Kosinski M, et al. SF-36 health survey: manual and interpretation guide. Boston: The Health Institute, New England Medical Centre, 1993.
- 3.Pocock SJ, Henderson RA, Seed P, et al. Quality of life, employment status, and anginal symptoms after coronary angioplasty or bypass surgery: 3-year follow-up in the randomized intervention treatment of angina (RITA) Trial. Circulation 1996;94:135–42. [DOI] [PubMed] [Google Scholar]
- 4.Pocock SJ, Henderson RA, Clayton T, et al. Quality of life after coronary angioplasty or continued medical treatment for angina: three-year follow-up in the RITA-2 trial (randomised intervention treatment of angina). J Am Coll Cardiol 2000;35:907–14. [DOI] [PubMed] [Google Scholar]
- 5.Patrick DL, Deyo RA. Generic and disease-specific measures in assessing health status and quality of life. Med Care 1989;27 (suppl 3) :217–32. [DOI] [PubMed] [Google Scholar]
- 6.Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. 2nd ed. Oxford: Oxford University Press, 1995.
- 7.Spertus JA, Winder JA, Dewhurst TA, et al. Development and evaluation of the Seattle angina questionnaire: a new functional status measure for coronary artery disease. J Am Coll Cardiol 1995;25:333–41. [DOI] [PubMed] [Google Scholar]
- 8.Wilson A, Wiklund I, Lahti T, et al. A summary index for the assessment of quality of life in angina pectoris. J Clin Epidemiol 1991;44:981–8. [DOI] [PubMed] [Google Scholar]
- 9.Wiklund I, Comerford MB, Dimenas E. The relationship between exercise tolerance and quality of life in angina pectoris. Clin Cardiol 1991;14:204–8. [DOI] [PubMed] [Google Scholar]
- 10.Marquis P, Fayol C, Joire JE. Clinical validation of a quality of life questionnaire in angina pectoris patients. Eur Heart J 1995;16:1554–60. [DOI] [PubMed] [Google Scholar]
- 11.Valenti L, Lim L, Heller RF, et al. An improved questionnaire for assessing quality of life after acute myocardial infarction. Qual Life Res 1996;5:151–61. [DOI] [PubMed] [Google Scholar]
- 12.Goodman H. Patients’ perceptions of their education needs in the first six weeks following discharge after cardiac surgery. J Adv Nurs 1997;25:1241–51. [DOI] [PubMed] [Google Scholar]
- 13.Wilson-Barnet J. Assessment of recovery: with special reference to a study with post-operative cardiac patients. J Adv Nurs 1981;6:435–45. [DOI] [PubMed] [Google Scholar]
- 14.Dillman DA. Mail and telephone surveys: the total design method. New York: Wiley, 1978.
- 15.Ware JE, Harris WJ, Gandek B, et al. MAP-R for Windows: multitrait/multi-item analysis program: revised user’s guide. version 1. Boston: Health Assessment Lab, 1997.
- 16.Lamping DL, Schroter S, Marquis P, et al. The community-acquired pneumonia symptom questionnaire: a new, patient-based outcome measure to evaluate symptoms in patients with community-acquired pneumonia. Chest 2002;122:920–9. [DOI] [PubMed] [Google Scholar]
- 17.Schroter S, Lamping DL. Psychometric evaluation of the coronary revascularisation outcome questionnaire (CROQ-CABG/PTCA) [abstract]. Qual Life Res 2000;9:311. [Google Scholar]
- 18.Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika 1951;16:297–334. [Google Scholar]
- 19.Nunnally JC, Bernstein IH. Psychometric theory. 3rd ed. New York: McGraw-Hill, 1994.
- 20.Anon. Assessing health status and quality-of-life instruments: attributes and review criteria. Scientific Advisory Committee of the Medical Outcomes Trust. Qual Life Res 2002;11:193–205. [DOI] [PubMed] [Google Scholar]
- 21.Cohen J. Statistical power analysis for the behavioural sciences. rev edn. New York: Academic Press, 1977.
- 22.Anon. Standard occupational classification, vol 3. Social classifications and coding methodology. Office of Population Censuses and Surveys. London: HMSO, 1991.
- 23.Reeves BC, Angelini GD, Bryan AJ, et al. A multi-centre randomised controlled trial of minimally invasive direct coronary bypass grafting versus percutaneous transluminal coronary angioplasty with stenting for proximal stenosis of the left anterior descending coronary artery. Health Technol Assess 2004;8:1–43. [DOI] [PubMed] [Google Scholar]
- 24.Ascione R, Reeves BC, Taylor FC, et al. Beating heart against cardioplegic arrest studies (BHACAS 1 and 2): quality of life at mid-term follow-up in two randomised controlled trials. Eur Heart J 2004;25:765–70. [DOI] [PubMed] [Google Scholar]
- 25.Pinna PP, Schroter S, Dilaghi R, et al. Translation and adaptation of the coronary revascularisation outcome questionnaire (CROQ) into Italian [abstract]. Qual Life Res 2000;9:312. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.