Abstract
Objectives
To examine the frequency and quality of reporting on quality of life in randomised controlled trials.
Design
Search of the Cochrane Controlled Trials Register 1980 to 1997 to identify trials from all disciplines, from oncology, and from cardiovascular medicine that reported on quality of life. Assessment of abstracts from articles published from 1993 to 1996. Assessment of a sample of full reports with a standardised instrument.
Main outcome measures
Prevalence of reporting on quality of life. Conditions and interventions studied in trials reporting on quality of life. Quality of reporting on quality of life.
Results
During 1980-97 reporting on quality of life increased from 0.63% to 4.2% for trials from all disciplines, from 1.5% to 8.2% for cancer trials, and from 0.34% to 3.6% for cardiovascular trials. Of 364 abstracts, 65% reported on drug interventions. Of a sample of 67 full reports, authors of 48 (72%) used 62 established quality of life instruments. In 15 reports (22%) authors developed their own measures, and in 2 (3%) methods were unclear. Response rates were given in 38 (57%), and complete reporting on all items and scales occurred in 31 (46%).
Conclusions
Less than 5% of all randomised controlled trials reported on quality of life, and this proportion was below 10% even for cancer trials. A plethora of instruments was used in different studies, and the reporting of methods and results was often inadequate. Standards for the measurement and reporting of quality of life in clinical trials research need to be developed.
Key messages
We examined the reporting on quality of life in randomised controlled trials listed in the Cochrane Controlled Trials Register
Although reporting on quality of life increased over time, fewer than 5% of trials overall and fewer than 10% of cancer trials included quality of life in 1997
Among 67 articles selected at random for detailed examination, a wide range of established and self developed measures of quality of life were used
Only about half of trials gave response rates, and less than half reported on all items and scales used
Standards for assessing and reporting quality of life in clinical research trials need to be developed
Introduction
Quality of life has become an accepted end point in clinical research trials in recent years, as interest in patients’ experiences and preferences has grown.1 Quality of life is of particular importance in trials comparing treatments with similar or no impact on disease progression and survival. However, the term quality of life is often used vaguely and without clear definition.2–4 This is not surprising, considering the broad nature of a concept that includes physical functioning (ability to carry out activities of daily living such as self care and walking around), psychological functioning (emotional and mental wellbeing), social functioning (relationships with others and participation in social activities), and perception of health status, pain, and overall satisfaction with life.5
The lack of a clear definition of quality of life is reflected in the many instruments that have been proposed to measure it. Generic measures (such as the sickness impact profile6 or the short form health survey SF-367) broadly assess physical, mental, and social health and can be used to compare conditions and treatments. Measures specific to illnesses can supplement generic measures or can be used independently.8 Other methods include measures focusing on a single aspect such as pain or anxiety and individualised measures, in which patients themselves define and rate the most important aspects of their quality of life.9
Inadequate reporting of randomised controlled trials is common and hampers the appraisal of the validity and generalisability of results.10–13 In an attempt to remedy this, consolidated standards of reporting trials (CONSORT) have been developed.14 To our knowledge, however, the quality of reporting on quality of life outcomes in trials has not been systematically assessed so far. In addition, it is unclear as to what extent the growth in attention to quality of life is reflected in an increasing number of reports from clinical trials. We addressed these questions in a bibliographic study based on the Cochrane Controlled Trials Register.
Methods
Prevalence of reporting on quality of life
We searched the Cochrane Controlled Trials Register on The Cochrane Library (issue 3, 1998)15 to identify randomised controlled trials published in any language between 1980 and 1997 that reported on quality of life. We entered the keyword “random*” to identify randomised controlled trials, the wild card (*) ensuring that all abstracts containing words with the stem “random” (such as randomly, randomised, randomisation) were included. We identified trials that reported on quality of life by combining MeSH and free text terms “quality of life.” For each year, we calculated the proportion of reports mentioning quality of life. We repeated the analysis for cancer trials, using the search strategy of the Cochrane Cancer Network for this purpose. We also examined cardiovascular trials: we exploded all subheadings of the MeSH term “cardiovascular-diseases” and combined the results with free text searches using a total of 28 key words (details of the search strategy are available on the BMJ website).
Assessment of abstracts
We examined the abstracts of reports published from 1993 to 1996. We had identified these in a search of an earlier issue of the Cochrane Controlled Trials Register (issue 4, 1997) using the same keywords. Abstracts from articles that reported the results of randomised controlled trials and included at least one outcome that we considered to be related to quality of life were assessed by CS and checked by JD. The following information was recorded: language of article, condition studied, and type of intervention.
Assessment of full reports
A sample of reports from the database of abstracts was assessed in more detail. Using a random number generator, we selected 20% of the trials reported in English. CS and DT independently examined the full reports of these using a standardised data sheet. The data sheet covered the following items: conditions and interventions studied, primary and secondary end points, sample sizes, type and documentation of the quality of life measures used, response rates, and the presentation and direction of quality of life results. If not explicitly stated by the authors, the primary end point was defined as the one that was given prominence in the report. The results relating to quality of life and other end points were also extracted. Finally, CS and DT rated trial quality using the score described by Jadad et al.16 The rate of agreement between the two observers was generally high (median 80%, range 58-100%). Discrepancies were resolved in discussions with ME.
Results
Prevalence of reporting on quality of life
The keyword “random*” identified 92 364 records, which shrank to 1939 (2.1%) when “quality of life” was added. There were 10 896 cancer trials, of which 443 (4.1%) reported on quality of life, and 18 398 cardiovascular trials, of which 393 (2.1%) reported on quality of life. The proportion of trials reporting on quality of life increased over time, from 0.63% in 1980 to 4.2% in 1997 for all trials, from 1.5% to 8.2% for cancer trials, and from 0.34% to 3.6% for cardiovascular trials (figure).
Assessment of abstracts
We found 492 reports that were published during 1993-6, of which we excluded 128 (26%) for not being a randomised controlled trial (n=59), not reporting on quality of life (n=12), duplicate entry (n=27), and not being a journal publication (n=30). We examined the abstracts of the remaining 364 reports: the most commonly studied conditions were cancer (29%) and cardiovascular disease (26%), with other diseases occurring in less than 10% of studies (table 1). The most common interventions were drugs (65%) and models of care (11%) (table 2). Most of the reports (93%) were published in English language journals.
Table 1.
Subjects studied in 364 randomised controlled trials reporting on quality of life
| Subject | No (%) of trials |
|---|---|
| Cancer | 107 (29) |
| Cardiovascular diseases | 96 (26) |
| Respiratory | 25 (7) |
| Gastrointestinal diseases | 22 (6) |
| Mental health | 17 (5) |
| Infection | 12 (3) |
| Urology | 11 (3) |
| Gynaecology | 11 (3) |
| Endocrinology | 10 (3) |
| Renal | 7 (2) |
| Rheumatology | 6 (2) |
| Neurological diseases | 6 (2) |
| Elderly patients | 5 (1) |
| Orthopaedics | 4 (1) |
| Other | 25 (7) |
Table 2.
Interventions studied in 364 randomised controlled trials reporting on quality of life
| Intervention | No (%) of trials |
|---|---|
| Drug | 238 (65) |
| Model of care | 39 (11) |
| Surgery | 30 (8) |
| Psychological | 11 (3) |
| Radiotherapy | 7 (2) |
| Nutrition | 6 (2) |
| Exercise | 4 (1) |
| Other | 29 (8) |
Assessment of full reports
We selected 67 English language reports at random and examined them in detail (references are available on the BMJ’s website). They were published in 52 journals. Ten journals published more than one report, and three (Circulation, Lancet, and New England Journal of Medicine) published more than two reports. The distribution of conditions and interventions studied was similar to that in the database of abstracts, with 30% of reports describing cancer trials, 25% describing cardiovascular conditions, and 64% reporting on drug interventions.
Instruments for assessing quality of life
—Forty eight of the trials used at least one of the 62 established instruments shown in the box, and most (54) quoted a reference that described the methodology. In 15 trials the authors used instruments or indicators that they had developed for their study. Most of these were questionnaires, and many used visual analogue scales. In a few cases authors constructed composite indices. One of these indices, for example, was described as “the sum of asthenia + unaided dressing + visits to the shops, etc + outside activities + number of consecutive days spent without leaving home + number of days with a body temperature of 38.5°C or less.”17 In two studies it was unclear how quality of life was measured.
Quality of life instruments used in 48 randomised controlled trials reporting on quality of life
| Generic | Psychological | Condition specific |
| 1 Nottingham health profile (NHP) (6) | 1 Psychological general well being index (PGWBI) (6) | 1 EORTC QLQ C-30 (4) |
| 2 Quality of wellbeing index (QWBI) (3) | 2 Lehman QOL interview (2) | 2 Spitzer QOL index (4) |
| 3 Karnosky performance status (KPS) (2) | 3 State trait anxiety inventory (STAI) (2) | 3 Functional living with cancer index (FLIC) (3) |
| 4 Activities of daily living questionnaire (2) | 4 Mental health index (2) | 4 Minnesota living with heart failure questionnaire (2) |
| 5 Short form 36 (SF-36) (2) | 5 Beck depression inventory (BDI) (2) | 5 Chronic respiratory disease index questionnaire (2) |
| 6 Sickness impact profile (SIP) (2) | 6 Hospital anxiety and depression scale (HADS) (2) | 6 Angina pectoris QOLQ |
| 7 QOLQ adapted from the Dartmouth COOP chart | 7 Bf-S wellbeing scale | 7 Heart condition assessment |
| 8 Functional status questionnaire | 8 Connecticut department of mental health QOL survey | 8 New York Heart Association functional scale (NYHA) |
| 9 LASA general wellbeing scale | 9 Psychological adjustment to illness scale (PAIS) | 9 Chronic heart failure questionnaire |
| 10 Social network questionnaire | 10 Hamilton anxiety scale (HAS) | 10 Dyspnoea fatigue rating scale |
| 11 Functional activities questionnaire (FAQ) | 11 Social adjustment scale (SAS) | 11 Rhinoconjunctivitis QOLQ |
| 12 EuroQol | 12 Center for epidemiologic studies depression scale (CES-D) | 12 Scott-Huskisson VAS for pain |
| 13 Self perceived health status scale (SPHS) | 13 Montgomery and Asberg depression scale (MADRS) | 13 Skin problem list |
| 14 General health questionnaire GHQ-30 | 14 Body cathexis list | 14 The side effects of symptom distress index |
| 15 Sheehan disability scale (SDS) | 15 Self esteem scale | 15 The sexual symptom distress scale |
| 16 McMaster health index | 16 Inventory list on association with others | 16 Sleep dysfunction scale |
| 17 Revised symptom checklist (SCL90-R) | 17 The Petty questionnaire | |
| 18 Goteborg QOL instrument (GQL) | 18 Subjective chemo IQ (SCI) | |
| 19 Short comprehensive assessment and referral evaluation (short-CARE) | 19 South West Oncology Group (SWOG) questionnaire | |
| 20 The wellbeing at work or daily routine scale | 20 Rotterdam symptom checklist | |
| 21 The life events index | 21 Morrow assessment of nausea and emesis (MANE) | |
| 22 Physical self maintenance scale (PSMS) | 22 The linear analogue self assessment scale | |
| 23 Modified hamburger visual analogue scales | ||
| 24 Perceived QOL scale (PQOLS) | ||
No of trials given in parentheses if instrument was used in more than one trial.
Principal end points
—Quality of life was the principal end point in 23 reports. Other end points were symptoms (n=12), disease progression or survival (n=8), exercise capacity (n=5), blood pressure (n=3), pulmonary function (n=2), and 14 other end points (one study each).
Quality of reporting on quality of life
—The response rate for quality of life end points was given in 38 of the studies, with response rates ranging from 51% to 100%. In 46 reports the authors clearly stated that the patients provided information on quality of life, whereas in the remaining articles it was unclear to what extent the information originated from patients, carers, or relatives. Complete reporting of all items and scales occurred in 31 trials. Probability values were presented in 52, but only 11 provided confidence intervals. The mean Jadad quality score was 2.68, corresponding to 54% of the maximum score of 5.
Quality of reporting on quality of life was higher in the trials with quality of life as the principal end point (table 3). These trials were significantly more likely to have used an established instrument, to state respondents, to give response rates, to report on all items and scales, and to present absolute differences. Jadad scores tended to be lower for these trials, although this did not reach significance (P=0.15).
Overall, 39 articles reported significantly (P<0.05) more favourable outcomes in terms of quality of life for one or more of the experimental groups. The proportion that did so was slightly higher among the trials with quality of life as the principal end point (16 (70%) v 23 (52%), P=0.17). Sample sizes were larger in studies with quality of life as the principal end point (table 3).
Discussion
Quality of life is widely accepted as an important end point in clinical trials.1,3,5 The prevalence and quality of reporting on quality of life in clinical research is unclear, however. We examined trends in the reporting of quality of life data in randomised controlled trials during 1980-97 and assessed the quality of such reporting in more detail for 1993-6. Based on the Cochrane Controlled Trials Register, the most comprehensive source of trials available, our results indicate that the prevalence of reporting on quality of life remains trivial. Overall, the proportion of trials reporting on quality of life increased from less than 1% in 1980 to about 4% in 1997. A higher proportion might be expected for cancer considering the nature of the disease and the fact that a number of key funding bodies recently introduced policies to promote the assessment of quality of life in randomised controlled trials.18 However, even among these the proportion reporting on quality of life was less than 10% in 1997. Furthermore, these low levels of reporting of quality of life may be an overestimate as almost a quarter of papers retrieved in our abstract search were considered to be irrelevant.
One reason why relatively few trialists embark on measuring quality of life may be because of methodological difficulties. Of the 67 studies sampled for detailed examination, 48 used 62 different pre-existing instruments and a further 15 studies reported new measures, with few following the methods proposed for the development and testing of instruments.8,19 The need for both generic and condition specific instruments means that a range of measures is required, but it is implausible that some 40 different measures of generic or psychological wellbeing could be justified in 48 trials. Critical analysis of these instruments was beyond the scope of our study, but the dimensions of quality of life assessed as well as the levels of validity and reliability of the instruments varied considerably. Such variations will obstruct comparisons between studies.
Quality of reporting on quality of life
Gill and Feinstein examined 75 articles with “quality of life” in their titles.2 Selection was independent of study design, and most studies were published during 1987-91. They focused on the investigators’ definition of quality of life, their reasons for choosing a given instrument, and the importance given to the patients’ own rating of quality of life. Gill and Feinstein showed that only 15% of investigators defined quality of life, only 36% gave reasons for their choice of instrument, and only 13% invited patients to contribute personal assessments. We focused on reporting of quality of life outcomes and found that similar problems apply in clinical trials. For example, not only were patients generally not asked to supplement the questionnaire based data with personal responses, in about 30% of studies it remained unclear whether patients had contributed any information at all. Clearly, some investigators continue to believe that health professionals can make a valid assessment of their patients’ quality of life, in spite of evidence to the contrary.20
Response rates are critical for reporting quality of life, as differential non-response can introduce serious bias. The fact that response rates were unreported in almost a half of the trials must be a matter of concern. Our results indicate that selective reporting of items and scales may also be a problem. We could be assured that investigators had reported all the information collected in less than half of the articles. Selective reporting of favourable or statistically significant results may thus have occurred in a number of articles.
We used the Jadad score16 to examine the quality of reporting of aspects of trial methodology that have been shown to be related to the validity of randomised controlled trials.21 With a mean of 54% of the maximum quality score, our sample of randomised controlled trials is of similarly low quality when compared with randomised controlled trials published in leading English language journals or trials published in languages other than English.11,22 Interestingly, the Jadad score tended to be higher among the reports with quality of life as a secondary end point. Reports focusing on quality of life were more likely to use established instruments, to make clear whether the patient’s view had been sought, and to offer complete reporting of items and scores. There may, however, have been some disregard for the conventions of trial design and reporting among those primarily interested in quality of life.
Conclusions
Our comprehensive analysis of the literature shows that, although it is increasing, the reporting on quality of life end points remains uncommon in randomised controlled trials, and the quality of reporting is often poor. An initiative similar to CONSORT14 that involves trialists, specialists in the measurement of quality of life, and journal editors is required to develop standards of assessing and reporting quality of life in clinical trials. Decades after the invention of the randomised controlled trial, such an initiative may finally lead to trials assessing end points that really matter to patients.
Figure.
Prevalence of reporting on quality of life in randomised controlled trials identified from Cochrane Controlled Trials Register during 1980-97
Table 3.
Indicators of quality of reporting on quality of life in 67 randomised controlled trials. Values are numbers (%) of trials unless stated otherwise
Acknowledgments
The department of social medicine at University of Bristol is part of the MRC Health Services Research Collaboration.
Footnotes
Funding: This work was partially funded by the University of Bristol.
Competing interests: None declared.
References
- 1.Patrick DL, Bergner M. Measurement of health status in the 1990s. Ann Rev Public Health. 1990;11:165–183. doi: 10.1146/annurev.pu.11.050190.001121. [DOI] [PubMed] [Google Scholar]
- 2.Gill TM, Feinstein AR. A critical appraisal of the quality of quality-of-life measurements. JAMA. 1994;272:619–626. [PubMed] [Google Scholar]
- 3.Guyatt G, Feeny D, Patrick D. Issues in quality of life measurement in clinical trials. Control Clin Trials. 1991;12:81–90S. doi: 10.1016/s0197-2456(05)80014-5. [DOI] [PubMed] [Google Scholar]
- 4.Fallowfield L. Quality of quality of life data. Lancet. 1996;348:421. doi: 10.1016/S0140-6736(05)64534-5. [DOI] [PubMed] [Google Scholar]
- 5.Naughton MJ, Shumaker SA. Assessment of health-related quality of life. In: Furberg CD, DeMets DL, editors. Fundamentals of clinical trials. 3rd ed. St Louis: Mosby Press; 1996. [Google Scholar]
- 6.Bergner M, Bobbitt RA, Carter WB, Gilson BS. The sickness impact profile: development and final revision of a health status measure. Med Care. 1981;19:787–805. doi: 10.1097/00005650-198108000-00001. [DOI] [PubMed] [Google Scholar]
- 7.Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). Conceptual framework and item selection. Med Care. 1992;30:473–483. [PubMed] [Google Scholar]
- 8.Guyatt GH, Bombardier C, Tugwell PX. Measuring disease-specific quality of life in clinical trials. Can Med Assoc J. 1986;134:889–895. [PMC free article] [PubMed] [Google Scholar]
- 9.McGee H, Hannah M, O’Boyle CA, Hickey A, O’Malley K, Joyce CRB. Assessing the quality of life of the individuals: the SEIQOL with a healthy and a gastroenterology unit population. Psychol Med. 1991;21:749–759. doi: 10.1017/s0033291700022388. [DOI] [PubMed] [Google Scholar]
- 10.Moher D, Jadad AR, Tugwell P. Assessing the quality of randomized controlled trials. Current issues and future directions. Int J Technol Assess Health Care. 1996;12:195–208. doi: 10.1017/s0266462300009570. [DOI] [PubMed] [Google Scholar]
- 11.Moher D, Fortin P, Jadad AR, Jüni P, Klassen T, Le Lorier J, et al. Completeness of reporting of trials published in languages other than English: implications for conduct and reporting of systematic reviews. Lancet. 1996;347:363–366. doi: 10.1016/s0140-6736(96)90538-3. [DOI] [PubMed] [Google Scholar]
- 12.McNamee D, Horton R. Lies, damn lies, and reports of RCTs. Lancet. 1996;348:562. doi: 10.1016/s0140-6736(05)64795-2. [DOI] [PubMed] [Google Scholar]
- 13.Rennie D. Reporting randomized controlled trials. JAMA. 1995;273:1054–1055. doi: 10.1001/jama.273.13.1054. [DOI] [PubMed] [Google Scholar]
- 14.Begg CB, Cho M, Eastwood S, Horton R, Moher D, Olkin I, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA. 1996;276:637–639. doi: 10.1001/jama.276.8.637. [DOI] [PubMed] [Google Scholar]
- 15.Cochrane Controlled Trials Register (CCTR/CENTRAL) The Cochrane Library. Cochrane Collaboration. Oxford: Update Software; 1997. [Google Scholar]
- 16.Jadad AR, Moore RA, Carrol D, Jenkinson C, Reynolds DJM, Gavaghan DJ, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials. 1996;17:1–12. doi: 10.1016/0197-2456(95)00134-4. [DOI] [PubMed] [Google Scholar]
- 17.Brocker P, Vellas B, Albarede JL, Poynard T. A two-centre, randomized, double-blind trial of ornithine oxoglutarate in 194 elderly, ambulatory, convalescent subjects. Age Ageing. 1994;24:303–306. doi: 10.1093/ageing/23.4.303. [DOI] [PubMed] [Google Scholar]
- 18.Quality of life and clinical trials [editorial] Lancet. 1995;346:1–2. [PubMed] [Google Scholar]
- 19.Streiner DL, Norman GR. Health measurement scales. Oxford: Oxford University Press; 1989. [Google Scholar]
- 20.Slevin ML, Plant H, Lynch D, Drinkwater J, Gregory WM. Who should measure quality of life, the doctor or the patient? Br J Cancer. 1988;57:109–112. doi: 10.1038/bjc.1988.20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273:408–412. doi: 10.1001/jama.273.5.408. [DOI] [PubMed] [Google Scholar]
- 22.Egger M, Zellweger-Zähner T, Schneider M, Junker C, Lengeler C, Antes G. Language bias in randomised controlled trials published in English and German. Lancet. 1997;350:326–329. doi: 10.1016/S0140-6736(97)02419-7. [DOI] [PubMed] [Google Scholar]