Abstract
Background
Investigators of clinical trials in which the list of outcomes include patient-reported outcomes (PROs) - usually labeled quality of life (QoL) - have a large number of instruments from which to choose. The extent and manner in which PRO instruments are used in clinical trials can be assessed using data from clinical trial registries. Most medical journals now require a clinical trial be registered before its results are considered for publication. This requirement is intended to discourage publication bias, such as the reporting of tests of hypotheses different from those stipulated at the start of the trial and selective reporting of partial results.
Purpose
To assess the usage of PRO instruments in registered trials by various trial characteristics and to determine if the instruments are adequately identified in the registry.
Methods
A local copy of the ClinicalTrials.gov database was made in September 2007. The outcomes of all interventional trials registered since September 2004 were assessed for usage of a PRO instrument. Odds ratios of PRO usage were estimated by a logistic regression model.
Results
Of 17,704 interventional trials, 2,481 (14.0%) used at least one PRO instrument. However, less than half of those trials (41.0%) identified the instrument to be used. PRO usage is positively associated with phase (III), randomization type (randomized), intervention type (behavior) and sponsorship type (university/research organization).
Conclusions
PRO instruments are used in a significant percentage but minority of clinical trials. Trial registries should require that all PRO instruments be identified, including the concepts or outcomes they are intended to measure.
Keywords: patient-reported outcomes, quality of life, clinical trial registration, publication bias
1. Introduction
Patient-reported outcome (PRO) instruments, i.e. questionnaires to be completed by the trial participants, are used in clinical trials to assess how a patient feels. Although PRO includes any patient-reported outcome, indexes have long used “quality of life” (QoL) in classification of published reports [1]. The increasing importance of PRO’s in the assessment of clinical trial outcomes is demonstrated by the establishment of initiatives such as the Patient-Reported Outcomes Measurement Information System (PROMIS) [2] and guidance on the usage of PRO instruments from the U.S. Food and Drug Administration (FDA) [3].
Reviews of published reports of clinical trials, however, have found that only a small percentage trials use PRO instruments. From a review of the Cochrane Controlled Trials Register, Sanders et al. [4] found that from 1980 to 1997 no more than 4.2% reported measures of QoL in all disciplines and no more than 8.2% of cancer trials reported measures of QoL. Gotay and Wilson [5] found that only 7% of trials published in the Journal of Clinical Oncology between 1992 and 1996 included PRO assessments. A more recent review of abstracts in PubMed by Naito et al. [6] found that only 4.4% of trial results published from 2000–2003 assessed QoL. As with the Sanders et al. study, Naito et al. found that the most common condition assessed for QoL was cancer.
The fact that all of these studies relied on published reports of clinical trials is possibly a common deficiency. From a review of registered clinical trials of cancer treatments, Ramsey and Scoggins [7] found that the results of only 17% of completed trials are published in peer-reviewed journals. Consequently, assessments of PRO usage in clinical trials from peer-reviewed articles could suffer from publication bias. Furthermore, it can take several years from the start of a clinical trial to the publication of its results. So even reviews of relatively recent publications might reflect the choices made by clinical trial investigators from many years in the past.
Ramsey and Scoggins based their conclusion from examination of data from ClinicalTrials.gov, a U.S. government-managed registry of clinical trials. Since the announcement on clinical trial registration by the International Committee of Medical Journal Editors (ICMJE), all major medical journals have made registration of clinical trials before the start of participant enrollment a precondition for publication of trial results [8]. The announcement was published in September 2004, but the final deadline for registration of trials already started was in September 2005. This concerted effort was made in the hopes that it would deter the kind of publication bias that occurs when outcome measures of trial results are selectively reported [9,10]. Therefore there is an emphasis on pre-accrual registration of the trial’s outcome measurements.
ClinicalTrials.gov, by far the largest registry of clinical trials in the world with 17,704 interventional clinical trials registered between September 1, 2004 and September 1, 2007, provides information on each trial’s sponsor, enrollment and design features, including its primary and secondary outcomes [11,12]. Consequently it provides a source of information with which to make an unbiased and timely assessment of PRO instrument usage in clinical trials.
2. Methods
In September 2007 a local copy of the entire ClinicalTrials.gov website was made. The study adopted the sponsor type classification by the registry: industry, university/research organization, National Institutes of Health (NIH), federal government (not NIH), network and other (foreign) government.
Any trial registrations that contained either “quality of life”, “qol”, “patient-reported outcome”, “health status”, “patient satisfaction” or “psychometric” were examined to determine if they were PRO trials. The lists of outcomes of these trials were then searched for 395 PRO instrument names and abbreviations found in the PROQOLID [13] database. Only occurrences of the search terms in conjunction with the instrument name or abbreviation within the same bullet point were designated as having identified a PRO instrument. The remaining possible PRO trial registrations were then examined by one of the authors for occurrences of PRO instrument names and abbreviations that were not included in the PROQOLID database.
We fitted the data to a logistic regression model with PRO usage as the dependent variable. The indicator variables were: sponsor type (university/research organization, industry, National Institutes of Health (NIH) and other); intervention type (drug, procedure, behavior, device and other); phase (I, II, III and IV) and randomization type (randomized, non-randomized and unknown).
3. Results
Of 17,704 interventional trials registered between September 2004 and September 2007, 2,481 (14.0%) stated that a PRO was one of the outcomes. Trials sponsored by university/research organizations were more likely to measure PRO than the other major sponsor types, i.e. commercial firms and the National Institutes of Health (OR: 1.12, 95% CI: 1.01–1.23). So were the remaining sponsor types, mostly networks of research centers (OR: 1.25, 95% CI: 1.09–1.45). Some trials were sponsored by more than one type of sponsor. Trials with behavior interventions (OR: 2.98, 95% CI: 2.57–3.45), procedure interventions (OR: 1.92, 95% CI: 1.69–2.18) and device interventions (OR: 1.75, 95% CI: 1.50–2.04) use PRO instruments more often than trials with drug interventions. Phase I trials (OR: 0.33, 95% CI: 0.27–0.39), phase II trials (OR: 0.66, 95% CI: 0.58–0.75) and phase IV trials (OR: 0.69–0.91) are each less likely to use PRO instruments than are phase III trials. Both non-randomized trials (OR: 0.76, 95% CI: 0.68–0.85) and trials for which randomization status is unknown (OR: 0.48, 95% CI: 0.38–0.59) are less likely to measure PRO than are randomized trials.
We found 160 different PRO instruments named in the registry, however only 1,012 of 2,481 (41%) registered trials that use PRO instruments identified the specific instrument. Thirty-six of the identified PRO instruments were generic assessments of health related quality of life. The other 124 instruments were designed to be disease specific. Although generic PRO instruments represented less than one fourth of the identified instruments, they were used in more than half (549/1012) of the trials which identified the PRO instrument.
Trials of treatments for diseases of the genitourinary tract use PRO instruments more than any other type of condition (22.2%). Trials of treatments for neoplasms used PRO instruments no more frequently than that by all other trials (13.9%).
Each ICD-10 classification of the condition being treated in a trial has its own most frequently used PRO instrument. The World Health Organization Quality of Life (WHOQOL) is the most popular PRO for trials of cancer treatments (87). The Short Form-36 (SF-36) was the most used instrument by trials of treatments for diseases of the nervous (16 trials), genitourinary (14 trials), digestive (9 trials), circulatory (27) and musculoskeletal (34 trials) systems as well as the most used instrument overall (177 trials).
4. Comment
The usage of PRO instruments in clinical trials is considerably greater than reported in previous studies. The difference is even greater when the information on which the earlier studies are based is considered. The characteristics of trials which are positively associated with publication, namely randomized phase III trials sponsored by non-commercial organizations, are also positively associated with PRO usage. So the 4% of published trials that assessed QoL in years past appears even smaller when compared to the 14% of all registered trials of today.
The growth in PRO instrument usage in clinical trials is not surprising. The development of expensive treatments for serious illnesses that extend life but with significant adverse side-effects, has created a greater need for PRO assessments in clinical trials. In spite of this increased need, there is only partial acceptance and usage of PRO instruments in clinical trials. Only 18.0% of phase III trials specify PRO as either a primary or secondary outcome. Trials of drug interventions and those sponsored by commercial organizations are particularly unlikely to use PRO instruments.
Clayson et al. [14] found that half of all QoL instruments used in published HIV/AIDS clinical trials were generic. We found that a similar proportion of registered PRO trials for all conditions use generic instruments, such as the European Quality of Life (EQ-5D) and World Health Organization Quality of Life (WHOQOL). A generic instrument is the most frequently used instrument in trials of treatments for six of the 14 ICD-10 classes of medical conditions. The SF-36 is the most frequently used PRO instrument in clinical trials today. In spite of well over a hundred PRO instruments used in clinical trials and developed to be used for specific medical conditions, generic PRO instruments remain very popular.
Both Sanders et al. [4] and Naito et al. [6] found that a relatively large proportion of QoL assessment trials were for the treatment of cancer. We found that the same percentage of cancer trials use PRO instruments as do all trials regardless of the condition being treated (14%). In terms of proportion of trials, PRO instrument usage in trials of treatments for the genitourinary system and injury & poison conditions is much more common than PRO instrument usage in cancer trials. The reason there are so many cancer trials that use PRO instruments is because cancer trials represent one fourth of all registered clinical trials.
One limitation of this study is that of the trials that do assess PRO, more than half do not specify in the registry which instruments are to be used. A prominent example is the Study of Tamoxifen and Raloxifene (STAR) for the Prevention of Breast Cancer in Postmenopausal Women [15]. This large (19,000 participants), ongoing trial started in May 1999 and is sponsored by the National Cancer Institute. The trial’s registry information lists as one of its objectives “[d]etermine the effect of these regimens on the quality of life of these participants”. The actual PRO instruments used, i.e. the SF-36, Center for Epidemiologic Studies Depression Scale, Medical Outcomes Study Sexual Activity Questionnaire and NSABP P-1 BCPT, were not identified until the results were presented at the 2006 American Society of Clinical Oncology Annual Meeting Proceedings [16]. Since the point of the registry is to guarantee that all the objectives and measured outcomes of the trials are clearly defined prior to the commencement of participant accrual, identification of any PRO instruments should be explicit.
Table 1.
PRO Usage in Interventional Trials Registered Between September 2004 and September 2007 by Type of Sponsor, Intervention, Phase and Randomization
| PRO Trials |
||||||
|---|---|---|---|---|---|---|
| 95% Conf. Int. |
||||||
| Category | No. of Trials | Percent | Odds Ratio | Lower | Upper | |
| Sponsor | ||||||
| Industry | 8,849 | 12.1% | 1.00 | n/a | - | n/a |
| University/RO | 8,381 | 15.3% | 1.12 | 1.01 | - | 1.23 |
| NIH | 2,673 | 14.3% | 1.00 | 0.87 | - | 1.15 |
| Other | 682 | 21.7% | 1.25 | 1.09 | - | 1.45 |
| Intervention | ||||||
| Drug | 12,722 | 12.0% | 1.00 | n/a | - | n/a |
| Procedure | 3,312 | 16.7% | 1.92 | 1.69 | - | 2.18 |
| Behavior | 1,499 | 27.5% | 2.98 | 2.57 | - | 3.45 |
| Device | 1,327 | 19.1% | 1.75 | 1.50 | - | 2.04 |
| Other | 852 | 4.8% | 0.65 | 0.51 | - | 0.83 |
| Phase | ||||||
| I | 2,984 | 6.6% | 0.33 | 0.27 | - | 0.39 |
| II | 5,121 | 12.4% | 0.66 | 0.58 | - | 0.75 |
| III | 3,955 | 18.0% | 1.00 | n/a | - | n/a |
| IV | 2,447 | 15.4% | 0.79 | 0.69 | - | 0.91 |
| Unknown | 3,197 | 17.5% | 0.58 | 0.50 | - | 0.67 |
| Randomization | ||||||
| Randomized | 11,523 | 16.2% | 1.00 | n/a | - | n/a |
| Non-randomized | 4,703 | 10.4% | 0.76 | 0.68 | - | 0.85 |
| Unknown | 1,478 | 8.7% | 0.48 | 0.38 | - | 0.59 |
| All categories | 17,704 | 14.0% | ||||
Table 2.
Percent of Trials that Measure Patient-Reported Outcomes by ICD-10 Classification
|
Table 3.
Number of PRO Instruments Identified by ICD-10 Classification
| Most Fequently Specified Instrument |
|||
|---|---|---|---|
| ICD-10 Classification | Number of Instruments | Instrument Name | Number of Trials |
| Mental and behavioral | 44 | Heinrichs-Carpenter Quality of Life Scale (QLS) | 24 |
| Nervous system | 44 | Short Form-36 (SF-36) | 16 |
| Neoplasms | 42 | World Health Organization Quality of Life (WHOQOL) | 87 |
| Circulatory system | 32 | Short Form-36 (SF-36) | 27 |
| Endocrine, nutritional and metabolic | 30 | Impact of Weight on Quality of Life (IWQOL) | 8 |
| Respiratory system | 28 | Asthma Quality of Life Questionnaire (AQLQ) | 17 |
| Digestive system | 25 | Short Form-36 (SF-36) | 9 |
| Genitourinary system | 24 | Short Form-36 (SF-36) | 14 |
| Musculoskeletal system | 23 | Short Form-36 (SF-36) | 34 |
| Blood and blood-forming organs | 16 | Functional Assessment of Cancer Therapy (FACT)* | 8 |
| Infectious and parasitic | 11 | MOS-HIV Health Survey (MOS-HIV) | 9 |
| Injury, poisoning and etc. | 11 | International Prostate Symptom Score (IPSS QoL) | 12 |
| Skin and subcutaneous tissue | 10 | Dermatology Life Quality Index (DLQI) | 4 |
| Eye and adnexa | 3 | NEI Visual Functioning Questionnaire (NEI-VFQ 25) | 3 |
|
| |||
| All | 160 | Short Form-36 (SF-36) | 177 |
Tied with SF-36
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Contributor Information
John F. Scoggins, Senior Research Fellow, Fred Hutchinson Cancer Research Center, 1100 Fairview Ave N, M3-B232, Seattle, WA 98109 Fax: 206.667.5977, Phone: 206.667.7424, e-mail: jscoggin@fhcrc.org.
Donald L. Patrick, School of Public Health and Community Medicine, University of Washington, Department of Health Services, Seattle Quality of Life Group, Box 359455, Seattle, WA 98195-9455.
References
- 1.Fayers PM, Machin D. Quality of life: assessment, analysis and interpretation. New York: John Wiley & Sons, Ltd; 2007. [Google Scholar]
- 2. [Accessed February 10, 2009]; http://www.nihpromis.org/default.aspx.
- 3.Food and Drug Administration Guidance for Industry, Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims, February 2006.
- 4.Sanders C, Egger M, Donovan J, et al. Reporting on Quality of Life in Randomized Controlled Trials: Bibliographic Study. BMJ. 1998;317:1191–1194. doi: 10.1136/bmj.317.7167.1191. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Gotay CC, Wilson M. Use of Quality-of-Life Outcome Assessments in Current Cancer Clinical Trials. Eval Health Prof. 1998;21:157. doi: 10.1177/016327879802100203. [DOI] [PubMed] [Google Scholar]
- 6.Naito M, Nakayama T, Fukuhara S. Quality of Life Assessment and Reporting in Randomized Controlled Trials: a Study of Literature Published from Japan. Health and Quality of Life Outcomes. 2004;2:31. doi: 10.1186/1477-7525-2-31. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Ramsey S, Scoggins J. Commentary: Practicing on the Tip of an Information Iceberg? Evidence of Underpublication of Registered Clinical Trials in Oncology. The Oncologist. 2008;13:925–929. doi: 10.1634/theoncologist.2008-0133. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.DeAngelis CD, Drazen JM, Frizelle FA, et al. Clinical Trial Registration: A Statement from the International Committee of Medical Journal Editors. JAMA. 2004;292(11):1363–1364. doi: 10.1001/jama.292.11.1363. [DOI] [PubMed] [Google Scholar]
- 9.Dickersin K, Rennie D. Registering Clinical Trials. JAMA. 2003;290(4):516. doi: 10.1001/jama.290.4.516. [DOI] [PubMed] [Google Scholar]
- 10.Steinbrook R. Public Registration of Clinical Trials. NEJM. 2004;351(4):315–317. doi: 10.1056/NEJMp048191. [DOI] [PubMed] [Google Scholar]
- 11.Zarin DA, Ide NC, Tse T, Harlan WR, West JC, Lindberg DAB. Issues in the Registration of Clinical Trials. JAMA. 2007;297(19):2112–2120. doi: 10.1001/jama.297.19.2112. [DOI] [PubMed] [Google Scholar]
- 12. [Accessed September 4–6, 2007]; http://ClinicalTrials.gov.
- 13. [Accessed March 3, 2008]; http://www.qolid.org.
- 14.Clayson DJ, Wild DJ, Quarterman P, et al. A Comparative Review of Health-Related Quality-of-Life Measures for Use in HIV/AIDS Clinical Trials. Pharmacoeconomics. 2006;24(8):751–765. doi: 10.2165/00019053-200624080-00003. [DOI] [PubMed] [Google Scholar]
- 15. [Accessed February 12, 2009]; http://clinicaltrials.gov/ct2/show/NCT00003906.
- 16.Ganz PA, Land SR, Wickerham DL, et al. The Study of Tamoxifen and Raloxifene (STAR): First Report of Patient-reported Outcomes (PROs) from the NSABP P-2 Breast Cancer Prevention Study. [Abstract] J Clin Oncol. 2006;24(Suppl 18):A-LBA561. [Google Scholar]