Abstract
Introduction:
Results from randomized trials are least prone to systematic bias and represent the highest level of evidence in medical practice. We carried out a demographic analysis examining randomized controlled trials (RCTs) in prostate cancer. Particular emphasis was placed on newly conducted phase II/III RCTs between January 1997 and March 2006.
Methods:
We searched the MEDLINE database using the heading “prostate neoplasms” between January 1997 and March 2006. The results were then crossed with the MeSHs “Clinical trial.mp. OR clinical trial.pt. OR random:.mp. OR tu.xs;” this cross-checking is considered an optimal search strategy for detecting RCTs in MEDLINE® literature. The search yielded 7831 articles in total for the defined period. Of this total number, 7314 articles were manually analyzed and excluded as they did not represent RCTs. The qualifying 517 articles were then analyzed with emphasis on modality of therapy, cohort size, principal author, participating country and journal type.
Results:
Among the 517 randomized trials, most trials investigated medical therapies (42.7%). This was followed by diagnostic studies (13.2%), while the remaining categories made up 44.1%. A trend towards more completed RCTs is noted in the later years of the cohort. Cohort sizes were generally greater than 100 participants (63.1%). Urologists were the lead investigators in 48.2% of the trials. Trials were largely conducted in Europe and the United States (43.1% and 38.3%, respectively). About 7% of studies were based in Canada. Articles were generally published in surgical journals (48.4%), followed by medical journals (36.9%).
Conclusions:
Given that initial searches yielded nearly 8000 articles listed as RCTs in prostatic oncology, only a small percentage (5.4% to 8.6%) of these were actually RCTs which reported novel results. Most of the published data were either review articles or commentaries. It is abundantly clear that new recruitment strategies need to be developed to encourage patients to enrol in RCTs and that such studies need to be undertaken in urologic oncology to provide definitive answers to the abundant and unanswered questions in urologic oncology.
Résumé
Introduction :
Les résultats d’essais randomisés sont les moins sujets à une partialité d’emblée et représentent le plus haut niveau de données probantes en médecine. Nous avons mené une analyse démographique des essais contrôlés et randomisés (ECR) sur le cancer de la prostate, en portant attention plus particulièrement aux plus récents essais de phase II/III, menés entre janvier 1997 et mars 2006.
Méthodologie :
Nous avons interrogé la base de données MEDLINE avec le titre « prostate neoplasms » pour la période entre janvier 1997 et mars 2006. Les résultats ont ensuite été recoupés avec une recherche dans la base de données MeSH avec les clés « Clinical trial.mp. OU clinical trial.pt. OU random:.mp. OU tu.xs »; ce recoupement est vu comme une stratégie optimale de recherche pour cerner les ECR dans les articles classés dans MEDLINE. La recherche a produit un total de 7 831 articles pour la période définie. De ce nombre, 7 314 articles ont été analysés manuelle-ment et exclus puisqu’ils ne traitaient pas d’ECR. Les 517 articles admissibles ont ensuite été analysés, avec une attention particulière aux modalités de traitement, à la taille de la cohorte, à l’auteur principal, au pays participant et au type de périodique.
Résultats :
Sur les 517 essais randomisés, la plupart examinaient des traitements médicamenteux (42,7 %). Venaient ensuite les études diagnostiques (13,2 %), alors que le reste (44,1 %) entrait dans les autres catégories. On a noté une tendance vers une complétude plus élevée des ECR dans les dernières années de la période. Les cohortes comptaient habituellement plus de 100 participants (63,1 %). Les chercheurs principaux étaient des urologues dans 48,2 % des essais. La grande majorité des essais ont été menés en Europe et aux États-Unis (43,1 % et 38,3 %, respectivement). Environ 7 % des études étaient menées au Canada. Règle générale, les articles étaient publiés dans des revues en chirurgie (48,4 %), suivies des revues médicales (36,9 %).
Conclusions :
Comme les recherches initiales ont généré près de 8 000 articles classés comme des ECR en oncologie prostatique, seul un petit pourcentage (5,4 % à 8,6 %) de ces articles était en fait des ECR signalant des résultats inédits. La majorité des données étaient publiées sous forme d’articles de synthèse ou de commentaires. Il est très clair que de nouvelles stratégies de recrutement doivent être établies pour encourager les patients à s’inscrire aux ECR et que de tels essais doivent être entrepris en oncologie urologique afin de fournir des réponses claires aux nombreuses questions qui restent à élucider dans le domaine.
Introduction
Randomized controlled trials (RCTs) are considered among the most reliable for evaluating scientific evidence and are most effective in removing bias and confounding factors that regularly compromise the validity of medical and scientific research. Other types of studies are of benefit, such as feasibility studies or those that assess inherently subjective parameters, such as patient perceptions. In any case, the need for properly constructed randomized trials to evaluate the effectiveness of a wide range of medical or surgical therapies cannot be overstated.
There is a vast amount of work that goes into a properly designed RCT, which most often consists of a sufficiently large sample size and rigorous study design. Oftentimes, blinding is not possible or is unethical, such as in studies with surgical arms; often, sufficient sample sizes are difficult to attain. Generally, to validate and implement innovative treatment regimens, at least one properly constructed RCT needs to be conducted.
We exam the urologic oncology literature through numerous MEDLINE searches to ascertain the number articles published that used data from prospective, phase II/III RCTs. These may also be follow-up studies to the original conducted trials. We then undertake a demographic analysis of RCT articles in prostate cancer for 5 major categories from 1997 to March 21, 2006. These are modality of intervention, cohort size, area of expertise of principle author, country of origin and journal type in which the article was published.
Methods
A MEDLINE search was performed using the OVID MEDLINE database from “1966 to March Week 3 2006” with the exploded medical subject heading for “prostatic neoplasms.” The results of the search was then limited to publications in English and was stratified on a yearly basis from 1997 to March 21, 2006. Each of the 10 search strings corresponding to each year for a given category was then combined with the search string with the MeSH “Clinical trial.mp. OR clinical trial.pt. OR random:.mp. OR tu.xs.” This search string has been shown to be an optimal search strategy with great sensitivity, as high as 99.8%, for detecting RCTs in the MEDLINE database (Appendix 1).10
Publications were then manually appraised for inclusion as RCTs by examining the abstract and article itself. Demographic data were then extracted from each article and tabulated.
For this study, only RCTs that use primary data were included in this study. Meta-analyses and review articles were excluded. We included follow-up publications to ongoing trials. This would lead to greater than one article being tabulated for a given trial, usually incorporated with RCT articles in a subsequent year.
The modalities of intervention covered the main areas of focus for trials in urologic oncology (Table 2, Table 3). These included trials that had only medical, surgical or radiotherapeutic treatment arms versus controls. Combinations of the above were also considered and included. Studies that focused on primarily screening interventions and measurements of quality of life parameters were also included.
Table 2.
Cumulative data for all 517 articles over 10-year period
| Parameter | % of total | ||
|---|---|---|---|
| Modality of intervention | Medical | 221 | 42.7% |
| Surgical | 21 | 4.1% | |
| Radiation | 53 | 10.3% | |
| Medical + surgery | 65 | 12.6% | |
| Surgery + radiation | 4 | 0.8% | |
| Medical + radiation | 47 | 9.1% | |
| Medical + radiation + surgery | 5 | 1.0% | |
| Screening | 68 | 13.2% | |
| Quality of life study | 33 | 6.4% | |
| Cohort size | 10–50 | 97 | 18.8% |
| 51–100 | 94 | 18.2% | |
| >100 | 326 | 63.1% | |
| Principle author | Urologist | 249 | 48.2% |
| Medical oncologist | 98 | 19.0% | |
| Radiation oncologist | 108 | 20.9% | |
| Other | 62 | 12.0% | |
| Country | Europe | 223 | 43.1% |
| US | 198 | 38.3% | |
| Canada | 35 | 6.8% | |
| Asia | 35 | 6.8% | |
| Africa | 2 | 0.4% | |
| South America | 2 | 0.4% | |
| Australia/New Zealand | 22 | 4.3% | |
| Journal type | Medical | 191 | 36.9% |
| Surgical | 250 | 48.4% | |
| Radiation oncology | 70 | 13.5% | |
| Basic science | 6 | 1.2% |
Each subset represents the breakdown expressed as percentages. Ex. 42.75% of all the articles over the 10 years looked at medical therapies as the focus of intervention.
Urology is included here.
Table 3.
Randomized controlled trials in urologic oncology from 1997 to March 2006
| Year under study | 1997 | 1998 | 1999 | 2000 | 2001 | 2002 | 2003 | 2004 | 2005 | 2006** | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Total no. RCTs per year | 35 | 31 | 40 | 50 | 46 | 52 | 70 | 94 | 93 | 6 | |
| Modality of intervention | |||||||||||
| Medical | 13 | 14 | 20 | 17 | 22 | 26 | 28 | 41 | 35 | 5 | |
| Surgical | 0 | 1 | 0 | 6 | 3 | 2 | 4 | 1 | 4 | 0 | |
| Radiation | 0 | 1 | 3 | 8 | 6 | 4 | 8 | 8 | 15 | 0 | |
| Medical + surgery | 15 | 10 | 8 | 4 | 5 | 7 | 4 | 7 | 5 | 0 | |
| Surgery + radiation | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 0 | |
| Medical + radiation | 4 | 0 | 1 | 1 | 3 | 5 | 8 | 10 | 15 | 0 | |
| Medical + radiation + surgery | 0 | 0 | 0 | 0 | 0 | 2 | 0 | 2 | 1 | 0 | |
| Screening | 2 | 4 | 7 | 9 | 5 | 2 | 8 | 16 | 14 | 1 | |
| Quality of life study | 1 | 1 | 0 | 4 | 2 | 4 | 10 | 8 | 3 | 0 | |
| Cohort size | 10–50 | 5 | 4 | 5 | 11 | 12 | 6 | 9 | 19 | 24 | 2 |
| 51–100 | 7 | 5 | 5 | 7 | 5 | 11 | 17 | 16 | 19 | 2 | |
| >100 | 23 | 22 | 30 | 32 | 29 | 35 | 44 | 59 | 50 | 2 | |
| Principle author | Urologist | 21 | 20 | 20 | 25 | 17 | 24 | 35 | 43 | 40 | 4 |
| Medical oncologist | 5 | 1 | 10 | 10 | 16 | 13 | 10 | 17 | 14 | 2 | |
| Radiation oncologist | 6 | 2 | 4 | 11 | 11 | 10 | 18 | 17 | 29 | 0 | |
| Other | 3 | 8 | 6 | 4 | 2 | 5 | 7 | 17 | 10 | 0 | |
| Country | Europe | 20 | 21 | 17 | 17 | 18 | 22 | 28 | 35 | 42 | 3 |
| US | 10 | 8 | 13 | 24 | 20 | 19 | 28 | 40 | 33 | 3 | |
| Canada | 3 | 2 | 5 | 2 | 1 | 5 | 4 | 8 | 5 | 0 | |
| Asia | 1 | 0 | 5 | 2 | 4 | 3 | 4 | 7 | 9 | 0 | |
| Africa | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | |
| South America | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 | |
| Australia/New Zealand | 1 | 0 | 0 | 5 | 2 | 3 | 5 | 3 | 3 | 0 | |
| Journal type | Medical | 7 | 7 | 19 | 15 | 18 | 17 | 24 | 47 | 36 | 1 |
| Surgical | 23 | 21 | 18 | 24 | 20 | 26 | 36 | 38 | 39 | 5 | |
| Radiation oncology | 5 | 2 | 2 | 9 | 8 | 8 | 10 | 8 | 18 | 0 | |
| Basic science | 0 | 1 | 1 | 2 | 0 | 1 | 0 | 1 | 0 | 0 | |
RCTs: randomized controlled trials.
Urology is included here.
Data for 2006 up to March 21, 2006.
Screening studies were included as long as they were found to be randomized trials that allocated participants to intervention arms appropriately. An example of this is a trial that studied prostate specific antigen or prostate biopsies in men randomly allocated to screening versus no intervention.
Trials that focused on quality of life factors were also included if the intervention the trials addressed had participants randomized to treatment and control groups appropriately. For example, a trial that measured quality of life parameters in patients randomized to surgery or watchful waiting would have been included.
It should be noted when allocating articles to the category for journal type, articles that were found in journals with a focus on a primarily surgical field were sited as such. For example, if an article was found in Urology or European Urology, the journal was categorized as being primarily a surgical one.
Results
In the above-listed database, there were 28 465 publications for the exploded MeSH term “prostatic neoplasms.” The search term “Clinical trial.mp. OR clinical trial.pt. OR random:.mp. OR tu.xs.” yielded 1 110 837 results. Searches were combined in MEDLINE to yield all the English publications on a yearly bases with the above two MeSHs. The total number of articles for all years yielded from the search was 7831. These were then individually sifted, and only 517 (6.6%) articles were found to be RCTs.
We illustrate the selection process for RCTs for prostatic neoplasms in 1997 (Fig. 1). A similar process was carried out for subsequent years.
Fig. 1.
Process used to identify randomized controlled trials for prostatic neoplasms in 1997 using MEDLINE and manual exclusion. ≠Exact MeSH terms are: “Clinical trial.mp. or clinical trial.pt. or random:.mp. or tu.xs.”
For the 7831 articles obtained from our MEDLINE searches, we broke the data by year and this demonstrated a steady increase in the number of RCT publications since 1997 (Table 1), although only a small percentage of articles indexed as RCTs in prostatic oncology are actually RCTs per given year (5.4%–8.6%).
Table 1.
Number of articles before and after manual appraisal of MEDLINE searches
| Year | Before Manual Appraisal (N) | After Manual Appraisal (N) | % of Articles from MEDLINE search* |
|---|---|---|---|
| Mar 2006 | 70 | 6 | 8.6% |
| 2005 | 1112 | 93 | 8.4% |
| 2004 | 1124 | 94 | 8.4% |
| 2003 | 1146 | 70 | 6.1% |
| 2002 | 903 | 52 | 5.8% |
| 2001 | 817 | 46 | 5.6% |
| 2000 | 800 | 50 | 6.3% |
| 1999 | 688 | 40 | 5.8% |
| 1998 | 577 | 31 | 5.4% |
| 1997 | 594 | 35 | 5.9% |
Search strings included English language papers from 1997–2006 that were MeSH listed as randomized controlled trials. These were then manually examined to ensure study criteria were met.
This column represents the percentage of RCTs for each year from the total number of articles generated by the search strings before manual exclusion was carried out. e.g., for 2005: 8.4% = 93/1112 × 100%.
We also looked at cumulative parameters over the 10-year period (Table 2). Modalities within these parameters varied. However, most trials over this period commonly examined medical therapies (42.7%) to treat prostatic neoplasms. This was followed by diagnostic studies (13.2%). Cohort sizes were generally greater than 100 (63.1%). Urologists were often the lead investigators of the trials (48.2%). Trials were generally based in European cities, followed by the United States with 43.1% and 38.3%, respectively. Articles were usually published in surgical journals (48.4%) followed by medical journals (36.9%).
We tallied the demographic data from the remaining 517 articles (Table 3). These are shown for each year and provide a detailed analysis of the distribution of RCTs found using our search criteria over the past 10 years.
Discussion
We present, through a series of MEDLINE searches, demographic data from 517 retrieved articles in prostatic oncology. Several search systems, such as EMBASE, MEDLINE, CINAHL, are valuable tools to help health care and non-health care professionals across a wide range of specialties retrieve RCTs. Although these databases allow researchers to acquire and sift through large amounts of published articles rather expediently, they are not all-comprehensive and in some studies, database-based searches identified only 46% to 51% of RCTs.2,3 Often, references are also listed as RCTs when they do not qualify as such. Carr and Klotz showed that this rate may be as high as 78%.4 Given such rates, it would not be surprising to discover that a large number of articles were not identified by our searches over such an extended study period of 10 years. In this respect, our MeSHs may have also limited our retrieval numbers.
In other searches we conducted (data not shown), RCTs have been indexed with MeSHs “Controlled Clinical Trials,” “Clinical Trials,” “Random Allocation” (to name a few). These were not included as part of our search terms as we primarily wanted to look at RCTs. Since a vast quantity of non-random trials would have been retrieved with multiple MeSHs as well, we chose to include only articles that had the MeSH “Clinical trial.mp. OR clinical trial.pt. OR random:.mp. OR tu.xs.” As discussed earlier, this has been shown to have high sensitivity for retrieving high-powered clinical trials through MEDLINE.
Of the 23 905 articles indexed with MeSH “prostatic neoplasms” retrieved over our 10-year time interval, only 517 (2.2%) were RCTs. This is consistent with findings by Carr and Klotz.4 It is worth noting that most excluded publications were review articles, meta-analyses, commentaries and proposals.
Although this paper does not directly compare the number of RCTs with other medical or surgical fields, for example RCTs in leukemia literature, others have shown that the number of published RCTs across many fields is also quite dismal when compared to other study designs with less internal validity. In one study, Sauerland and colleagues demonstrated that the proportion of RCTs published in many surgical and medical journals representing a wide range of surgical and medical specialties was quite low, 2.5% and 3.6%, respectively.5 From the paucity of RCT publications, it is not unreasonable to infer that an equally low number of such trials are being conducted.
Despite the sparse number of RCTs conducted each year in prostatic oncology, there is some evidence to suggest that a more rigorous approach to study design and investigation is taking place. Al-Futaisi and colleagues compared the number of clinical trials and evaluated the quality of research in pediatric urology in 1990 and 2000.6 They showed that although a similar number of clinical trials were being conducted in the 2 years, there was a statistically significant increase in analytic design (21.8% vs. 39.5%, p = 0.01) and an increase in case control studies (11.3% vs. 22.9%, p = 0.02) from 1990 to 2000, respectively.6 This occurrence was also noted by Mylonas who also found that clinical research in neurosurgery in 1990 used studies with significantly stronger research designs than in 1975.7
It has been shown in this work and several other quoted references that the number of RCT studies published each year are meagre. The question remains: what are the obstacles to RCTs being performed in urologic oncology and other surgical and medical specialties?
Firstly, one of the major difficulties in conducting RCTs in surgery are issues of ethical principles that arise regarding randomization. It is unimaginable, in light of current surgical practices, to enter a control group a radical prostatectomy, for example. Concordantly, many patients have a difficult time accepting the concept of randomization in surgical clinical trials. O’Reilly and colleagues write: “Men with a life expectancy of 10 to 25 years who develop prostate cancer will not allow themselves to be randomized to a “watchful waiting group.”8
One should also comment on the innate differences in surgical practice between and within institutions and surgical centres everywhere and the intrinsic variability in surgical treatment due to human error. Kranenbarg and Van de Velde wrote that: “Randomization of patients to evaluate surgical procedures involves problems in addition to those associated with medical experimentation. Surgery, unlike a pill, is not a standardized, reproducible entity, but rather a unique product whose details are defined by variables which include, for example, the skill of the surgeon. The skill level will not only vary among surgeons, but will increase for the same surgeon as he gains experience.”9
The very essence of RCTs is to provide a high degree of intrinsic validity so that unique parameters can be evaluated with few confounding errors. Introducing human experience to such a degree, as inherent in surgery, has profound effects on outcomes or procedures and patient decision-making.
Secondly, the cost of definitive, large cohort size RCTs is a major cause for their scarceness. Weinberger and colleagues discuss the obstacles, both scientific and operational issues, involved in properly designed RCT; they also examine the innate enormous financial burden of conducting large unisite or multisite RCTs.10
Thirdly, RCTs especially in surgical fields have greater challenges than those in other medical fields; this is especially true in Canadian centres where there are limits on operating room hours and the resulting competition between surgical specialties for this limited time puts a cap on the amount of procedures that can be completed. For example, it is much more efficient for physicians to administer and investigate the effects of an oral therapy given in pill form to individuals inflicted with an ailment than it is for surgeons to carry out the same number of procedures. Obviously fewer time and resources are needed to administer a pill than to perform surgery.
Other obstacles to RCT realization specific for surgery include poor outcome and complications to surgery (for example, when a patient expires as a result of postoperative infection and not necessarily as a consequence of the offending tumour). Also dropout rates and loss to follow-up can have significant effects on results, as would occur in other RCTs.11
There are many barriers which prevent extremely well-designed RCTs from being universally generalizable.
Conclusion
Our study examines the number of RCTs retrieved from numerous MEDLINE searches in prostatic oncology from 1997 to March 2006 inclusive and supports previous studies that show a relatively low number of RCTs being conducted in prostatic oncology.
Significant efforts are needed to overcome the numerous hurdles so that the body of evidence supporting therapeutic interventions may be increased.
Appendix 1. Medline search strategy
| 0 Database: Ovid MEDLINE(R) <1996 to March Week 3 2006> Search Strategy: | |
|---|---|
| 1 | Clinical trial.mp. or clinical trial.pt. or random:.mp. or tu.xs. (1110837) |
| 2 | exp Prostatic Neoplasms/ (28465) |
| 3 | limit 2 to (english language and yr=“2006”) (237) |
| 4 | exp Prostatic Neoplasms/ (28465) |
| 5 | limit 4 to (english language and yr=“2005”) (3324) |
| 6 | exp Prostatic Neoplasms/ (28465) |
| 7 | limit 6 to (english language and yr=“2004”) (3462) |
| 8 | exp Prostatic Neoplasms/ (28465) |
| 9 | limit 8 to (english language and yr=“2003”) (3265) |
| 10 | exp Prostatic Neoplasms/ (28465) |
| 11 | limit 10 to (english language and yr=“2002”) (2748) |
| 12 | exp Prostatic Neoplasms/ (28465) |
| 13 | limit 12 to (english language and yr=“2001”) (2545) |
| 14 | exp Prostatic Neoplasms/ (28465) |
| 15 | limit 14 to (english language and yr=“2000”) (2381) |
| 16 | exp Prostatic Neoplasms/ (28465) |
| 17 | limit 16 to (english language and yr=“1999”) (2068) |
| 18 | exp Prostatic Neoplasms/ (28465) |
| 19 | limit 18 to (english language and yr=“1998”) (1946) |
| 20 | exp Prostatic Neoplasms/ (28465) |
| 21 | limit 20 to (english language and yr=“1997”) (1929) |
| 22 | 1 and 3 (70) |
| 23 | 1 and 5 (1112) |
| 24 | 1 and 7 (1124) |
| 25 | 1 and 9 (1146) |
| 26 | 1 and 11 (903) |
| 27 | 1 and 13 (817) |
| 28 | 1 and 15 (800) |
| 29 | 1 and 17 (688) |
| 30 | 1 and 19 (577) |
| 31 | 1 and 21 (594) |
Footnotes
This paper has been peer-reviewed.
Competing interests: None declared.
References
- 1.Haynes BR, McKibbon AK, Wilczynski NL, et al. Optimal search strategies for retrieving scientifically strong studies of treatment from Medline: analytical survery. BMJ. 2005;330:1179. doi: 10.1136/bmj.38446.498542.8F. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Poynard T, Conn HO. The retrieval of randomized clinical trials in liver disease from the medical literature. A comparison of MEDLARS and manual methods. Control Clin Trials. 1985;6:271–9. doi: 10.1016/0197-2456(85)90103-5. [DOI] [PubMed] [Google Scholar]
- 3.Solomon MJ, Laxamana A, Devore L, et al. Randomized controlled trials in surgery. Surgery. 1994;115:707–12. [PubMed] [Google Scholar]
- 4.Carr LK, Klotz LH. Randomized controlled trials in urologic oncology: a demographic analysis. Can J Urol. 1994;1:83–6. [PubMed] [Google Scholar]
- 5.Sauerland S, Bouillon B, Neugebauer E. The practice of evidence-based medicine in surgery--evidence and experience. Z Arztl Fortbild Qualitatssich. 2003;97:271–5. [PubMed] [Google Scholar]
- 6.Al-Futaisi A, Shevell M. Comparative audit of clinical research in pediatric neurology. J Child Neurol. 2004;19:882–6. doi: 10.1177/08830738040190110601. [DOI] [PubMed] [Google Scholar]
- 7.Mylonas C. Research in neurosurgical journals. Br J Neurosurg. 1992;6:41–6. doi: 10.3109/02688699209002900. [DOI] [PubMed] [Google Scholar]
- 8.O’Reilly P, Martin L, Collins G. Few patients with prostate cancer are willing to be randomised to treatment. BMJ. 1999;318:1556. doi: 10.1136/bmj.318.7197.1556a. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Kranenbarg EK, van de Velde CJ. Importance of organizing surgical trials in oncology. Jpn J Clin Oncol. 1999;29:185–6. doi: 10.1093/jjco/29.4.185. [DOI] [PubMed] [Google Scholar]
- 10.Weinberger M, Oddone EZ, Henderson WG, et al. Multisite randomized controlled trials in health services research: scientific challenges and operational issues. Med Care. 2001;39:627. doi: 10.1097/00005650-200106000-00010. [DOI] [PubMed] [Google Scholar]; Young JM, Solomon MJ. Improving the evidence-base in surgery: evaluating surgical effectiveness. ANZ J Surg. 2003;73:507–10. doi: 10.1046/j.1445-1433.2003.02662.x. [DOI] [PubMed] [Google Scholar]
- 11.Solomon MJ, Pager CK, Young JM, et al. Patient entry into randomized controlled trials of colorectal cancer treatment: Factors influencing participation. Surgery. 2003;133:608–13. doi: 10.1067/msy.2003.119. [DOI] [PubMed] [Google Scholar]