Abstract
Purpose
To assess the quality of randomized controlled trials (RCTs) on varicocele published from 1979 to 2017.
Materials and Methods
We searched for original RCT on varicocele published between 1979 and 2017. Jadad scale, van Tulder scale, and Cochrane Collaboration Risk of Bias Tool were used to analyze RCT quality over time. Effects on RCT quality including funding source, Institutional Review Board (IRB) approval, and intervention were assessed. Treatment parameters of varicocele were also analyzed.
Results
Blinding and allocation concealment were described in 25.9% and 9.4% of RCT, respectively. Both tended to increase and a sharp dip in allocation concealment was observed in 2010–2017. Jadad scores increased steadily from 1979 to 2017 (1.28±0.59 to 2.19±1.10, p<0.01). Van Tulder scores tended to increase from 1979 to 2017 (4.21±0.94 to 5.58±1.58, p<0.01). RCTs with funding statements had higher Jadad (Yes vs. No, 3.25±0.50 vs. 1.70±0.97; p<0.01) and van Tulder (Yes vs. No, 7.25±1.26 vs. 4.81±1.26; p<0.01) scores than unfunded RCTs. IRB approval and intervention were associated with better quality.
Conclusions
The number of RCTs on varicocele increased from 1979 to 2017. Also, quality improved over time with increasing IRB approval, funding, and multicenter trial. Most RCTs on varicocele reported the use of surgical treatment. RCTs of surgical treatments have limitations to satisfy the condition of RCT to conduct, but their quality has improved over time.
Keywords: CONSORT statement, Randomized control trial, Reporting quality, Varicocele
INTRODUCTION
Evidence-based medicine (EBM) involves the application of medical treatment that derives high-quality results through a systematic process that combines clinical evidence [1]. With the emphasis on EBM in the medical field, randomized controlled trial (RCT), in which evidence is most highly valued, are becoming important. This is because the RCT is the most reliable method for evaluating the effectiveness of medical treatment by minimizing bias [2]. For this reason, the number of RCT is rapidly increasing worldwide, while the conditions and criteria for RCT are high [3,4]. To improve RCT quality, it is necessary to evaluate objective methodological study quality [5]. This is because bias can occur if the basic elements of the RCT, from design to performance, are not properly implemented [6]. The International Committee of Medical Journal Editors (ICMJE) recommends the CONSORT statement as a guideline for improving RCT quality [7,8]. Although the CONSORT statement may help investigators conduct RCTs easily and appropriately, it is a just guideline, not a quality evaluator [9]. RCT methodological quality evaluation methods include scales, checklists, and individual markers, among others. Of them, scales have the advantage of better enabling quantitative assessments of clinical trial quality than other methods [8].
The Jadad scale, van Tulder scale (VTS), and Cochrane Collaboration Risk of Bias Tool (CCRBT) are representative quality assessment tools for the scale. The Jadad scale is a tool for evaluating randomization, double-blinding, and dropout to reduce bias [10]. Although the Jadad scale is simple and easy to evaluate, it does not include allocation concealment, an indicator of individual markers, so it is difficult to avoid selection bias in assigning patients to treatment. The VTS and CCRBT contain evaluation items for allocation concealment, which has the advantage of evaluating selection bias.
A varicocele is a state in which the pampiniform plexus, a vein in the scrotal sac, is palpably dilated secondarily due to retrograde blood flow of the vein to the testis [11]. Varicocele occurs in 10% to 15% of men and causes primary infertility in approximately 35% to 50% and secondary infertility in 81% [12]. There are many ways to treat varicocele: inguinal varicocelectomy, subinguinal microscopic varicocelectomy, percutaneous embolization, and laparoscopic varicocele ligation. However, controversy persists about the ideal treatment methods [13,14].
Erectile dysfunction, premature ejaculation, varicocele, and etc. are representative diseases that have been conducted a lot of randomized controlled studies in the andrology field. For this reason, we conducted our study.
No studies to date have evaluated varicocele RCT quality. Therefore, we suggest the direction of further studies by evaluating varicocele RCT quality using the three representative RCT quality assessment tools: the Jadad scale, VTS, and CCRBT.
MATERIALS AND METHODS
1. Subjects
The subjects of the analysis were RCTs identified based on a search of “varicocele” in PubMed and MEDLINE. Original RCTs of varicocele were searched from 1979 to 2017 (Supplement Table 1). An RCT quality analysis was performed by dividing the studies into three periods (early, 1979–2000; mid, 2001–2009; and late, 2010–2017) considering the CONSORT statement was first published in 2001 and revised in 2010 considering application term.
2. RCT selection
Two reviewers (WJS, HJP) independently searched PubMed and MEDLINE. We extracted and validated articles using the search terms “randomized,” “randomization,” “randomly,” “varicocele”, and “varicocelectomy.” Differences in extraction results between reviewers were subjected to data adjustment (Fig. 1).
Fig. 1. Flow diagrams. RCT: randomized controlled trial.
3. Evaluation method using quality assessment tools
Two reviewers (JHC, KSK) separately analyzed the RCTs using the Jadad scale, VTS, and CCRBT. When differences occurred, one other reviewer settled them.
1) Jadad scale
The Jadad scale, also known as the Oxford quality scoring system, consists of three items. It consists of two scores for randomization, two scores for blinding, and one score for dropout. Regarding randomization, one point is given if the article mentions randomization, while another point is given if it includes the appropriate randomization method and 1 point is subtracted for inappropriate randomization for a total of 0–2 points. Regarding blinding, 1 point is given when double blinding is mentioned, and an additional 1 point is given when it includes an appropriate blinding method and 1 point is subtracted for inappropriate blinding for a maximum of 2 points. If the article mentioned dropout, 1 point was given. The quality evaluation is assessed by a total score of 5 points, with 0–2 points indicating low quality and 3–5 points indicating high quality [15].
2) van Tulder scale
The VTS evaluates RCT quality using the following 11 items: appropriateness of the randomization, concealment of treatment assignment, similarity of baseline characteristics, blinding of the patients, blinding of the treatment provider, observer blinding, co-intervention, compliance, dropout/failure rate, end-point assessment time point, and intention-to-treat analysis. Each item is assessed as Yes, No, or Do Not Know; if the total score is 5 or higher, the RCT is considered of high quality [16].
3) Cochrane Collaboration Risk of Bias Tool
Cochrane coalitions are divided into sequence generation, allocation concealment, blinding, incomplete outcome data, selective outcome reporting, and other potential threats to validity. Each item is answered with Yes, No, or Unclear, and the reviewers assess the answers Yes, No, or Unclear of each item according to the detailed criteria. Yes indicates high quality, No indicates low quality, and Unclear indicates insufficient information. A study is classified as having a low risk of bias when the first three questions are answered as Yes and no important matters related to the last three domains are identified, as having a moderate risk of bias when Unclear or No is answered in ≤2 domains, and as having a high risk of bias when Unclear or No is answered in ≥3 domains [17].
4. Quality assessment of RCTs using other factors
We qualitatively analyzed the RCTs according to the presence or absence of Institutional Review Board (IRB) approval, funding, intervention, multicenter design, and allocation. We also assessed the articles against the treatment measures that we wanted to see in each study. The treatment methods for varicocele and numerical distribution of the journals that published RCTs on varicocele were confirmed.
5. Statistical analysis methods
One-way analysis of variance was used to compare differences in scores among the assessment tools. The comparisons of percentages of high-quality RCTs and the qualitative assessment by CCRBT were analyzed using the chi-square test. In addition, Student's t-test was used to compare RCT quality according to the presence or absence of IRB approval, funding, blinding, and intervention. SPSS version 22.0 was used for the statistical analysis, and values of p<0.05 were considered statistically significant.
6. Ethics statement
This study is based on analysis of previous published articles. We did not approved IRB.
RESULTS
1. Quantitative change in RCTs over time
From 1979 to 2017, 85 original RCTs were published on varicocele. Of them, 29 were published in the early period, 30 were published in the mid period, and 26 were published in the late period. From the early stage to the late stage, studies including blinding, concealed allocation, funding, IRB approval, and multicenter design tended to increase (Table 1).
Table 1. Characteristics of RCTs by publication year with quality assessment of RCTs.
| Factor | Early (1979–2000) | Mid (2001–2009) | Late (2010–2017) | Total | p-value | |
|---|---|---|---|---|---|---|
| Original articles | 20 | 24 | 41 | 85 | ||
| Blinding | 2 (10.0) | 7 (29.2) | 13 (31.7) | 22 (25.9) | 0.09 | |
| Concealment of allocation | 0 (0.0) | 1 (4.2) | 7 (17.1) | 8 (9.4) | 0.02 | |
| Multicenter | 0 (0.0) | 1 (4.2) | 6 (14.6) | 7 (8.2) | 0.04 | |
| Funding | 0 (0.0) | 0 (0.0) | 4 (9.8) | 4 (4.7) | 0.06 | |
| IRB | 0 (0.0) | 9 (37.5) | 35 (85.4) | 44 (51.8) | <0.01 | |
| Intervention | 19 (95.0) | 23 (95.8) | 41 (100.0) | 83 (97.6) | 0.19 | |
| Jadad scale | ||||||
| Score | 1.20±0.52 | 1.63±0.92 | 2.15±1.09 | 1.78±1.00 | <0.01 | |
| High quality | 1 (5.0) | 5 (20.8) | 17 (41.5) | 23 (27.1) | <0.01a | |
| VTS | ||||||
| Score | 4.05±0.95 | 4.88±1.08 | 5.39±1.46 | 4.93±1.35 | <0.01 | |
| High quality | 5 (25.0) | 12 (50.0) | 28 (68.3) | 45 (52.9) | <0.01a | |
| CCRBT | ||||||
| High risk | 18 (90.0) | 17 (70.8) | 26 (63.4) | 61 (71.8) | 0.02a | |
| Moderate risk | 2 (10.0) | 7 (29.2) | 11 (26.8) | 20 (23.5) | ||
| Low risk | 0 (0.0) | 0 (0.0) | 4 (9.8) | 4 (4.7) | ||
Values are presented as number only, number (%), or mean±standard deviation.
RCT: randomized controlled trial, IRB: Institutional Review Board, VTS: van Tulder scale, CCRBT: Cochrane Collaboration Risk of Bias Tool.
Analysis of variance (ANOVA) test and achi-square test were used.
2. Qualitative changes in RCTs over time
1) Jadad quality assessment scale
From 1979 to 2017, the average Jadad scale score of the RCTs was 1.78±1.00: 1.20±0.52 in early the early period, 1.63±0.92 in the mid period, and 2.15±1.90 in the late period, showing a steady increase in scores (p<0.01). In addition, the number of high-quality studies was 1 (5.0%) in the early period, 5 (20.8%) in the mid period, and 17 (41.5%) in the late period (p<0.01). The total number of high quality RCTs was 23 (27.1%) (Table 1).
2) van Tulder scale
The average scores in the early, mid, and late periods increased to 4.05±0.95, 4.88±1.08, and 5.39±1.46, respectively (p<0.01). The number of high-quality studies was 5 (25.0%), 12 (50.0%), and 28 (68.3%) in the early, mid, and late periods, respectively (p<0.01), while the total number of high-quality studies was 45 (52.9%; p<0.01).
3) Cochrane Collaboration Risk of Bias Tool
The numbers of low risk of bias RCTs on the CCRBT assessment were 0 (0.0%), 0 (0.0%), and 4 (9.8%) in the early, mid, and late period, respectively (p<0.01; Table 1).
3. Analysis of factors related to study quality
When analyzing Jaded scale and VTS scores, the number of high-quality RCTs with funding and IRB approval was significantly higher. On the CCRBT analysis, the number of RCTs with a low risk of bias was also significantly higher as well when funding and IRB approval were performed (Table 2).
Table 2. Factors associated with quality of RCTs.
| Factor | No. of RCTs | Jadad scale | van Tulder scale | Cochrane Collaboration Risk of Bias Tool | |||||
|---|---|---|---|---|---|---|---|---|---|
| Score | High quality | Score | High quality | High risk | Moderate risk | Low risk | |||
| Funding source | |||||||||
| Yes | 4 (4.7) | 3.25±0.50 | 4 (100.0) | 7.25±1.26 | 4 (100.0) | 0 (0.0) | 3 (75.0) | 1 (25.0) | |
| No | 81 (95.3) | 1.70±0.97 | 19 (23.5) | 4.81±1.26 | 41 (50.6) | 61 (75.3) | 17 (21.0) | 3 (3.7) | |
| p-value | <0.01 | <0.01a | <0.01 | 0.05a | <0.01a | ||||
| Reviewed by IRB | |||||||||
| Yes | 44 (51.8) | 2.23±1.03 | 19 (43.2) | 5.61±1.37 | 33 (75.0) | 23 (52.3) | 17 (38.6) | 4 (9.1) | |
| No | 41 (48.2) | 1.29±0.87 | 4 (9.8) | 4.20±0.87 | 12 (29.3) | 38 (92.7) | 3 (7.3) | 0 (0.0) | |
| p-value | <0.01 | <0.01a | <0.01 | <0.01a | <0.01a | ||||
| Allocation | |||||||||
| Yes | 8 (9.4) | 2.88±0.99 | 6 (75.0) | 7.25±1.04 | 8 (100.0) | 4 (50.0) | 3 (37.5) | 1 (12.5) | |
| No | 77 (90.6) | 1.66±0.94 | 17 (22.1) | 4.69±1.14 | 37 (48.1) | 0 (0.0) | 17 (22.1) | 60 (77.9) | |
| p-value | <0.01 | <0.01a | <0.01 | <0.01a | <0.01a | ||||
| Intervention | |||||||||
| Yes | 83 (97.6) | 1.80±1.01 | 23 (27.7) | 4.96±1.35 | 45 (54.2) | 59 (71.1) | 20 (24.1) | 4 (4.8) | |
| No | 2 (2.4) | 1.00±0.00 | 0 (0.0) | 3.50±0.71 | 0 (0.0) | 2 (100.0) | 0 (0.0) | 0 (0.0) | |
| p-value | 0.27 | 0.38a | 0.13 | 0.13a | 0.67a | ||||
| Multicenter | |||||||||
| Yes | 7 (8.2) | 1.29±0.49 | 0 (0.0) | 4.43±1.27 | 3 (42.9) | 6 (85.7) | 1 (14.3) | 0 (0.0) | |
| No (single center) | 78 (91.8) | 1.82±1.03 | 23 (29.5) | 4.97±1.36 | 42 (53.8) | 55 (70.5) | 19 (24.4) | 4 (5.1) | |
| p-value | 0.18 | 0.09a | 0.31 | 0.58a | 0.66a | ||||
Values are presented as number (%) or mean±standard deviation.
RCT: randomized clinical trial, IRB: Institutional Review Board.
Student's t-test and achi-square test were used.
4. Assessment of RCT quality according to varicocele treatment parameters
With regard to varicocele treatment distribution, RCT studies on sperm parameters, treatment feasibility, and other variables were published, in that order (Table 3). In the analysis of RCT quality according to varicocele treatment subjects, the mean Jadad scale scores were 1.57±0.89, 1.38±0.81, and 2.85±0.99 for sperm parameter, treatment feasibility, and other, respectively (p<0.01), while the numbers of high-quality RCTs were 7 (16.7%), 2 (9.5%), and 10 (76.9%), respectively (p<0.01). The VTS scores for sperm parameter, treatment feasibility, and other were 4.64±1.19, 4.48±1.33, and 6.15±1.35, respectively. There were also 19 (45.2%), 7 (33.3%), and 11 (84.6%) high-quality RCTs for sperm parameter, treatment feasibility, and other, respectively as assessed using the VTS (p=0.01). The numbers of RCTs assessed by CCBRT as having a low risk of bias for sperm parameter, treatment feasibility, and other were 0 (0.0%), 1 (4.8%), and 2 (15.4%), respectively (p<0.01; Table 3).
Table 3. Assessment of RCT quality according to treatment parameters of varicocele.
| Region | Jadad scale | van Tulder scale | Cochrane Collaboration Risk of Bias Tool | ||||
|---|---|---|---|---|---|---|---|
| Score | High quality | Score | High quality | High risk | Moderate risk | Low risk | |
| Treatment feasibility (n=21) | 1.38±0.81 | 2 (9.5) | 4.48±1.33 | 7 (33.3) | 17 (81.0) | 3 (14.3) | 1 (4.8) |
| Postoperation pain (n=3) | 3.00±0.00 | 3 (100.0) | 6.67±0.58 | 3 (100.0) | 0 (0.0) | 2 (66.7) | 1 (33.3) |
| Sperm parameter (n=42) | 1.57±0.89 | 7 (16.7) | 4.64±1.19 | 19 (45.2) | 36 (85.7) | 6 (14.3) | 0 (0.0) |
| Testis function (n=3) | 1.33±0.58 | 0 (0.0) | 4.67±0.58 | 2 (66.7) | 3 (100.0) | 0 (0.0) | 0 (0.0) |
| Pregnancy (n=3) | 2.00±1.00 | 1 (33.3) | 5.33±0.58 | 3 (100.0) | 2 (66.7) | 1 (33.3) | 0 (0.0) |
| Other (n=13) | 2.85±0.99 | 10 (76.9) | 6.15±1.35 | 11 (84.6) | 3 (23.1) | 8 (61.5) | 2 (15.4) |
| p-value | <0.01 | <0.01 | <0.01 | 0.01 | <0.01 | ||
| Total (n=85) | 1.78±1.00 | 23 (27.1) | 4.93±1.35 | 45 (52.9) | 61 (71.8) | 20 (23.5) | 4 (4.7) |
Values are presented as mean±standard deviation or number (%).
RCT: randomized clinical trial.
Analysis of variance (ANOVA) and chi-square test were used.
5. Distribution of RCTs of varicocele treatments
RCTs on varicocele have reported treatment results, and we have summarized the treatment types used. The most studied treatment methods accounted for 27.4%, followed by open varicocelectomy (23.5%), other methods (16.5%), laparoscopic varicocelectomy (11.8%), and embolization (9.4%) (Fig. 2A). Journals with publications on RCTs related to varicocele included Urology, Journal of Urology, and Andrologia (Fig. 2B).
Fig. 2. Distribution of (A) randomized control trial of treatment and (B) journals publishing randomized control trial in varicocele. JLAST: Journal of Laparoendoscopic & Advanced Surgical Techniques, Int J Surg: International Journal of Surgery, Urol J: Urology Journal, J Androl: Journal of Andrology, Int J Androl: International Journal of Andrology, Asian J Androl: Asian Jornal of Andrology, Can J Urol: Canadian Journal of Urology, Eur Urol: European Urology, Fertil Steril: Fertility and Sterility, Hum Reprod: Human Reproduction, BJU Int: British Journal of Urology International, J Urol: Jornal of Urology.
DISCUSSION
In the quality assessment of RCTs conducted on varicocele revealed that the quality steadily improved from 2007 to 2012 using the Jadad scale, VTS, and CCRBT, and the number of RCTs also steadily improved. RCT quality was higher in cases of IRB approval and funding. Many RCTs on varicocele were associated with sperm parameters, and the most frequently RCTs on varicocele were published in Urology. The number of RCTs increased in most international journals. This increase in RCTs is considered the reason for the growing importance of EBM. Scales et al [18] reported that the number and percentage of RCTs increased over time in journals such as The Journal of Urology, Urology, European Urology, and BJU International from 1996 to 2004. Jo et al [19] found that the number of RCTs increased over time by analyzing RCTs published in the Journal of Sexual Medicine from 2004 to 2012. This study revealed that the number of RCTs published on varicocele has been increasing steadily. Distribution of RCTs of varicocele treatments showed the most studied treatment methods accounted for followed by comparing treatment, open varicocelectomy, and etc. Treatment of varicocele have complication of recurrence. This probably made number one distribution of RCTs of varicocele treatments with comparing treatment.
As the value of EBM in clinical medicine increases, a systematic and scientific approach to RCTs is emphasized since quality is important [20]. In these studies, the CONSORT statement has been used as a guideline for conducting valid RCTs. In addition, in recent years, when authors submit their RCTs to journals, they are supposed to ensure that RCTs are conducted in accordance with the CONSORT statement by completing tasks in a checklist before submitting their manuscripts for publication. A quality assessment is important because it plays an important role in determining whether a study is actually acceptable or further research is necessary. In addition, a quality assessment evaluates the bias occurring in the study process, validity of the study conclusions, and need for further studies. Thus, RCT quality assessment should be conducted [21,22]. In the past, there were limitations in analyzing RCT quality, and only statistical elements were conducted [5,9]. There are many tools for quantitatively assessing RCT quality such as Campell, Moher, Chalmers, Jadad, VTS, Newell's, and CCRBT, but the most accurate and valid tool has yet to be identified [5]. In this study, we used three tools, the Jadad scale, VTS, and CCRBT, which are able to comprehensively analyze the various elements of the CONSORT statement to assess RCT quality. The Jadad scale is simple and easy to evaluate a tool for randomization, double-blinding, and dropout to reduce bias. The VTS and CCRBT contain evaluation items for allocation concealment, which has the advantage of evaluating selection bias which is limited by the Jadad scale. Allocation concealment helps to prevent selection bias during the assignment of patients to the treatment arms.
An RCT quality assessment using these tools was recently conducted in several places. Kim et al [23] analyzed RCT quality in the International Journal of Impotence Research using the Jadad scale, VTS, and CCRBT. Decreases were observed in the mean Jadad scale scores of the RCTs and in the percentage of high-quality RCTs over time. On the VTS, the scale score and the high quality ratio remained constant; on the CCRBT, the ratio of studies with a low risk of bias was on the rise. Overall, efforts to improve quality are needed in the future. Lee et al [24] analyzed the RCTs published in Neurourology and Urodynamics from 1993 to 2012 using the Jadad scale, VTS, and CCRBT and found that the number of RCTs increased but the quality did not improve.
The number of RCTs on varicocele published from 1979 to 2017 has steadily increased. On the Jadad scale, RCT quality improved continuously from the early to late period. On the VTS, the percentage of high-quality RCTs gradually increased from the early to late periods. On the CCRBT, the percentage of studies with a low risk of bias was 0% in the early and mid periods but increased in the late period. This means that the RCT studies on varicocele show gradual quality improvements because of increasing the conditions of CONSORT statement such as double blinding, randomization, concealment of allocation, and etc. When Clifford et al [25] analyzed the association between quality and funding source in 100 RCTs that were published by five high-impact peer-reviewed general medical journals with high impact factors, they reported that they were irrelevant. In contrast, Kim et al [23] reported that RCTs that were financially supported tended to be larger-scale studies that had better organized research designs; thus, the number of high-quality RCTs was higher among those that received funding. In this study, the presence of funding influenced RCT quality, that is, the funded studies had higher quality scores than the non-funded studies. This implies that funding improved the quality of RCTs on varicocele.
In recent years, studies have been published on the relationship between qualitative RCT studies and IRB approval. IRB approval is a step that is recognized for the validity of design and implementation in the planning phase of a study; it is considered an international standard. Due to the importance of IRB approval, almost all journals now request IRB approval for clinical trials. Study quality is better in cases in which IRB approval is obtained [26]. Bridoux et al [27] reported that when RCT studies are approved by an IRB, they tend to be of higher quality. In this study, the quality differences based on IRB approval were also investigated. Of the RCTs on varicocele, the number of studies obtained IRB approvals increased over time, and RCTs with IRB approval had higher quality scores on the Jadad scale and VTS than those without IRB approval. The percentage of high-quality RCTs was also high. IRB approval played a major role in improving varicocele RTC quality. A valid study plan for obtaining IRB approval in an RCT plays a key role in improving the percentage of high-quality studies. Schulz and Grimes [28] reported that, when the allocation was not properly concealed, randomization in clinical research may be compromised, and even with initial randomization, the effect of the intervention could be distorted by as much as 40% [28]. Hewitt et al [29] assessed RCT quality published in the New England Journal of Medicine, BMJ, JAMA, and Lancet and found that 46% of studies inadequately concealed the allocation. Of the RCTs on varicocele, 8 (9.4%) concealed the allocation. RCTs studies that concealed the allocation showed high-quality results on the Jadad scale and VTS. A low percentage of RCT studies concealed the allocation, a major component in improving study quality. Thus, RCT studies including allocation should be performed in the future.
With regard to study limitations, no representative assessment tool is available to qualitatively analyze RCTs; moreover, a single tool that can evaluate all items listed in the CONSORT statement does not exist. The major limitation of RCT for diseases that require surgical or interventional treatment for varicocele is difficult to design with double-blinding, there is no placebo, and there is no same evaluation tools, which is the limitation of the dataset. With definite tool of evaluating quality of RCT could show us restriction. However, here we tried to compensate for these limitations by using the three tools that were the most widely used to evaluate RCT quality. As another limitation of this study, because the research and evaluation were performed manually, the researchers' subjective opinions could have affected the evaluation process. Our results include studies from 40 years ago. It would not be nice to do a quality assessment using research from 40 years ago. The study of the early period does not mean much.
This study is of significance because no studies have evaluated varicocele RCT quality. Suggestions are needed to direct better further studies. Also, varicocele is a disease that causes scrotum pain and infertility, and recently, in treatment of infertility is incresing.
CONCLUSIONS
The number of published RCTs on varicocele increased from 1979 to 2017. RCT quality improved over time with increases in blinding, IRB approval, funding, multicenter design, and allocation concealment. Most RCTs on varicocele focused on surgical treatments, and although they have limitations to satisfy the condition of RCT to conduct, their quality has improved over time.
Footnotes
Conflict of Interest: The authors have nothing to disclose.
- Conceptualization: SWL.
- Data curation: KSK, HJP.
- Formal analysis: JHC.
- Funding acquisition: none.
- Investigation: KSK.
- Methodology: JHC, WJS.
- Project administration: SWL, WJS.
- Resources: KSK, BHL.
- Software: KSK, JHC, BHL.
- Supervision: JHC.
- Validation: SWL.
- Visualization: KSK.
- Writing — original draft: KSK.
- Writing — review & editing: KSK, SWL.
Data Sharing Statement
The data that support the findings of this study are available from the corresponding author upon reasonable request.
Supplementary Material
Supplementary material can be found via https://doi.org/10.5534/wjmh.200167.
List of randomized controlled trials of varicocele
References
- 1.Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ. 1996;312:71–72. doi: 10.1136/bmj.312.7023.71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Uetani K, Nakayama T, Ikai H, Yonemoto N, Moher D. Quality of reports on randomized controlled trials conducted in Japan: evaluation of adherence to the CONSORT statement. Intern Med. 2009;48:307–313. doi: 10.2169/internalmedicine.48.1358. [DOI] [PubMed] [Google Scholar]
- 3.Altman DG. Better reporting of randomised controlled trials: the CONSORT statement. BMJ. 1996;313:570–571. doi: 10.1136/bmj.313.7057.570. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Gluud C, Nikolova D. Likely country of origin in publications on randomised controlled trials and controlled clinical trials during the last 60 years. Trials. 2007;8:7. doi: 10.1186/1745-6215-8-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA. 1996;276:637–639. doi: 10.1001/jama.276.8.637. [DOI] [PubMed] [Google Scholar]
- 6.Featherstone K, Donovan JL. Random allocation or allocation at random? Patients' perspectives of participation in a randomised controlled trial. BMJ. 1998;317:1177–1180. doi: 10.1136/bmj.317.7167.1177. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al. CONSORT. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. Int J Surg. 2012;10:28–55. doi: 10.1016/j.ijsu.2011.10.001. [DOI] [PubMed] [Google Scholar]
- 8.Jüni P, Altman DG, Egger M. Systematic reviews in health care: assessing the quality of controlled clinical trials. BMJ. 2001;323:42–46. doi: 10.1136/bmj.323.7303.42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Gagnier J, Boon H, Rochon P, Barnes J, Moher D, Bombardier C CONSORT Group. Improving the quality of reporting of randomized controlled trials evaluating herbal interventions: implementing the CONSORT statement [corrected] Explore (NY) 2006;2:143–149. doi: 10.1016/j.explore.2005.12.003. [DOI] [PubMed] [Google Scholar]
- 10.Clark HD, Wells GA, Huët C, McAlister FA, Salmi LR, Fergusson D, et al. Assessing the quality of randomized trials: reliability of the Jadad scale. Control Clin Trials. 1999;20:448–452. doi: 10.1016/s0197-2456(99)00026-4. [DOI] [PubMed] [Google Scholar]
- 11.Simforoosh N, Ziaee SA, Behjati S, Beygi FM, Arianpoor A, Abdi H. Laparoscopic management of varicocele using bipolar cautery versus open high ligation technique: a randomized, clinical trial. J Laparoendosc Adv Surg Tech A. 2007;17:743–747. doi: 10.1089/lap.2006.0174. [DOI] [PubMed] [Google Scholar]
- 12.Diagnosis and treatment of early melanoma. NIH Consensus Development Conference. January 27-29, 1992. Consens Statement. 1992;10:1–25. [PubMed] [Google Scholar]
- 13.Diegidio P, Jhaveri JK, Ghannam S, Pinkhasov R, Shabsigh R, Fisch H. Review of current varicocelectomy techniques and their outcomes. BJU Int. 2011;108:1157–1172. doi: 10.1111/j.1464-410X.2010.09959.x. [DOI] [PubMed] [Google Scholar]
- 14.Armağan A, Ergün O, Bağ E, Oksay T, Koşar A. Long-term effects of microsurgical varicocelectomy on pain and sperm parameters in clinical varicocele patients with scrotal pain complaints. Andrologia. 2012;44 Suppl 1:611–614. doi: 10.1111/j.1439-0272.2011.01238.x. [DOI] [PubMed] [Google Scholar]
- 15.Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary. Control Clin Trials. 1996;17:1–12. doi: 10.1016/0197-2456(95)00134-4. [DOI] [PubMed] [Google Scholar]
- 16.van Tulder M, Furlan A, Bombardier C, Bouter L Editorial Board of the Cochrane Collaboration Back Review Group. Updated method guidelines for systematic reviews in the cochrane collaboration back review group. Spine (Phila Pa 1976) 2003;28:1290–1299. doi: 10.1097/01.BRS.0000065484.95996.AF. [DOI] [PubMed] [Google Scholar]
- 17.Higgins JPT, Green S. Cochrane handbook for systematic reviews of interventions version 5.1.0 [Internet] London: The Cochrane Collaboration; c2011. [cited 2019 Dec 7]. Available from: https://handbook-5-1.cochrane.org/ [Google Scholar]
- 18.Scales CD, Jr, Norris RD, Keitz SA, Peterson BL, Preminger GM, Vieweg J, et al. A critical assessment of the quality of reporting of randomized, controlled trials in the urology literature. J Urol. 2007;177:1090–1094. doi: 10.1016/j.juro.2006.10.027. discussion 1094–5. [DOI] [PubMed] [Google Scholar]
- 19.Jo JK, Chung JH, Kim KS, Lee JW, Lee SW. Randomized controlled trials in the journal of sexual medicine: a quality assessment and relevant clinical impact. J Sex Med. 2014;11:894–900. doi: 10.1111/jsm.12455. [DOI] [PubMed] [Google Scholar]
- 20.Sutton AJ, Cooper NJ, Jones DR. Evidence synthesis as the key to more coherent and efficient research. BMC Med Res Methodol. 2009;9:29. doi: 10.1186/1471-2288-9-29. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Chalmers TC, Celano P, Sacks HS, Smith H., Jr Bias in treatment assignment in controlled clinical trials. N Engl J Med. 1983;309:1358–1361. doi: 10.1056/NEJM198312013092204. [DOI] [PubMed] [Google Scholar]
- 22.Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273:408–412. doi: 10.1001/jama.273.5.408. [DOI] [PubMed] [Google Scholar]
- 23.Kim KS, Jo JK, Chung JH, Kim JH, Choi HY, Lee SW. Quality analysis of randomized controlled trials in the International Journal of Impotence Research: quality assessment and relevant clinical impact. Int J Impot Res. 2017;29:65–69. doi: 10.1038/ijir.2016.48. [DOI] [PubMed] [Google Scholar]
- 24.Lee JW, Chung JH, Jo JK, Lee SW. Assessing the quality of randomized controlled trials published in neurourology and urodynamics from 1993 to 2012. Neurourol Urodyn. 2014;33:472–474. doi: 10.1002/nau.22457. [DOI] [PubMed] [Google Scholar]
- 25.Clifford TJ, Barrowman NJ, Moher D. Funding source, trial outcome and reporting quality: are they related? Results of a pilot study. BMC Health Serv Res. 2002;2:18. doi: 10.1186/1472-6963-2-18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Kim KS, Chung JH, Jo JK, Kim JH, Kim S, Cho JM, et al. Quality of randomized controlled trials published in the International Urogynecology Journal 2007-2016. Int Urogynecol J. 2018;29:1011–1017. doi: 10.1007/s00192-017-3465-6. [DOI] [PubMed] [Google Scholar]
- 27.Bridoux V, Moutel G, Roman H, Kianifard B, Michot F, Herve C, et al. Methodological and ethical quality of randomized controlled clinical trials in gastrointestinal surgery. J Gastrointest Surg. 2012;16:1758–1767. doi: 10.1007/s11605-012-1952-0. [DOI] [PubMed] [Google Scholar]
- 28.Schulz KF, Grimes DA. Allocation concealment in randomised trials: defending against deciphering. Lancet. 2002;359:614–618. doi: 10.1016/S0140-6736(02)07750-4. [DOI] [PubMed] [Google Scholar]
- 29.Hewitt C, Hahn S, Torgerson DJ, Watson J, Bland JM. Adequacy and reporting of allocation concealment: review of recent trials published in four general medical journals. BMJ. 2005;330:1057–1058. doi: 10.1136/bmj.38413.576713.AE. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
List of randomized controlled trials of varicocele
Data Availability Statement
The data that support the findings of this study are available from the corresponding author upon reasonable request.