Abstract
Purpose
The selection of suitable outcomes and sample size calculation are critical factors in the design of a randomised controlled trial (RCT). The goal of this study was to identify the range of outcomes and information on sample size calculation in RCTs on geographic atrophy (GA).
Methods
We carried out a systematic review of age-related macular degeneration (AMD) RCTs. We searched MEDLINE, EMBASE, Scopus, Cochrane Library, www.controlled-trials.com, and www.ClinicalTrials.gov. Two independent reviewers screened records. One reviewer collected data and the second reviewer appraised 10% of collected data. We scanned references lists of selected papers to include other relevant RCTs.
Results
Literature and registry search identified 3816 abstracts of journal articles and 493 records from trial registries. From a total of 177 RCTs on all types of AMD, 23 RCTs on GA were included. Eighty-one clinical outcomes were identified. Visual acuity (VA) was the most frequently used outcome, presented in 18 out of 23 RCTs and followed by the measures of lesion area. For sample size analysis, 8 GA RCTs were included. None of them provided sufficient Information on sample size calculations.
Conclusions
This systematic review illustrates a lack of standardisation in terms of outcome reporting in GA trials and issues regarding sample size calculation. These limitations significantly hamper attempts to compare outcomes across studies and also perform meta-analyses.
Introduction
Geographic atrophy (GA) is an advanced manifestation of age-related macular degeneration (AMD)—the leading cause of severe vision loss for people over 55 years old in the industrialised world.1 Geographic atrophy accounts for ~20% of legal blindness registration in North America.1 It is associated with formation of large, confluent drusen, degeneration of retinal epithelium and loss of photoreceptors eventually leading to irreversible sight loss.1 Although some progress has been made to understand the pathophysiology of GA and there is no treatment to prevent progression of GA,1 a number of randomised controlled trials (RCTs) testing potential interventions have taken place.2
RCTs are considered the most powerful tools when determining the efficacy of interventions.3 However, poor planning of RCTs may result in substandard studies. To enable readers to access methodological rigour of an RCT, reports of trials should include transparent descriptions of key information.4 One of the critical aspects for planning an RCT is the selection of appropriate primary and secondary outcomes.5 It has been recognised that there is a large variability in outcome selection in RCTs for different diseases.6 This variability results in difficulties in synthesising medical evidence7 and a higher risk of outcome reporting bias.8 Standardisation of outcomes solves this problem by creating a core outcome set (COS) for a specific disease. This group of outcomes, which should be reported in each interventional trial within a particular area, would reduce inconsistencies in outcome reporting between studies and improve design of RCTs.9 This concept has been adapted in several fields of medicine such as rheumatology, gastroenterology, and paediatrics.10
The selection of outcomes in an RCT has also critical implications for the sample size calculation. Sample size estimation is typically based upon the primary outcome. The sample size should be suitable to answer with enough power the research question regarding efficacy of an intervention. It also ought to constitute a guarantee that a study is informative.11 However, insufficient attention has been paid to sample size estimations in some areas of vision research.12
Despite dry AMD being the most prevalent form of AMD, such major breakthrough as anti-VEGF (vascular endothelial growth factor) drugs for neovascular AMD has not been yet introduced.2 Potential GA therapeutic options have just stepped in the phase of clinical randomised trials.2 Thus, it gives a good opportunity to reflect on what were the main outcomes selected and how the sample size was determined in recent GA trials.
The aim of this paper is to identify the variety of outcomes and reporting of sample size calculations within GA trials.
Materials and methods
In our systematic review, we adhered to Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines (Transparent Reporting in Systematic Reviews and Meta-analyses, www.prisma-statement.org). We registered the systematic review in PROSPERO (International prospective register of systematic reviews, www.crd.york.ac.uk/PROSPERO/) with registration number: CRD42014010040.
Search strategy
A systematic review was conducted because of recent developments in AMD area of research. We limited our search to RCTs published after January 2010. The last search date was November 2013. The following databases were searched: Ovid MEDLINE Full, Ovid EMBASE, SCOPUS, and Cochrane Library. We ran a modified sensitive search strategy with subject subheadings and text terms. In order to find eligible abstracts, we examined two types of terms: AMD or AMD synonyms (for instance: ARMD, maculae senilis) and RCTs and words associated with RCTs (eg, randomly, single-blind, double-blind study). An ‘AND' operator was used to merge the results from each area. The search strategies from the different electronic databases are presented in Supplementary Appendix 1. We exported and combined the yielded abstracts in Refworks. In addition, the clinical trials' registries (www.controlled-trials.com and www.clinicaltrials.gov) were searched. A keyword ‘macular degeneration' limited to the same time period was used to attain the required trials' records.
Inclusion and exclusion criteria
Two investigators independently screened all abstracts. We accepted only RCTs on AMD published in English. RCTs from references and appendices of Cochrane Systematic Reviews were also searched and identified. All records were grouped as ‘include', ‘uncertain', or ‘exclude'. We discussed any inconsistencies between the reviewers in categorising records to reach an agreement. Full articles of those classified as ‘Uncertain' and ‘include' were obtained and then a final decision of inclusion was made.
Data extraction strategy
Data were captured using online forms (www.maculaesenilis.com), specifically designed for this project. The information on the type of AMD, outcome (clinical, patient reported outcome, safety, or other) and the category of outcome (primary or secondary) was collected by one investigator. Information on sample size calculation and population of identified RCTs was also congregated. One tenth of included RCTs was selected at random and evaluated by a masked second investigator to evaluate agreement of data extracted.
On-going RCTs and those from appendices of Cochrane Reviews did not have information on the sample size calculation.
Results
Our search identified 3816 abstracts. From the abstracts, 103 RCTs were included. The additional trials' registries search found 493 records. Of these, 74 RCTs were included. A combination of both searches resulted in a total number of 177 RCTs of all AMD types. The number of identified GA trials was 23. (Figure 1, PRISMA flow chart). Of these, eight had published results, including study design, while information on 15 trials was obtained from the trial registries and appendices of Cochrane Systematic Reviews. Masked investigators agreed on identification of the primary outcomes measures in 100% of articles assessed. However, there was one RCT, where the first reviewer recognised outcome information as one primary outcome, while the second one assessed it as two outcomes—one primary and ‘other primary outcome'. This discrepancy was resolved after discussion when reviewers agreed to classify retrieved information as one primary outcome.
Figure 1.
Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flow diagram illustrating the process of randomised controlled trials (RCTs) selection. Asterisk denotes additional potential RCTs from references of the articles and appendixes of the Cochrane Systematic Reviews.
Primary outcomes were identified in 21 GA RCTs (91.3%), whereas in two RCTs (from appendices of Cochrane Systematic Reviews), they were not clearly stated. Three studies reported more than one primary outcome. In two trials (from the trial registries), two primary outcomes were identified—‘original primary' and ‘current primary'—and in one trial, different biomarkers were used as main outcomes.
In six RCTs, lesion area and visual acuity (VA) were used as a primary outcome (26.1%). It was followed by fundus autofluorescence (five RCTs, 21.74%). There was variability in the way that the primary outcomes were measured (Table 1).
Table 1. Primary outcomes in geographic atrophy randomised controlled trials.
| Primary outcomea | Number of outcomes for GA RCTs | Examples of outcome description |
|---|---|---|
| Visual acuity | 6 | ETDRS change in BCVA at 12 months, change in BCVA at 24 months, increase in VA at week 4, loss of 3 or more lines of VA |
| Lesion area | 6 | Change in the total GA area from baseline to 24 months, rate of change in area of GA from baseline, rate of enlargement in area of GA in the study eye during the treatment period, annualised lesion growth rates |
| Autofluorescence | 5 | The mean reduction in the rate of growth of GA area, growth of GA lesions from baseline to month 6, change in area of atrophy from baseline to week 48 |
| Biomarkers | 4 | Plasma xanthophyll concentration, fatty acids profile, reological markers |
| Adverse and safety outcomes | 2 | Number of eyes with serious ocular adverse events occurring over the first 12 months of the study, time between baseline treatment and need to escape to standard of care |
| Electroretinogram | 1 | Demonstration of the short-term multifocal electroretinogram effect |
| Fundus photography | 1 | Rate of change in area of GA at 2 years compared with baseline |
Abbreviations: BCVA, best corrected visual acuity; ETDRS, The Early Treatment Diabetic Retinopathy Study Eye Charts; GA, geographic atrophy; RCT, randomised controlled trial; VA, visual acuity.
Number of RCTs does not have to be the same as number of primary outcomes as each RCT may have multiple primary outcomes.
Overall, a total of 81 clinical outcomes (including primary and secondary) were reported (71.7% of all 113 GA outcomes). VA was the most common outcome (reported 24 times, 29.6% of all clinical outcomes). It was used in 18 GA RCTs (78.3%). There was heterogeneity in the methods used to analyse and report visual acuity in GA trials (some examples are shown in Table 1).
The second most common type of outcome was lesion area and this was reported in eight RCTs (34.8%). Anatomical outcomes were also described in a variety of ways depending on the imaging modality or method of assessment used. In general, GA RCTs focused on drusen and geographic atrophy lesion size (Table 2). Autofluorescence, fundus photography, and OCT measures were the most frequently used methods of anatomical changes assessment.
Table 2. Descriptions of lesion area in geographic atrophy randomised controlled trials.
| GA lesion size measured in fundus photography |
| Total drusen area from baseline |
| Change in the total GA area from baseline |
| Change in drusen area measured by colour fundus photography |
| Lesion size graded on fluorescein angiography |
| Increase from baseline in area of GA |
| Mean change from baseline in GA lesion size |
| Rate of change in area of GA |
| GA lesion growth rates in fundus photography |
| Growth rate of GA lesion area from baseline |
| Growth of GA lesions as measured in fundus autofluorescence |
| Rate of enlargement in area of GA |
Detailed information on the most common types of clinical outcomes and their frequency in GA trials is presented in Table 3.
Table 3. Types and frequency of clinical outcomes (primary and secondary) in geographic atrophy randomised controlled trials.
| Type of clinical outcome | Number of GA RCTs (%) N=23 |
|---|---|
| Visual acuity | 18 (78.3) |
| Lesion area | 8 (34.8) |
| Contrast sensitivity | 6 (26.1) |
| Fundus photography | 5 (21.7) |
| Retinal sensitivity | 4 (17.4) |
| Fundus autofluorescence | 4 (17.4) |
| OCT measurements | 3 (13) |
| Progression and stage of AMD | 3 (13) |
Abbreviations: AMD, age-related macular degeneration; GA, geographic atrophy; OCT, optical coherence tomography; RCT, randomised controlled trial.
Patient reported outcomes were used in two GA trials. The questionnaires used were the Visual Functioning Questionnaire-14 (VFQ-14) and the National Eye Institute VFQ-25.
Regarding sample size calculation, of the eight GA RCTs from the database search with details of study design, none of the trials included relevant information according to the CONSORT statement. In one trial, information on sample size was limited to the statement that Lehr formula was used to determine a sample size. Another trial described that target number of participants (covered by the grant) was reached.
Discussion
A selection of appropriate outcomes is a critical step in the design of an RCT. To facilitate interpretation of results and comparisons between studies, interventional trials on the same condition should share a standard component known as COS.7 This systematic review shows that there is currently no standardisation in the selection of outcomes in GA RCTs.
Although effective treatments for GA remain elusive, various promising therapeutics for GA are currently in the pipeline and in the years to come more and more trials for GA are likely to be instigated.2 Visual function outcomes other than visual acuity, such as contrast sensitivity or dark adaptation,13 have been considered for use in GA RCTs, as VA measures in isolation may be insensitive to slow progression of GA.13 Progression, anatomic features, size of atrophy were among the most commonly used outcomes for GA.
Other scientists and clinicians have acknowledged this issue of optimal outcome selection and trial design.14 Our study also demonstrates that a broad diversity exists in the methods of reporting outcomes in GA RCTs.15, 16 The most frequently used outcome in GA RCTs was VA, yet it differed widely in what metric was used as the focus of the data analysis. Some studies used the change in number of letters read, others determined the proportion of participants achieving a particular criterion such as a specific number of lines gained or lost. This diversity in how VA data are analysed is likely to make comparisons between different trials problematic. To help in planning evidence synthesis and analysis of visual acuity measures, attention should be drawn to the association between different ways of reporting VA measures.17
Despite the fact that AMD is known as a disease that causes both, visual impairment, and decrease in quality of life associated with limitations in everyday living and depression,18 we have observed that only two GA RCTs have used patient-reported outcome measures (PROMs). Ideally outcomes relevant to patients and PROMs should be used in all AMD trials.19
The unambiguous selection of relevant outcomes during the design of interventional trials is essential, because it allows health care providers to compare scientific evidence and then make well-informed decisions about treatment,20 which in turn enables the efficient development of healthcare systems.9 A useful aid for researchers to select appropriate outcomes for their RCTs is creating a COS. The COMET initiative has popularised this idea since 2010.7 This must-include minimum set of outcomes for RCTs on a particular condition would reduce heterogeneity in outcome reporting, facilitate comparisons between studies, and decrease the level of outcome reporting bias.7, 9 Previous studies have confirmed the need for COS in such chronic ophthalmologic condition as glaucoma,21 whereas our review verifies the need for consensus in outcome selection and type of measurement for GA AMD trials.
The need for accurate sample size calculations has been increasingly acknowledged in recent years12 as it allows achieving power sufficient to answer the main research question of a study22 and promotes meaningful research that does not waste resources or expose participants to potentially harmful interventions without advancing knowledge.23 Yet, our review has shown that information on sample size calculation in GA RCTs is limited. This deficiency in sample size calculation may impair interpretation of study results. Failure in reaching planned sample size makes results of a study less reliable and delays implementation of results in clinical practice. Determination of statistical significance that a study is designed to detect is another key consideration of sample size calculation, as incorrectly specified target difference may impair decision making.11 To avoid that, a method tailored to the aim, context, and statistical framework of a study should be implemented to define a target difference.24 Taking into account the importance of sample size calculation and the relevant information collected on GA RCTs, we confirm the observation that some vision research has struggled with this area.12 However, we are aware that some trials are designed to generate the pilot data, which may be used to inform sample size calculation in larger and definitive RCTs.
Overall, reporting of ophthalmological RCTs has improved since the CONSORT (Consolidated Standards for Reporting Trials) statement was introduced.25 The CONSORT statement includes items regarding a title, an introduction, methods, results, discussion, and others to guide reporting of RCTs.4 Our interest in primary/secondary outcome selection and sample size are important elements of the methods section of the CONSORT statement. In this review, we have observed that primary outcomes for dry AMD trials were clearly stated and there was a range of secondary outcomes described. However, the observed variability of time, metrics, and the way of assessment of outcomes would require further attention. Regarding sample size calculation, improvements of reporting of this aspect are needed to improve standard of publications and research.
Our review was prepared according to PRISMA recommendations (www.prisma-statement.org) in a systematic way and it includes RCTs on GA population. A potential limitation is that we took into consideration only published and on-going RCTs and those within a limited time frame. It is also plausible that some eligible RCTs were not included as we did not seek the ‘grey literature' and we limited our search to English language only. However, our approach was sufficient to notice deficiencies and variability of current GA trials' design. The development of a COS for the area of geographic atrophy is a great opportunity to improve standards of RCTs planning and reporting, and facilitate direct comparisons between interventions.
In summary, our systematic review clearly demonstrated outcomes and ways of measuring them used in the reporting of GA RCTs vary widely. We recommend that the ophthalmic community should focus on developing a COS for future AMD RCTs with potentially separate sets developed for neovascular AMD, GA, and early AMD. Through our review, we also wish to draw researchers' attention to the need for explicitly stating the a priori sample size calculation in the reporting of GA RCTs.
The authors declare no conflict of interest.
Footnotes
Supplementary Information accompanies this paper on Eye website (http://www.nature.com/eye)
Supplementary Material
References
- Holz FG, Strauss EC, Schmitz-Valckenberg S, Campagne van Lookeren M. Geographic atrophy. Clinical features and potential therapeutic approaches. Ophthalmology 2014; 121: 1079–1091. [DOI] [PubMed] [Google Scholar]
- Velez-Montoya R, Oliver SC, Olson JL, Fine SL, Mandava N, Quiroz-Mercado H. Current knowledge and trends in age-related macular degeneration: today's and future treatments. Retina 2013; 33(8): 1487–1502. [DOI] [PubMed] [Google Scholar]
- Sibbald B, Roland M. Understanding controlled trials: why are randomised controlled trials important? BMJ 1998; 316(7126): 201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schulz KF, Altman DG, Moher D. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol 2010; 63: 834–840. [DOI] [PubMed] [Google Scholar]
- Tetzlaff J, Chan A, Kitchen J, Sampson M, Tricco AC, Moher D. Guidelines for randomized clinical trial protocol content: a systematic review. Syst Rev 2012; 1: 43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tugwell P, Boers M, Brooks P, Simon L, Strand V, Idzerda L. OMERACT: an International Initiative to improve Outcome Measurement in Rheumatology. Trials 2007; 8: 38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Comet Initiativehttp://www.comet-initiative.org.
- Dwan K, Altman DG, Arnaiz JA et al. Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias. PLoS One 2008; 3: e3081. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Williamson P, Altman D, Blazeby J, Clarke M, Devane D, Gargon E et al. Developing core outcome sets for clinical trials: issues to consider. Trials 2012; 13: 132. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Clarke M. Standardising Outcomes in Paediatric Clinical Trials. PLoS Med 2008; 5: e102. [Google Scholar]
- Cook J, Hislop J, Adewuyi T, Harrild K, Altman DG, Ramsay CR et al. Assessing methods to specify the target difference for a randomised controlled trial: DELTA (Difference ELicitation in TriAls) review. Health Technol Assess 2014; 18(28)v-vi 1–175. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Goodall EA, Moore J, Moore T. The estimation of approximate sample size requirements necessary for clinical and epidemiological studies in vision sciences. Eye 2009; 23: 1589–1597. [DOI] [PubMed] [Google Scholar]
- Lesmes LA, Jackson ML, Bex P. Visual function endpoints to enable dry AMD clinical trials. Drug Discov Today Ther Strat 2013; 10: e43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Csaky KG, Richman EA, Ferris FL III. Report from the NEI/FDA ophthalmic clinical trial design and endpoints symposium. Invest Ophthalmol Visual Sci 2008; 49: 479–489. [DOI] [PubMed] [Google Scholar]
- Beck R, Maguire M, Bressler N, Glassman AR, Lindblad AS, Ferris FL. Visual acuity as an outcome measure in clinical trials of retinal diseases. Ophthalmology 2007; 114: 1804–1809. [DOI] [PubMed] [Google Scholar]
- Wittes J, Downs M. Outcome Measures to Assess Efficacy of Treatments for Age-Related Macular Degeneration. Ophthalmology 2009; 116: S8–S14. [DOI] [PubMed] [Google Scholar]
- Burzykowski T, Buyse M. The correlation structure of longitudinal measurements of vision in patients with macular degeneration. Pharm Stat 2011; 10: 115–121. [DOI] [PubMed] [Google Scholar]
- Slakter JS, Stur M. Quality of Life in Patients With Age-related Macular Degeneration: Impact of the Condition and Benefits of Treatment. Surv Ophthalmol 2005; 50(3): 263–273. [DOI] [PubMed] [Google Scholar]
- Krezel AK, Hogg RE, Azuara-Blanco A. Patient-reported outcomes in randomised controlled trials on age-related macular degeneration. Br J Ophthalmol 2015; e-pub ahead of print 30 April 2015; doi: 10.1136/bjophthalmol-2014-306544. [DOI] [PubMed]
- Williamson P, Altman D, Blazeby J, Clarke M, Gargon E. Driving up the quality and relevance of research through the use of agreed core outcomes. J Health Serv Res Policy 2012; 17: 1–2. [DOI] [PubMed] [Google Scholar]
- Ismail R, Azuara-Blanco A, Ramsay CR. Variation of clinical outcomes used in glaucoma randomised controlled trials: a systematic review. Br J Ophthalmol 2014; 98: 464–468. [DOI] [PubMed] [Google Scholar]
- Zhong B. How to calculate sample size in randomized controlled trial? J Thorac Dis 2009; 1(1): 51–54. [PMC free article] [PubMed] [Google Scholar]
- Lenth R. Some practical guidelines for effective sample size determination. Am Stat 2001; 55: 187–193. [Google Scholar]
- Hislop J, Adewuyi TE, Vale LD, Harrild K, Fraser C, Gurung T et al. Methods for specifying the target difference in a randomised controlled trial: The Difference ELicitation in TriAls (DELTA) systematic review. PLoS Medicine 2014; 11: e1001645. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fung AE, Palanki R, Bakri SJ, Depperschmidt E, Gibson A. Applying the CONSORT and STROBE statements to evaluate the reporting quality of neovascular age-related macular degeneration studies. Ophthalmology 2009; 116: 286–296 e4. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.