Abstract
Background
Atrial fibrillation (AF) is the arrhythmia most commonly diagnosed in clinical practice. It is associated with significant morbidity and mortality. Prevalence of AF and complications of AF, estimated by hospitalisations, have increased dramatically in the last decade. Being able to predict AF would allow tailoring of management strategies and a focus on primary or secondary prevention. Models predicting recurrent AF would have particular clinical use for the selection of rhythm control therapy. There are existing prognostic models which combine several predictors or risk factors to generate an individualised estimate of risk of AF. The aim of this systematic review is to summarise and compare model performance measures and predictive accuracy across different models and populations at risk of developing incident or recurrent AF.
Methods
Methods tailored to systematic reviews of prognostic models will be used for study identification, risk of bias assessment and synthesis. Studies will be eligible for inclusion where they report an internally or externally validated model. The quality of studies reporting a prognostic model will be assessed using the Prediction Study Risk Of Bias Assessment Tool (PROBAST). Studies will be narratively described and included variables and predictive accuracy compared across different models and populations. Meta-analysis of model performance measures for models validated in similar populations will be considered where possible.
Discussion
To the best of our knowledge, this will be the first systematic review to collate evidence from all studies reporting on validated prognostic models, or on the impact of such models, in any population at risk of incident or recurrent AF. The review may identify models which are suitable for impact assessment in clinical practice. Should gaps in the evidence be identified, research recommendations relating to model development, validation or impact assessment will be made. Findings will be considered in the context of any models already used in clinical practice, and the extent to which these have been validated.
Systematic review registration
PROSPERO (CRD42018111649).
Keywords: Atrial fibrillation, Incidence, Recurrence, Prognostic model, Risk score, Model validation, Model performance, Systematic review
Background
Atrial fibrillation (AF) is the most common arrhythmia diagnosed in clinical practice, and the worldwide incidence and prevalence are increasing [1]. AF is predicted to affect around 18 million people in Europe by 2060, and 6–12 million in the USA by 2050 [1, 2]. Drivers for the increase in prevalence include an ageing population, better survival from conditions such as ischaemic heart disease, but also increasing multimorbidity [3, 4]. AF is associated with increased morbidity, such as stroke or heart failure, and increased mortality, particularly cardiovascular related [3, 4]. Currently available treatments can reduce the mortality and morbidity associated with AF, particularly via anticoagulation for prevention of strokes [4], but patients with AF remain at high risk of cardiovascular complications even on optimal therapy, often manifesting as heart failure or sudden death [5]. AF also commonly occurs after cardiac surgery, with patients subsequently at increased risk of stroke, congestive heart failure, and haemodynamic instability [6]. Post-operative AF rates have been estimated at up to 60% for cardiothoracic surgery and 5–10% in non-cardiothoracic surgery [7].
Given the growing burden of AF, there has been a recent shift in attention to primary and secondary prevention strategies [8, 9]. It has long been known that approximately 70% of patients experience recurrent AF after, e.g. a cardioversion [10], while a small population of AF patients does not show progression of AF, with rare recurrences over decades [11]. Being able to identify those patients who are most likely to develop recurrent AF, or progress from paroxysmal to sustained AF, would be beneficial for tailoring treatment strategies and implementing targeted preventative measures. There is a range of risk factors associated with the development of AF, the best-established ones being age, hypertension, diabetes mellitus, and heart failure, as well as environmental factors such as smoking and alcohol consumption [12, 13]. Less validated risk factors include subclinical hyperthyroidism, obesity and sleep apnoea syndrome [12]. There may also be a role for biomarkers in assessing AF risk, including serum biomarkers such as brain natriuretic peptide (BNP) [14, 15] or fibroblast growth factor 23 (FGF-23) [14], imaging of atrial function, ECG-based parameters, and genetic factors [12].
A number of prognostic models, or risk scores, which attempt to combine several predictors to generate an individualised estimate of risk in different populations have been developed for AF prediction. These include the Cohorts for Ageing and Research in Genomic Epidemiology (CHARGE)-AF score [16], the HAVOC score [17], the ATLAS score [18], the HATCH score [19], and the APPLE score [20].
Scoping searches for existing and ongoing systematic reviews of prognostic models for AF were carried out in MEDLINE, Embase, the Cochrane Library, and PROSPERO (October 2018). There were several systematic reviews relating to either the association of individual risk factors with AF or focussing on a single AF risk prediction model; only one identified systematic review gave an overview of available risk scores; this had a focus on recurrence following catheter ablation [21]. The review had a limited search strategy and did not include formal quality appraisal of the identified models. More recent primary studies were identified since the publication of this review. There is therefore a need for an up-to-date systematic review, which includes all relevant populations and uses methodologically robust methods for study identification, risk of bias assessment, and synthesis.
Aim
To undertake a systematic review of prognostic models predicting incident or recurrent AF, with meta-analysis of model performance measures where possible.
More specifically to
Identify all studies which report on the development, validation, or impact of a score for predicting AF
Take forward for risk of bias assessment and analysis, all studies reporting validated models (internal and/or external validation)
Summarise and compare model performance measures and predictive accuracy
Undertake meta-analysis of model performance measures where possible
Methods
Study eligibility criteria
Study design
Studies of any design reporting the following types of prognostic modelling will be eligible for inclusion as guided by the CHARMS [22] checklist:
Prediction model development with internal validation
Prediction model development with external validation
External model validation (with or without model updating)
Studies reporting impact assessment of a prognostic model will also be eligible for inclusion.
Studies that have developed a model but not validated this will not be taken forward for risk of bias assessment and analysis, but a record of these studies will be kept.
For the purpose of this review, a prognostic model will be defined as a combination of two or more predictors within a statistical model, which is used to predict an individual’s risk of the outcome [23]. Published and unpublished studies, as well as studies published in abstract form only, will be eligible for inclusion. Studies that have looked at the association between a single risk factor and the development or recurrence of AF will be excluded, as they are limited in their utility for individual risk prediction [24].
Patient group
Any population at risk of incident or recurrent AF will be eligible for inclusion. This will include the general population; people with cardiovascular disease or other comorbidities; people who have undergone surgery and who may be at risk of post-operative AF; and people who have been treated for AF and are at risk of recurrent AF. There will be no restriction on the number or type of previous treatments. People with paroxysmal, persistent, or permanent AF will be eligible, and in this case, models predicting progression from one type of AF to another will be of interest.
Outcomes
The clinical outcome of interest is incident or recurrent AF and/or progression from paroxysmal to persistent or permanent AF. Model performance will be assessed by calibration measures, i.e. how well the predicted risks compare to the observed outcome, and discrimination measures, i.e. how well the model differentiates between those with and without the outcome [25].
Search strategy
The following bibliographic databases will be searched (from inception to November 2018) using combinations of text and index terms relating to the incident or recurrent AF and models: MEDLINE (Ovid), MEDLINE In-Process (Ovid), Embase (Ovid), and Cochrane CENTRAL. The terms relating to the ‘model’ component of the search strategy will be informed by the updates to two existing study design filters as described by Geersing (2012) [26]. Several alternate terms will be used, as a prognostic model may also be described as a prognostic (or prediction) index or rule, risk (or clinical) prediction model, or predictive model [23]. The term ‘clinical score’ will also be used as well as names of known scores identified through scoping searches. Searches will be updated shortly before project completion to ensure the systematic review is up-to-date at the time of submission. See Appendix for sample search strategy in Embase.
There will be no date or language restrictions. Reference lists of included articles and relevant reviews will be checked and subject experts consulted. ClinicalTrials.gov and the WHO International Clinical Trials Registry Platform will be searched for ongoing studies. The Conference Proceedings Citation Index will be searched for conference abstracts.
Study selection
All identified studies will be screened independently by two reviewers (JD, NC) using predefined screening criteria, with disagreements resolved through discussion or referral to a third reviewer (YT). A sample of records will be screened by two reviewers to pilot and amend the screening criteria if necessary, before screening the remainder. Full texts will be screened where necessary. Part or full-text translation will be undertaken where necessary to make selection decisions. The screening process will be facilitated by the use of reference management software (EndNote X7) and the selection process documented using a PRISMA [27] flow diagram.
Data extraction
Data extraction will be undertaken by one reviewer and checked by a second. A pre-defined and piloted data extraction form (Excel 2016) will be used. Items to data extract will be guided by the CHARMS [22] checklist and will include
Participants (e.g. eligibility and recruitment method, comorbidities, type and length of treatment for AF, surgery prior to AF)
Study design (e.g. randomised controlled trial, prospective study, sample size, length of follow-up)
Candidate predictors (e.g. number and type, method of measurement)
Outcome measures (e.g. incident or recurrent AF, how diagnosed)
Model development (e.g. modelling method, method for selection of predictors)
Model performance (discrimination measures, e.g. c-statistic, and calibration measures, e.g. ratio of observed and expected events (E/O ratio)
Model validation (e.g. method for testing model performance)
Assessment of risk of bias
The quality of studies reporting the development or validation of a prognostic model will be assessed using the Prediction Study Risk Of Bias Assessment Tool (PROBAST) [28]. This assesses criteria within five domains: participant selection (e.g. were inclusions and exclusions of participants appropriate); predictors (e.g. were predictors measured blind to outcome data); outcomes (e.g. was the same definition used for outcomes in all patients); sample size and patient flow (e.g. was there a pre-specified sample size based on estimated number of outcome events, handling of missing data); and analysis (e.g. was selection of predictors based on univariable analysis avoided).
Should impact studies be identified, then additional quality assessment tools will be used depending on study designs (e.g. Cochrane risk of bias tool [29] for randomised controlled trials).
Synthesis
All studies reporting validated prognostic models will be narratively described, with key findings tabulated (e.g. predictor variables included in different models, reported predictive accuracy of models). Models relating to different populations will be considered separately. We will compare model performance across different models, taking into consideration the quality of the study and thus the likelihood that model findings are accurate. We will also compare risk factors included in different models in order to identify those that are contributing most to predictive accuracy.
If the same model is validated in several studies, and the same discrimination or calibration statistics are reported (e.g. C-statistic, E/O ratio), multivariate random effects meta-analysis to jointly summarise calibration and discrimination to obtain average model performance will be considered. A random effects model is more likely to be suitable as validation studies typically differ in design and case-mix, and meta-analysis should allow for the presence of heterogeneity [25]). The I2 statistic will be used to estimate the proportion of heterogeneity that is due to between-study variability. Meta-analysis will only be undertaken for groups of studies including similar populations. Exploration of heterogeneity through subgroup analyses or meta-regression is unlikely to be feasible, as a minimum of 10 studies per variable is recommended [25] and scoping searches indicate lower numbers of studies relating to individual models. This also precludes formal exploration of publication bias using funnel plots.
The body of evidence identified will be considered and interpreted in the context of the domains described in the GRADE approach (risk of bias, imprecision, inconsistency, indirectness, and publication bias); however, a formal GRADE score will not be calculated.
Reporting
PRISMA guidelines [27] will be followed for the reporting of the systematic review.
Discussion
The increasing burden from AF means that a focus on prevention is becoming more important, and being able to identify patients most at risk of incident or recurrent AF is vital for tailoring management strategies. To the best of our knowledge, this will be the first systematic review to collate evidence from all studies reporting on validated prognostic models, or on the impact of such models, in any population at risk of incident or recurrent AF.
We will use recognised systematic review methods for identifying, appraising, and synthesising the available evidence on existing prognostic models, which will strengthen the robustness of any findings. Publication bias, poor reporting, and extensive heterogeneity are recognised issues in prognostic research [23–25]. Careful consideration of heterogeneity before analysis will ensure that studies are not inappropriately grouped. The potential impact of publication bias on any findings will be discussed, and recommendations made for future reporting of prognostic models if appropriate.
Should one or more well-validated models with high predictive accuracy be identified, then this will be useful for planning future comparative studies on the impact of using such models in clinical practice. The review will also identify gaps in the evidence, i.e. where model validity is poor or where models are lacking for a particular population. Review findings will therefore inform research recommendations relating to model development, validation, or impact assessment. Findings will be considered in the context of any models already used in clinical practice, and the extent to which these have been validated.
Acknowledgements
Not applicable
Abbreviation
- AF
Atrial fibrillation
Appendix
Search strategies Embase and MEDLINE.
atrial fibrillation.ti,ab.
exp atrial fibrillation/
1 OR 2
exp recurrent disease/
recur$.ti,ab.
exp incidence/
inciden$.ti,ab.
new onset.ti,ab.
new$ diagnos$.ti,ab.
progress$.ti,ab.
prevalen$.ti,ab.
exp prevalence/
OR/ 4-12
prognos$ model$.ti,ab.
exp prognosis/ and exp model/
predict$ model$.ti,ab.
exp prediction/ and exp model/
prognos$ index.ti,ab.
predict$ index.ti,ab.
risk$ predict$.ti,ab.
risk$ model$.ti,ab.
predict$ rule$.ti,ab.
prognos$ rule$.ti,ab.
risk score$.ti,ab.
clinical score$.ti,ab.
prognos$ risk.ti,ab.
decision rule$.ti,ab.
model$ score$.ti,ab.
model performance.ti,ab.
decision model.ti,ab.
validation.ti,ab.
stratification.ti,ab.
ROC curve.ti,ab.
exp receiver operating characteristic/
discriminat$.ti,ab.
c-statistic.ti,ab.
"Area under the curve".ti,ab.
AUC.ti,ab.
Calibration.ti,ab.
indices.ti,ab.
algorithm.ti,ab.
multivaria$.mp.
(HATCH adj3 scor$).ti,ab.
(ALARMEc adj3 scor$).ti,ab.
(BASE-AF2 adj3 scor$).ti,ab.
(APPLE adj3 scor$).ti,ab.
(CAAP-AF adj3 scor$).ti,ab.
(MB-LATER adj3 scor$).ti,ab.
(CHARGE-AF adj3 scor$).ti,ab.
(HAVOC adj3 scor$).ti,ab.
(ATLAS adj3 scor$).ti,ab.
(FHS adj3 scor$).ti,ab.
(ARIC adj3 scor$).ti,ab.
(WHS adj3 scor$).ti,ab.
(CHADS2 adj3 scor$).ti,ab.
(CHA2DS2-VASC adj3 scor$).ti,ab.
(R2CHADS2 adj3 scor$).ti,ab.
OR/14-57
3 AND 13 AND 58
Authors’ contributions
JD wrote the protocol and is the guarantor. PK, LF, WC, and DK provided clinical expertise and contributed to sections of the protocol. YT, NC, and JJD provided the methodological expertise and contributed to the sections of the protocol. SB advised on the search strategy. All authors commented on, and approved, the final version.
Funding
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement no. 633196 [CATCH ME]. DK is funded by a National Institute for Health Research (NIHR) Career Development Fellowship (CDF-2015-08-074). The opinions expressed are those of the authors and do not represent the NIHR or the UK Department of Health.
Availability of data and materials
Not applicable
Ethics approval
Not applicable
Consent for publication
Not applicable
Competing interests
DK, JD, JJD, NC, SB, WC, YT declare that they have no competing interests. PK has received research support from several drug and device companies active in atrial fibrillation and has received honoraria from several such companies in the past. LF and PK are listed as inventors on two patents held by University of Birmingham (Atrial Fibrillation Therapy WO 2015140571, Markers for Atrial Fibrillation WO 2016012783). LF has received institutional research grants from governmental and charity funding agencies (EU, DFG, MRC, BHF). LF has received institutional research grants from industry in the past (Gilead).
Footnotes
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Contributor Information
Janine Dretzke, Email: j.dretzke@bham.ac.uk.
Naomi Chuchu, Email: n.chuchu@bham.ac.uk.
Winnie Chua, Email: w.chua.1@bham.ac.uk.
Larissa Fabritz, Email: l.fabritz@bham.ac.uk.
Susan Bayliss, Email: s.bayliss@bham.ac.uk.
Dipak Kotecha, Email: d.kotecha@bham.ac.uk.
Jonathan J. Deeks, Email: j.deeks@bham.ac.uk
Paulus Kirchhof, Email: p.kirchhof@bham.ac.uk.
Yemisi Takwoingi, Email: y.takwoingi@bham.ac.uk.
References
- 1.Chugh SS, Havmoeller R, Narayanan K, Singh D, Rienstra M, Benjamin EJ, et al. Worldwide epidemiology of atrial fibrillation: a global burden of disease 2010 study. Circulation. 2014;129(8):837–847. doi: 10.1161/CIRCULATIONAHA.113.005119. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Morillo CA, Banerjee A, Perel P, Wood D, Jouven X. Atrial fibrillation: the current epidemic. J Geriatr Cardiol. 2017;14(3):195–203. doi: 10.11909/j.issn.1671-5411.2017.03.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Sankaranarayanan R, Kirkwood G, Visweswariah R, Fox DJ. How does chronic atrial fibrillation influence mortality in the modern treatment era? Curr Cardiol Rev. 2015;11(3):190–198. doi: 10.2174/1573403X10666140902143020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Kirchhof P, Benussi S, Kotecha D, Ahlsson A, Atar D, Casadei B, et al. 2016 ESC guidelines for the management of atrial fibrillation developed in collaboration with EACTS. Eur Heart J. 2016;37(38):2893–2962. doi: 10.1093/eurheartj/ehw210. [DOI] [PubMed] [Google Scholar]
- 5.Marijon E, Le Heuzey JY, Connolly S, Yang S, Pogue J, Brueckmann M, et al. Causes of death and influencing factors in patients with atrial fibrillation: a competing risk analysis from the randomized evaluation of long-term anticoagulant therapy study. Circulation. 2013;128(20):2192–2201. doi: 10.1161/CIRCULATIONAHA.112.000491. [DOI] [PubMed] [Google Scholar]
- 6.Rho RW. The management of atrial fibrillation after cardiac surgery. Heart. 2009;95(5):422–429. doi: 10.1136/hrt.2007.132795. [DOI] [PubMed] [Google Scholar]
- 7.Danelich IM, Lose JM, Wright SS, Asirvatham SJ, Ballinger BA, Larson DW, et al. Practical management of postoperative atrial fibrillation after noncardiac surgery. J Am Coll Surg. 2014;219(4):831–841. doi: 10.1016/j.jamcollsurg.2014.02.038. [DOI] [PubMed] [Google Scholar]
- 8.Boriani G, Proietti M. Atrial fibrillation prevention: an appraisal of current evidence. Heart. 2018;104(11):882–887. doi: 10.1136/heartjnl-2017-311546. [DOI] [PubMed] [Google Scholar]
- 9.Benjamin EJ, Chen PS, Bild DE, Mascette AM, Albert CM, Alonso A, et al. Prevention of atrial fibrillation: report from a national heart, lung, and blood institute workshop. Circulation. 2009;119(4):606–618. doi: 10.1161/CIRCULATIONAHA.108.825380. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Kirchhof P, Andresen D, Bosch R, Borggrefe M, Meinertz T, Parade U, et al. Short-term versus long-term antiarrhythmic drug treatment after cardioversion of atrial fibrillation (Flec-SL): a prospective, randomised, open-label, blinded endpoint assessment trial. Lancet. 2012;380(9838):238–246. doi: 10.1016/S0140-6736(12)60570-4. [DOI] [PubMed] [Google Scholar]
- 11.Jahangir A, Lee V, Friedman PA, Trusty JM, Hodge DO, Kopecky SL, et al. Long-term progression and outcomes with aging in patients with lone atrial fibrillation: a 30-year follow-up study. Circulation. 2007;115(24):3050–3056. doi: 10.1161/CIRCULATIONAHA.106.644484. [DOI] [PubMed] [Google Scholar]
- 12.Kirchhof P, Lip GY, Van Gelder IC, Bax J, Hylek E, Kaab S, et al. Comprehensive risk reduction in patients with atrial fibrillation: emerging diagnostic and therapeutic options—a report from the 3rd atrial fibrillation competence NETwork/European Heart Rhythm Association consensus conference. Europace. 2012;14(1):8–27. doi: 10.1093/europace/eur241. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Brandes A, Smit MD, Nguyen BO, Rienstra M, Van Gelder IC. Risk factor Management in Atrial Fibrillation. Arrhythm Electrophysiol Rev. 2018;7(2):118–127. doi: 10.15420/aer.2018.18.2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Chua Winnie, Purmah Yanish, Cardoso Victor R, Gkoutos Georgios V, Tull Samantha P, Neculau Georgiana, Thomas Mark R, Kotecha Dipak, Lip Gregory Y H, Kirchhof Paulus, Fabritz Larissa. Data-driven discovery and validation of circulating blood-based biomarkers associated with prevalent atrial fibrillation. European Heart Journal. 2019;40(16):1268–1276. doi: 10.1093/eurheartj/ehy815. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Schnabel RB, Larson MG, Yamamoto JF, Sullivan LM, Pencina MJ, Meigs JB, et al. Relations of biomarkers of distinct pathophysiological pathways and atrial fibrillation incidence in the community. Circulation. 2010;121(2):200–207. doi: 10.1161/CIRCULATIONAHA.109.882241. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.O'Neal WT, Alonso A. The appropriate use of risk scores in the prediction of atrial fibrillation. J Thorac Dis. 2016;8(10):E1391–E13E4. doi: 10.21037/jtd.2016.10.96. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Kwong C, Ling AY, Crawford MH, Zhao SX, Shah NH. A clinical score for predicting atrial fibrillation in patients with cryptogenic stroke or transient ischemic attack. Cardiology. 2017;138(3):133–140. doi: 10.1159/000476030. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Mesquita João, Ferreira António Miguel, Cavaco Diogo, Moscoso Costa Francisco, Carmo Pedro, Marques Hugo, Morgado Francisco, Mendes Miguel, Adragão Pedro. Development and validation of a risk score for predicting atrial fibrillation recurrence after a first catheter ablation procedure – ATLAS score. EP Europace. 2017;20(FI_3):f428–f435. doi: 10.1093/europace/eux265. [DOI] [PubMed] [Google Scholar]
- 19.Suenari K, Chao TF, Liu CJ, Kihara Y, Chen TJ, Chen SA. Usefulness of HATCH score in the prediction of new-onset atrial fibrillation for Asians. Medicine (Baltimore) 2017;96(1):e5597. doi: 10.1097/MD.0000000000005597. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Kornej J, Hindricks G, Shoemaker MB, Husser D, Arya A, Sommer P, et al. The APPLE score: a novel and simple score for the prediction of rhythm outcomes after catheter ablation of atrial fibrillation. Clin Res Cardiol. 2015;104(10):871–876. doi: 10.1007/s00392-015-0856-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Deng H, Bai Y, Shantsila A, Fauchier L, Potpara TS, Lip GYH. Clinical scores for outcomes of rhythm control or arrhythmia progression in patients with atrial fibrillation: a systematic review. Clin Res Cardiol. 2017;106(10):813–823. doi: 10.1007/s00392-017-1123-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Moons KG, de Groot JA, Bouwmeester W, Vergouwe Y, Mallett S, Altman DG, et al. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Med. 2014;11(10):e1001744. doi: 10.1371/journal.pmed.1001744. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Steyerberg EW, Moons KG, van der Windt DA, Hayden JA, Perel P, Schroter S, et al. Prognosis research strategy (PROGRESS) 3: prognostic model research. PLoS Med. 2013;10(2):e1001381. doi: 10.1371/journal.pmed.1001381. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Riley RD, Hayden JA, Steyerberg EW, Moons KG, Abrams K, Kyzas PA, et al. Prognosis research strategy (PROGRESS) 2: prognostic factor research. PLoS Med. 2013;10(2):e1001380. doi: 10.1371/journal.pmed.1001380. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Debray TP, Damen JA, Snell KI, Ensor J, Hooft L, Reitsma JB, et al. A guide to systematic review and meta-analysis of prediction model performance. BMJ. 2017;356:i6460. doi: 10.1136/bmj.i6460. [DOI] [PubMed] [Google Scholar]
- 26.Geersing GJ, Bouwmeester W, Zuithoff P, Spijker R, Leeflang M, Moons KG. Search filters for finding prognostic and diagnostic prediction studies in Medline to enhance systematic reviews. PLoS One. 2012;7(2):e32844. doi: 10.1371/journal.pone.0032844. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009;62(10):e1–34. doi: 10.1016/j.jclinepi.2009.06.006. [DOI] [PubMed] [Google Scholar]
- 28.Wolff RF, Moons KGM, Riley RD, Whiting PF, Westwood M, Collins GS, Reitsma JB, Kleijnen J, Mallett S, on behalf of the PROBAST group. PROBAST: A tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med. 2018; In press. [DOI] [PubMed]
- 29.Higgins JP, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928. doi: 10.1136/bmj.d5928. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Not applicable