Abstract
Background/Aims
With the recent explosion of medical data, it is necessary to develop software tools to support the process of evidence review and synthesis that is fundamental to Evidence-Based Medicine. Furthermore, as predictive models are being increasingly integrated into clinical decision making and personalized guidelines for individual patients, there is an even greater need for a practical tool that enables development, validation, and uncertainty quantification of models in a robust, automatic and efficient way.
Methods
We developed Risk Engine Evaluation Software, a software platform that enables users to compare and synthesize evidence from multiple data sources, to build robust and accurate predictive models to estimate and stratify disease risks, to validate risk equations and to quantify the accuracy of a predictive model in different subpopulations.
Results
We applied the tool to data on cardiovascular outcomes from a large number of trials, observational studies and electronic medical records, including Framingham Heart Study, Atherosclerosis Risk in Communities Study, Cardiovascular Health Study, and ALLHAT. We demonstrated that users can evaluate the performance of predictive models for different subpopulations in real time. Several metrics were used for model evaluation: cumulative incidence, calibration plots, receiver operating curve (ROC) and net reclassification index. The software also allows users to generate and then assess models for risks of MI, stroke or heart failure, for a predefined subpopulation from selected datasets.
Conclusions
Risk Engine Evaluation Software proves to be a useful tool for advancing the development and application of predictive modeling in medicine.
Keywords: Risk model, Individualized guidelines