Abstract
Background
The therapeutic armamentarium for patients with metastatic breast cancer is becoming more and more specific. Recommendations from clinical trials are not available for all treatment situations and patient subgroups, and it is therefore important to collect real-world data.
Summary
To develop recommendations for up-to-date treatments and participation in clinical trials for patients with metastatic breast cancer, the Prospective Academic Translational Research PRAEGNANT Network was established to optimize the quality of oncological care in the advanced therapeutic setting. The main aim of PRAEGNANT is to systematically record medical care for patients with metastatic breast cancer in the real-life setting, including the outcome and side effects of different treatment strategies, to monitor quality-of-life changes during therapy, to identify patients eligible for participation in clinical studies, and to allow targeted therapies based on the molecular structures of breast carcinomas.
Key Messages
This article describes the PRAEGNANT network and sheds light on the question of whether the various end points from clinical trials can be transferred to the real-world treatment situation.
Keywords: Breast cancer, Luminal, Metastasis, PRAEGNANT, Real-world evidence
Background for Real-World Evidence
Clinical Aspects
In a real-world registry, routine clinical care and treatment patterns can be defined, as well as factors that influence prognosis, safety, and health economics [1, 2]. In addition, patients' compliance and the effects of nutrition and exercise can be investigated. These data can be compared with the results of clinical trials and published in this context.
Inadequate documentation of side effects and their influence on patients' safety is currently the subject of research studies. Real-world registries provide an opportunity to systematically evaluate side effects. While acute adverse events are normally identified in the approval trials, late and rare adverse events are often seen only in the later post-approval period. Well-known examples of this are musculoskeletal complications associated with aromatase inhibitors, and everolimus-associated stomatitis [3, 4].
Real-World Registries
Several real-world registries have been initiated in Germany and other countries. The PRAEGNANT [3] and OPAL [5] studies are German registries that exclusively document the courses of breast cancer patients. The Epidemiological Strategy and Medical Economics (ESME) program (Unicancer) is the French equivalent [6]. One of the largest registries in which various − even rare − tumor entities are documented is the American Flatiron Registry, currently including 2.4 million patients from more than 800 sites [7]. The Adelphi Registry is internationally based and covers more than 60 disease entities, allowing cross-country comparisons [8]. The section below describes the German PRAEGNANT network in detail.
The PRAEGNANT study is an academic research network that features central documentation of clinical courses together with a biobank for the evaluation of biomarkers. All breast cancer patients receiving (neo-)adjuvant therapy or treatment for metastatic spread, irrespective of the treatment line, can be included. The primary goal of PRAEGNANT is to define the molecular characteristics of patients with metastatic breast cancer to allow further screening for inclusion into clinical trials. In addition, side effects and quality of life in clinical routine work are documented over (ideally) the patient's whole lifespan.
Patient Population
Inclusion in the study is not limited to patients receiving specific treatment lines. Re-staging procedures are carried out every 3 months or at disease progression. The target patient number is n = 5,000. This high number will ensure that rare molecular subtypes can be detected. At least 150 patients per treatment line are needed for this purpose, in order to obtain a representative view of the prognosis and quality of life.
Biomarker Program
Within the PRAEGNANT study, every patient has blood sampled at the baseline and at each change in treatment in order to bank serum, plasma, leukocyte RNA, and circulating tumor DNA (ctDNA). Blood samples are taken irrespective of the reason for the change in therapy. The samples are collected in a central biobank and processed (Fig. 1, 2).
Germline DNA
DNA is collected in all patients who consented to germline DNA genotyping at inclusion. Germline DNA is tested for genetic variants and correlated with clinical parameters, side effects, response to medication, and prognosis. Germline DNA is also used as a reference for comparison with tumor DNA.
Formalin-Fixed Paraffin-Embedded Tumor Tissue
Tumor blocks are archived centrally in the biobank for all patients in the PRAEGNANT network who have consented to central tumor banking. The tumor blocks are used for the central immunohistochemical evaluation of prognostic factors. In addition, the blocks are used for extraction of DNA and RNA, as well as for generation of tissue microarrays (TMAs). TMAs are prepared for all primary tumors and subsequent biopsies of metastatic lesions.
Informed Consent and Aims for Patients
Before study entry, each patient must provide written informed consent after receiving extensive information. At this stage, the patient decides actively about which results she wants to be informed of, for example molecular data if they may influence her therapy, or whether she decides to exclude information about hereditary tumor disease. In addition to supporting the medical community in achieving a better understanding of the diagnosis, course, and treatment of the disease, the benefit for the individual patient lies in those molecular tests with potential therapeutic consequences.
Science
As the PRAEGNANT study is essentially a biomaterials collection, it is possible to respond to recent academic questions. The aim is to identify biomarkers that may help predict the response to therapy. Quality-of-life questionnaires are used at regular intervals during different treatments to allow long-term longitudinal analysis.
Identification for Clinical Trials
Clinical trials require defined tumor features for inclusion-activating mutations − for example, the alpha-isoform of PI3K, PIK3CA mutation, or AKT mutations. The registry makes it possible to filter out patients with these mutations and offer them a targeted treatment option within a trial. This can improve recruitment into clinical trials.
Identification of Individual Therapy Trials for Patients
Individual trial options can be identified for individual patients. With tumor genome sequencing, genetic abnormalities can be found for patients who will no longer benefit from approved medications.
Feedback of Biomarker Results to Centers
The biomarker results are unblinded to the centers if the patient has consented to receive unrestricted informed feedback about all results, which is expected to assist in the choice of the optimal therapy for the individual patient. The same applies to germline mutations if the result of the scientific program has been confirmed by approved tests in clinical routine work.
Current Results from the PRAEGNANT Registry
Recruitment
The PRAEGNANT network recruits patients from hospitals and private oncological practices. The recruiting physicians are medical oncologists or gynecological oncologists. Currently, 58 centers are actively recruiting into the PRAEGNANT registry. From August 2014 to August 2020, a total of 3,948 patients had started and documented within a period of 84.6 months.
The median age of the patients recruited is 61 years (±12.8). There are no relevant age differences among the patients included in relation to the treatment line at inclusion. As expected, patients with TNBC formed the smallest group (14%), followed by the group with HER2-positive disease (26%) and luminal A cancers (48%). The distribution of molecular subtypes did not vary relative to the treatment line at the inclusion time point.
Treatment Patterns
The following questions can be answered using structured analyses of the PRAEGNANT documentation. Are standard therapeutic schedules, as recommended in guidelines, also used in clinical routine work? Are treatment sequences being followed in accordance with previous clinical trials? Or are rare older, outdated treatment patterns still being pursued?
From July 2014 to March 2017, the use of different forms of chemotherapy and endocrine treatments in first-line therapy and later treatment lines were investigated in the real-world situation. Patients were categorized on the basis of their treatment schedule. Although chemotherapy is not the first recommendation for patients with hormone receptor-positive metastatic breast cancer, it was found that 42% of the patients had received chemotherapy and 45.9% had endocrine treatment as the first palliative therapy. Thus, despite the preferential recommendation that endocrine therapy or a combination with everolimus should be used, chemotherapy was still being chosen to the same extent [9].
Real-world data on treatment courses in patients with HER2-positive metastatic breast cancer were evaluated. The sequence of trastuzumab, pertuzumab, and trastuzumab emtansine (T-DM1) or lapatinib was identified in the PRAEGNANT database. In general, a combination of trastuzumab and pertuzumab is the standard of care, followed by T-DM1 at disease progression [10].
Prognosis
The prognosis for patients receiving systemic therapy can be assessed in the PRAEGNANT study using the precise documentation of patient characteristics and therapeutic sequences, in contrast to the rigid groups of patients defined by the inclusion and exclusion criteria in clinical trials. Real-world analysis allows prognostic estimates for metastatic disease as well as neoadjuvant treatments, through retrospective assessment based on optimal documentation of patient data.
Molecular Markers
Apart from prognosis, treatment sequences, and quality of life, the PRAEGNANT study also focuses on the analysis of molecular biomarkers. At study entry, at the baseline, and before any change in therapy, blood is taken for the analysis of different markers such as molecular subtypes, genetic and epigenetic factors, circulating tumor cells, and gene expression. The aim is to promote individualized therapy and evaluate the longitudinal course of biomarkers in order to adapt targeted therapies to targetable molecular markers in patients with breast cancer in the future [3]. This approach is currently being tested in a clinical phase 2 substudy called plasmaMATCH in a group of 1,150 patients with metastatic breast cancer. Through ctDNA testing, patients with genetic alterations (ESR1, HER2, AKT1, AKT, or PTEN) are identified and assigned to treatment regimens that match the mutation (fulvestrant, olaparib, ceralasertib, capivasertib, or neratinib). Testing ctDNA is a less invasive approach for detecting genetic alterations than biopsies and allows longitudinal evaluations. The data currently available show a confirmed response rate of 23% in HR+ HER2– patients, with a duration of response of 5.7 months. The authors of the published report conclude that neratinib, with or without fulvestrant, is effective and safe in patients with metastatic breast cancer and HER2 mutations [11].
Patients in the PRAEGNANT registry are analyzed for mutations in genes such as BRCA1, BRCA2, CHEK2, PALB2, ATM, RAD51D, and BARD1. Mutations in these genes have been confirmed in 8.4% of 1,462 patients. Retrospective analyses show that patients with brain and visceral metastases have a higher incidence of these mutations than patients with bone metastases. In particular, patients with PALB2 mutations had an increased incidence of brain metastases [12].
Big Data and Artificial Intelligence
One of the main objectives of the PRAEGNANT network is to identify breast cancer patients for inclusion in clinical trials, using clinical features or molecular markers [3]. Using artificial intelligence or programmed searches in the PRAEGNANT data bank, an effort is being made to identify patients for clinical trials based on the prerequisites of the inclusion and exclusion criteria.
This computer-based approach to identifying potential study patients has been used in 326 patients in the PRAEGNANT network. The question was whether patients would qualify for inclusion in the EMBRACA trial [13]. On the basis of individual expert opinions, 120 of 326 patients appeared to fulfill the inclusion criteria. Using the computer algorithm, the figure was 129. The sensitivity of the computer algorithm was 0.87, with a specificity of 0.88 [14].
The degree of interest in and the practicability of using electronic questionnaires on quality of life were also evaluated in the PRAEGNANT study. Seventeen patients in the PRAEGNANT network were interviewed regarding their potential use of and interest in an electronic questionnaire and tested it. Seven of 15 subjects did not prefer the electronic questionnaire, while 5 said they would prefer the electronic questionnaire and 3 were hesitant, although 76% of the patients did not have any difficulties in using the document. Introducing electronic questionnaires is therefore feasible but depends on patients' compliance [15].
The data in the PRAEGNANT registry have also been used to develop a computer-aided disposition system. For this purpose, real-world data consisting of patients' characteristics, information about the primary tumor, metastatic spread, local relapses, radiotherapy, and hospitalizations in 2,869 patients were entered into a recurrent neuronal network using an encoder-decoder framework in order to generate computer-assisted treatment decisions. The physician can then receive an overview of the therapeutic recommendations that have been given in comparable situations and can be warned in case of decisions that do not match current standards [16, 17, 18].
Comparison of Study Results and Real-World Data
Clinical Trials versus Registry Data
Evidence from randomized clinical studies is generally regarded as being more valuable and reliable than real-world data. However, real-world evidence (RWE) can be used to supplement results from clinical trials. Both clinical trials and real-world data have limitations in their design and potential for results to be transferred to individual patients.
RWE comes from everyday practice. Traditionally, evidence is generated by randomized controlled and blinded clinical trials (RCTs). In RCTs, randomization, blinding, and selection of groups of patients on the basis of inclusion and exclusion criteria are used to test the hypotheses generated as objectively as possible in comparable cohorts and to avoid any potential bias [19].
At the same time, however, these methods of objectivation limit the results of RCTs. In order to set up approval trials, researchers try to generate a homogeneous, representative cohort. Many patients are consequently excluded due to comorbidities, age, or divergent laboratory results, while in the real world they would need to receive a defined therapy [20, 21]. The strict conclusion from this would be that results from RCTs can only be transferred to patients who meet the original inclusion criteria.
RWE is capable of adding information to the results of RCTs here so that the results can be transferred to a realistic patient group. It is very important that such real-world studies should be planned on the basis of a study protocol in order to reduce susceptibility to bias. Ideally, the 2 forms of evidence should not be regarded as competitive, but rather as complementary [22].
Table 1 provides an overview of the differences between RCTs and studies based on RWE. The advantages and limitations of RWE in comparison with RCTs are listed in Table 2 [23].
Table 1.
Variable | RCTs | RWE |
---|---|---|
Aim | Efficacy | Efficiency |
Treatment context | Experimental | Clinical routine |
Follow-up | Prospectively planned and limited | Unlimited |
Treatment | Defined regimen | Variable regimen(s) |
Patient collective | Homogeneous | Inhomogeneous |
Attending physician | Study physician | Multiple, nonspecialized physicians |
Comparison between | Placebo/defined alternative intervention | Multiple alternative interventions |
Patient monitoring | Continuous, per protocol | Variable |
RCTs, randomized clinical trials; RWE, real-world evidence.
Table 2.
Advantages |
− Less time-consuming and cheaper in comparison with RCTs |
− Shorter duration of clinical evaluations |
− No defined time frame for patient recruitment and patient inclusion |
− Scientific and clinical questions that cannot be addressed in RCTs can be approached here |
− Option for safe observation of very small and high-risk collectives |
Limitations |
− Large datasets are needed for valid results |
− Data quality management is very time-consuming |
− Highly qualified experts are needed for data analysis |
− High potential for bias (lack of causality, selection bias, immortality bias) |
RCTs, randomized clinical trials; RWE, real-world evidence. |
Correlation between End Points
A recently published meta-analysis investigated the transfer of study data concerning treatment choices in 21 phase 3 studies in relation to overall survival (OS) and surrogate parameters such as progression-free survival (PFS) and time to progression (TTP) in patients with breast cancer, colorectal cancer, lung cancer, ovarian cancer, and pancreatic carcinoma. Epidemiologically identified real-world data were compared with the OS results [24]. Patients from the real-world comparative groups were selected on the basis of the inclusion and exclusion criteria used in the clinical trials.
The OS advantage between the data from RCTs and the real-world comparative groups was quite comparable. The surrogate end points that were used in the RCTs − PFS and TTP − overestimated the actual OS benefit by approximately 16%. The authors concluded that the results can lead to improved assessment of the calculated benefits of a specific therapy on the basis of surrogate markers, and help in choosing the right end points in clinical trials and in interpreting and transferring end-point results from clinical trials into the real treatment scenario.
Conclusions
The fact that more and more individualized treatment options are available for patients with metastatic breast cancer means that new approaches are needed for generating evidence. Systematically established real-world data can be used to support the evidence here, particularly for rare treatment situations and subgroups. This makes the generation and systematic evaluation of real-world data indispensable.
Conflict of Interest Statement
E.B. has received honoraria from Novartis, Celgene, Riemser, Pfizer, Hexal, Amgen, and onkowissen.de for consulting, clinical research management, and medical education activities. M.W.B.'s institution has received research funding from Novartis and BioNTech. J.E. has received honoraria from AstraZeneca, Daiichi Sankyo, Eisai, Lilly, Novartis, Pierre Fabre, and Pfizer. T.N.F. has received honoraria from AstraZeneca, Celgene, Eisai, Pfizer, and Roche. P.A.F. reports grants from Novartis, Cepheid, and BioNTech, personal fees from Novartis, Roche, Pfizer, Celgene, Daiichi Sankyo, TEVA, AstraZeneca, Merck Sharp & Dohme, Myelo Therapeutics, Macrogenics, Eisai, and Puma. A.D.H. has received honoraria from Teva, Genomic Health, Celgene, AstraZeneca, Novartis, Pfizer, Lilly, MSD, Eisai, and Roche. W.J. has received honoraria and research grants from Novartis. H.-C.K. has received honoraria from Carl Zeiss Meditec, TEVA, Theraclion, Novartis, Amgen, AstraZeneca, Pfizer, Janssen-Cilag, GSK, LIV Pharma, MSD, Onkowissen, SurgVision, Roche, and Genomic Health. D.L. has received honoraria from Amgen, AstraZeneca, Celgene, Lilly, Loreal, MSD, Novartis, Pfizer, Tesaro, Teva. M.P.L. has received honoraria from Pfizer, Lilly, Roche, MSD, Hexal, Novartis, AstraZeneca, Eisai, Medac, and Exact Sciences for consultancy services, lectures, and travel support. V.M. has received speaker honoraria from Amgen, AstraZeneca, Celgene, Daiichi Sankyo, Eisai, Pfizer, Pierre Fabre, Novartis, Roche, Teva, and Janssen-Cilag, and consultancy honoraria from Genomic Health, Roche, Pierre Fabre, Amgen, Daiichi Sankyo, and Eisai. F.O. has received speaker and consultancy honoraria from Amgen, AstraZeneca, Bayer, Beiersdorf, BMS, Boehringer, Celgene, Cellex, Chugai, Eisai, Gilead, Hexal, Ipsen, Janssen, Merck, MSD, Novartis, Novonordisc, Riemser, Roche, Servier, Shire, Tesaro, and Teva. A.S. has received honoraria from Roche, Celgene, AstraZeneca, Novartis, Pfizer, Zuckschwerdt Verlag GmbH, Georg Thieme Verlag, Aurikamed GmbH, MCI Deutschland GmbH, bsh medical communications GmbH, and proMedicis GmbH. H.T. has received honoraria from Novartis, Roche, Celgene, TEVA, and Pfizer, and travel support from Roche, Celgene, and Pfizer. M.W. has received speaker honoraria from AstraZeneca, Celgene, and Novartis. S.Y.B. has received honoraria from Roche, Novartis, Pfizer, MSD, Teva, and Astra Zeneca. All of the remaining authors (D.W., A.T., and P.P.) have declared that they have no conflicts of interest.
Funding Sources
PRAEGNANT is supported by research grants from Novartis, Celgene, and Roche. It is financed through translational research resources (TRRs) in the Department of Gynecology and Obstetrics at the University of Erlangen.
Author Contributions
D.L. and P.A.F. are responsible for the concept of the paper and the final manuscript. A.T. and P.P. contributed to the first draft. A.D.H., M.P.L., F.O., H.T., M.W., V.M., M.W.B., E.B., W.J., T.N.F., H.-C.K., J.E., D.W., A.S., and S.Y.B. contributed to parts of the manuscript and the final editing.
verified
References
- 1.Gliklich RE, Leavy MB, Dreyer NA, editors. 4th ed. Rockville (MD): Agency for Healthcare Research and Quality; 2020. Registries for evaluating patient outcomes: a user's guide. Available from https://www.ncbi.nlm.nih.gov/books/n/cerregistries4ed/pdf/ [PubMed] [Google Scholar]
- 2.LaBresh KA, Gliklich R, Liljestrand J, Peto R, Ellrodt AG. Using “get with the guidelines” to improve cardiovascular secondary prevention. Jt Comm J Qual Saf. 2003 Oct;29((10)):539–50. doi: 10.1016/s1549-3741(03)29064-x. [DOI] [PubMed] [Google Scholar]
- 3.Fasching PA, Brucker SY, Fehm TN, Overkamp F, Janni W, Wallwiener M, et al. Biomarkers in patients with metastatic breast cancer and the PRAEGNANT study network. Geburtshilfe Frauenheilkd. 2015 Jan;75((1)):41–50. doi: 10.1055/s-0034-1396215. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Lo Muzio L, Arena C, Troiano G, Villa A. Oral stomatitis and mTOR inhibitors: A review of current evidence in 20,915 patients. Oral Dis. 2018 Mar;24((1-2)):144–71. doi: 10.1111/odi.12795. [DOI] [PubMed] [Google Scholar]
- 5.Decker T, Steering Board OPAL Die klinische Tumorregisterplattform OPAL stellt sich vor: Umfassende Daten zu Behandlung und Outcome von Patienten mit fortgeschrittenem Mammakarzinom aus der Routinebehandlung. Forum. 2019 Feb;34((1)):74–6. [Google Scholar]
- 6.Pérol D, Robain M, Arveux P, Mathoulin-Pélissier S, Chamorey E, Asselain B, et al. The ongoing French metastatic breast cancer (MBC) cohort: the example-based methodology of the Epidemiological Strategy and Medical Economics (ESME) BMJ Open. 2019 Feb;9((2)):e023568. doi: 10.1136/bmjopen-2018-023568. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Quek RG, Mardekian J. Clinical outcomes, treatment patterns, and health resource utilization among metastatic breast cancer patients with germline BRCA1/2 mutation: a real-world retrospective study. Adv Ther. 2019 Mar;36((3)):708–20. doi: 10.1007/s12325-018-0867-x. [DOI] [PubMed] [Google Scholar]
- 8.Anderson P, Benford M, Harris N, Karavali M, Piercy J. Real-world physician and patient behaviour across countries: Disease-Specific Programmes − a means to understand. Curr Med Res Opin. 2008 Nov;24((11)):3063–72. doi: 10.1185/03007990802457040. [DOI] [PubMed] [Google Scholar]
- 9.Hartkopf AD, Huober J, Volz B, Nabieva N, Taran FA, Schwitulla J, et al. Treatment landscape of advanced breast cancer patients with hormone receptor positive HER2 negative tumors − Data from the German PRAEGNANT breast cancer registry. Breast. 2018 Feb;37:42–51. doi: 10.1016/j.breast.2017.10.002. [DOI] [PubMed] [Google Scholar]
- 10.Lux MP, Nabieva N, Hartkopf AD, Huober J, Volz B, Taran FA, et al. Therapy landscape in patients with metastatic HER2-positive breast cancer: data from the PRAEGNANT real-world breast cancer registry. Cancers (Basel) 2018 Dec;11((1)):10. doi: 10.3390/cancers11010010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Macpherson I, Kilburn L, Kernaghan S, Wardley AM, Baird RD, Roylance R, et al. Results from plasmaMATCH trial treatment cohort A: a phase II trial of extended-dose fulvestrant in patients with an ESR1 mutation identified via ctDNA screening (CRUK/15/010) Cancer Res. 2020;80((4 Suppl)) P1-19-04. [Google Scholar]
- 12.Fasching P, Hu C, Hart S, Polley E, Lee K, Gnanolivu R, et al. Cancer predisposition genes in metastatic breast cancer − association with metastatic pattern patient and tumor characteristics [abstract] Cancer Res. 2018;78((4)) PD1-02-PD1-02. [Google Scholar]
- 13.Litton JK, Rugo HS, Ettl J, Hurvitz SA, Gonçalves A, Lee KH, et al. Talazoparib in patients with advanced breast cancer and a germline BRCA mutation. N Engl J Med. 2018 Aug;379((8)):753–63. doi: 10.1056/NEJMoa1802905. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Hein A, Gass P, Walter CB, Taran FA, Hartkopf A, Overkamp F, et al. Computerized patient identification for the EMBRACA clinical trial using real-time data from the PRAEGNANT network for metastatic breast cancer patients. Breast Cancer Res Treat. 2016 Jul;158((1)):59–65. doi: 10.1007/s10549-016-3850-8. [DOI] [PubMed] [Google Scholar]
- 15.Wallwiener M, Heindl F, Brucker SY, Taran FA, Hartkopf A, Overkamp F, et al. Implementation and feasibility of electronic patient-reported outcome (ePRO) data entry in the PRAEGNANT real-time advanced and metastatic breast cancer registry. Geburtshilfe Frauenheilkd. 2017 Aug;77((8)):870–8. doi: 10.1055/s-0043-116223. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Yang Y, Fasching PA, Tresp V. Modeling clinical decisions with multinomial hierarchical classification. Proc Machine Learning Res (Sydney, Australia); 2017 Available from: http://www.praegnant.org/fileadmin/PRAEGNANT/downloads/Yinchong_Yong_et_al._Modeling_Clinical_Decisions_with_Multinomal_Hierarchical_Classification.pdf. [Google Scholar]
- 17.Yang Y, Fasching PA, Tresp V, Predictive modeling of therapy decisions in metastatic breast cancer with recurrent neural network encoder and multinomial hierarchical regression decoder 2017 IEEE International Conference on Healthcare Informatics (ICHI) Park City, UT, USA: IEEE. 2017:pp. 46–55. [Google Scholar]
- 18.Yang Y, Fasching PA, Wallwiener M, Fehm TN, Brucker SY, Tresp V. Predictive clinical decision support system wtih RNN encoding and tensor decoding. Barcelona, Spain: arXiv preprint. :p 1612.00611v1. Available from: https://arxiv.org/pdf/1612.00611.pdf. [Google Scholar]
- 19.Suvarna VR. Real world evidence (RWE) − Are we (RWE) ready? Perspect Clin Res. 2018 Apr-Jun;9((2)):61–3. doi: 10.4103/picr.PICR_36_18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.May GS, DeMets DL, Friedman LM, Furberg C, Passamani E. The randomized clinical trial: bias in analysis. Circulation. 1981 Oct;64((4)):669–73. doi: 10.1161/01.cir.64.4.669. [DOI] [PubMed] [Google Scholar]
- 21.Viera AJ, Bangdiwala SI. Eliminating bias in randomized controlled trials: importance of allocation concealment and masking. Fam Med. 2007 Feb;39((2)):132–7. [PubMed] [Google Scholar]
- 22.Nallamothu BK, Hayward RA, Bates ER. Beyond the randomized clinical trial: the role of effectiveness studies in evaluating cardiovascular therapies. Circulation. 2008 Sep;118((12)):1294–303. doi: 10.1161/CIRCULATIONAHA.107.703579. [DOI] [PubMed] [Google Scholar]
- 23.Wilk N, Wierzbicka N, Skrzekowska-Baran I, Moćko P, Tomassy J, Kloc K. Study types and reliability of Real World Evidence compared with experimental evidence used in Polish reimbursement decision-making processes. Public Health. 2017 Apr;145:51–8. doi: 10.1016/j.puhe.2016.12.025. [DOI] [PubMed] [Google Scholar]
- 24.Lakdawalla DN, Shafrin J, Hou N, Peneva D, Vine S, Park J, et al. Predicting real-world effectiveness of cancer therapies using overall survival and progression-free survival from clinical trials: empirical evidence for the ASCO value framework. Value Health. 2017 Jul − Aug;20((7)):866–75. doi: 10.1016/j.jval.2017.04.003. [DOI] [PubMed] [Google Scholar]