Skip to main content
NCBI home page
BMJ Open Access logoLink to BMJ Open Access
. 2023 Nov 6;33(11):1733–1742. doi: 10.1136/ijgc-2023-004605

Optimizing disease progression assessment using blinded central independent review and comparing it with investigator assessment in the PRIMA/ENGOT-ov26/GOG-3012 trial: challenges and solutions

Thomas J Herzog 1,✉, Shaun A Wahab 2, Mansoor R Mirza 3, Bhavana Pothuri 4, Ignace Vergote 5, Whitney S Graybill 6, Izabela A Malinowska 7, Whitney York 8, Jean A Hurteau 9, Divya Gupta 10, Antonio González-Martin 11, Bradley J Monk 12
PMCID: PMC10646892  PMID: 37931976

Abstract

Objective

Progression-free survival is an established clinically meaningful endpoint in ovarian cancer trials, but it may be susceptible to bias; therefore, blinded independent centralized radiological review is often included in trial designs. We compared blinded independent centralized review and investigator-assessed progressive disease performance in the PRIMA/ENGOT-ov26/GOG-3012 trial examining niraparib monotherapy.

Methods

PRIMA/ENGOT-ov26/GOG-3012 was a randomized, double-blind phase 3 trial; patients with newly diagnosed stage III/IV ovarian cancer received niraparib or placebo. The primary endpoint was progression-free survival (per Response Evaluation Criteria in Solid Tumors [RECIST] v1.1), determined by two independent radiologists, an arbiter if required, and by blinded central clinician review. Discordance rates between blinded independent centralized review and investigator assessment of progressive disease and non-progressive disease were routinely assessed. To optimize disease assessment, a training intervention was developed for blinded independent centralized radiological reviewers, and RECIST refresher training was provided for investigators. Discordance rates were determined post-intervention.

Results

There was a 39% discordance rate between blinded independent centralized review and investigator-assessed progressive disease/non-progressive disease in an initial patient subset (n=80); peritoneal carcinomatosis was the most common source of discordance. All reviewers underwent training, and as a result, changes were implemented, including removal of two original reviewers and identification of 10 best practices for reading imaging data. Post-hoc analysis indicated final discordance rates between blinded independent centralized review and investigator improved to 12% in the overall population. Median progression-free survival and hazard ratios were similar between blinded independent centralized review and investigators in the overall population and across subgroups.

Conclusion

PRIMA/ENGOT-ov26/GOG-3012 highlights the need to optimize blinded independent centralized review and investigator concordance using early, specialized, ovarian-cancer-specific radiology training to maximize validity of outcome data.

Keywords: Ovarian Neoplasms

WHAT IS ALREADY KNOWN ON THIS TOPIC

  • Blinded independent centralized radiological review is often included in ovarian cancer trial designs to provide an independent, objective review of imaging data, and to help validate progression-free survival as a primary endpoint. Ovarian cancer trials have reported high concordance between local and independent review committee assessments of progression status; however, when high discordance is apparent, little has been published regarding best practices to optimize blinded independent centralized review performance.

WHAT THIS STUDY ADDS

  • Due to training intervention of the independent reviewers, 10 of the most essential considerations in terms of best practices for radiologists were identified, and changes were implemented to the PRIMA imaging charter; two original reviewers with high reader variability were removed.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

  • This study demonstrates the benefit of using specialized, disease-specific training for the independent reviewers in an ovarian cancer trial, and highlights the need to evaluate concordance early on in trials.

INTRODUCTION

Successful clinical trials must adopt study design practices to reduce bias and ensure that appropriate endpoints are selected. While overall survival is the gold standard endpoint in oncology trials, demonstrating an effect on overall survival in front-line ovarian cancer studies is challenging, despite clear evidence of benefit to progression-free survival.1–3 Progression-free survival is regarded by the Gynecological Cancer InterGroup (GCIG) consensus as an acceptable primary endpoint in front-line ovarian cancer due to its temporal relevance in a rapidly evolving treatment landscape.4 5 Since 2014, new drug approvals by the US Food and Drug Administration (FDA) have substantially risen for ovarian cancer, with over a dozen approvals largely based on progression-free survival data;5 6 thus supporting progression-free survival as the primary endpoint for front-line ovarian cancer.7

However, methodologic flaws can compromise the validity of progression-free survival as a primary endpoint. A lack of consensus in determining disease progression between radiologists can bias the estimated treatment effect and increase the informative censoring in a trial.8 In addition to symmetry in disease assessment between trial arms, the use of either a placebo and/or disease response assessment conducted by blinded independent centralized radiological review are necessary elements of a well conducted registrational trial.5 9 Due to the unique disease characteristics of ovarian cancer, imaging ovarian cancer can be particularly challenging compared with imaging other solid tumors.5 10 Blinded independent centralized review has been implemented in ovarian cancer trials to provide an independent, objective review of imaging data by external experienced radiologists.5 Advantages of blinded independent centralized review and investigator-assessed progression-free survival are described in table 1; however, little has been published regarding best practices to optimize blinded independent centralized review performance.

Table 1.

Advantages of blinded independent centralized review and investigator-assessed progression-free survival 5

Advantages of blinded independent centralized review assessed progression-free survival Advantages of investigator-assessed progression-free survival
1. Reduces bias in assessment of progression-free survival as an endpoint in trials, particularly in trials with insufficient placebo blinding or for regulatory approvals Evaluation is performed by clinicians and radiologists specialized in ovarian cancer
2. Progression-free survival determined by reviewer with no prior knowledge of the patient, thereby reducing the chance of inappropriately maintaining patients on promising investigational regimens Investigators have access to a patient’s medical history and clinical data (eg, CA125 levels, prior therapies, all prior imaging data); this is particularly important in ovarian cancer as patients can often present with changes in imaging that do not necessarily meet RECIST v1.1 criteria
3. Blinded independent centralized review standardizes assessments across a trial and enhances consistency of image assessment Decreased informative censoring (in cases of discordance between investigator and blinded independent centralized review assessed progressive disease)
4. Blinded independent centralized review is strongly encouraged for studies with progression-free survival or objective response rates as the primary endpoint, both by FDA guidance for industry on clinical trials endpoints for the approval for cancer drugs and biologics30 and by EMA guidelines on the methodological consideration for using progression-free survival or disease-free survival in confirmatory trials31 Reduced waiting time for patients when progressive disease is suspected
5. Overarching goal is to provide consistency in reader interpretation:

  • Limited number of pre-qualified reviewers and adjudicator(s) for reader discordance

  • Formal protocol-specific training (and re-training PRN) and imaging charter

  • Images presented in same manner to all reviewers, who have access to the same tools

  • Edit checks and audit trials

  • Scheduled periodic evaluation of reviewers via evaluation of reader variability rates and adjudicator agreement rates (or other metrics)

 Radiological reviews by study investigators are considered a ‘real-world’ assessment of the clinical study results from a practitioner’s point of view
Open in a new tab

CA125, cancer antigen 125; EMA, European Medicines Agency; FDA, Food and Drug Administration; PRN, pro re nata; RECIST, Response Evaluation Criteria in Solid Tumors.

PRIMA/ENGOT-OV26/GOG 3012 was a phase 3 front-line trial that resulted in FDA and European Medicines Agency (EMA) approval of niraparib maintenance therapy in patients with advanced ovarian cancer who responded to platinum-based therapy, irrespective of homologous recombination deficiency (HRD) status.11–13 The primary endpoint in PRIMA was progression-free survival as assessed by blinded independent centralized review.11 Our objective was to use this trial to illustrate the importance of conducting blinded independent centralized review assessments properly and to ensure concordance was optimized with investigators via implementation of training and imaging charter amendments.

METHODS

PRIMA Study

PRIMA was a randomized, double-blind, placebo-controlled phase 3 trial in patients with newly diagnosed stage III or IV ovarian cancer, for which ethical approval was obtained by the appropriate Institutional Review Board as listed in online supplemental table 1 (eg, Western Institutional Review Board Copernicus Group covered several US sites) as well as appropriate written informed consent. Full descriptions of the patient eligibility and trial design have previously been published.11

Supplementary data

ijgc-2023-004605supp001.pdf (245.6KB, pdf)

The primary study endpoint was progression-free survival in patients with HRD-positive (tested using Myriad MyChoice® CDx) ovarian cancer and in the overall population, as determined by hierarchical testing. Progression-free survival was defined as the time from randomization after the last platinum-based chemotherapy to the earliest date of objective disease progression via imaging (per Response Evaluation Criteria in Solid Tumors [RECIST] v1.114) or death from any cause. Tumor assessment occurred at screening and then every 12 weeks (±7 days) from the cycle 1/day 1 visit, using computerized tomography (CT), magnetic resonance imaging (MRI), or positron emission tomography (PET)/CT until progression, when a final follow-up set of imaging was required. Progression was assessed by RECIST v1.1 criteria and clinical criteria using blinded independent centralized review by two independent radiologists and, if necessary, an arbiter, as well as by an independent oncologist when the radiology review did not meet the criteria for progressive disease. In total, five blinded independent centralized reviewers were assigned to PRIMA. Disease progression would also be reviewed if CA125 levels increased, per the GCIG criteria,15 in conjunction with histological proof or clinical symptoms, as specified in the protocol.

Blinded Independent Central Review Process

Upon determination of progression by the investigator or discontinuation of treatment, all imaging and supportive clinical data were submitted for blinded central review by two independent radiologists according to RECIST v1.1. Clinical data for radiology review only included information regarding prior procedures and radiotherapy, as previously irradiated regions would not be assessed by blinded independent centralized review. Whereas for a clinical review, supportive data included CA125 levels, all reported adverse events (except hematologic toxicities), and any other imaging data acquired (eg, ultrasound). Upon blinded independent centralized reviewer selection, specific experience in ovarian cancer imaging was not mandatory, assuming that reviewers were qualified to read abdominal/pelvic CT/MRI scans (for RECIST assessment of patients undergoing PET–CT scans, central reviewers interpreted the CT scans; PET scans were read by nuclear medicine physicians and results were provided as part of clinical review dossier). However, if the determination of progression differed between the two independent radiologists, a third independent radiologist completed adjudication. If central radiology review failed to determine progressive disease, the central blinded clinician then reviewed clinical information for determination of progressive disease, and the site was informed of the central progressive disease determination. If the investigator determined progressive disease, but central review did not, the patient continued study treatment provided safety and treatment criteria were met. All scheduled patient imaging was performed until radiologic or clinical progressive disease was confirmed by blinded independent centralized review, subsequent therapy initiated, or study discontinuation.

The central blinded clinician reviewed clinical and radiographic data supporting clinical progression to determine whether the patient had protocol-defined clinical progressive disease (and if so, at which time point), but did not advise on timing of radiographic progression, nor modify lesion selection performed by independent radiologists. Lesions assessed by physical examination and documented by the site were assessed qualitatively and incorporated into the central blinded clinician’s assessment. Blinded independent centralized review assessments were used for study purposes only, and were not used to inform clinical decision making for patients.

Intervention Training

To optimize blinded independent centralized reviewer performance and reduce bias in the assessment of progressive disease, a training intervention was developed for blinded central reviewers, which was composed of live reader training and an offline review of provided materials. The reader training focused on the application of RECIST v1.1 in the context of ovarian cancer, and was provided by one of the authors (SAW), a fellowship-trained body imaging radiologist with expertise in ovarian cancer imaging. The first part consisted of a didactic presentation to provide the context for understanding the different pathways of metastatic disease spread in ovarian cancer, including direct extension and lymphatic, hematologic, and peritoneal spread. The second part included a discussion of the application of RECIST v1.1 as it specifically pertains to ovarian cancer, with an emphasis on lesion selection and assignment of progression. Details such as the importance of imaging data quality and what constitutes measurable lesions versus non-measurable lesions were emphasized. Reviewers were encouraged to identify and measure at least one lesion at baseline and to capture subtle disease as non-target lesions if they did not meet the size criteria; failing the identification of any disease, reviewers had to confirm no lesions were identified. During follow-up, any lesion considered malignant seen for the first time on follow-up, not measured, or selected at baseline for comparison were considered new lesions. Reviewers were reminded to consider all anatomy to assess the disease burden and possible new lesions at each time point. The third part of the training consisted of a mix of CT and MRI case examples to emphasize the main points discussed and key points to consider, including assessment under the diaphragm, use of the coronal plane when imaging, and presence of calcifications.

Lastly, an interactive question and answer session concluded the training. All five of the central reviewers who were originally assigned to the PRIMA study attended the initial training. Radiologists then reviewed materials, such as the system user manual v1.0 plus radiology addendum v1.0, and the reviewer communication log v3.0 offline.

Assessments

Discordance

Discordance rates between progressive disease and non-progressive disease were assessed routinely for investigator progression-free survival and blinded independent centralized review progression-free survival data as part of study conduct. Rates were calculated as the percentage of patients with unconfirmed blinded independent centralized review progressive disease or non-progressive disease (including censored patients). Final discordance rates were determined post-hoc for the overall patient population, and the following biomarker subgroups: HRD-positive, HRD-positive BRCA mutated (BRCAm), HRD-positive BRCA wild-type (BRCAwt), and HRD-negative populations.

Progression-Free Survival Assessments

At the primary analysis (data cut-off 17 May 2019), median progression-free survival and hazard ratios (HR) for the overall patient population, HRD-positive BRCAm, HRD-positive BRCAwt, and HRD-negative populations were assessed and compared between blinded independent centralized review and investigator.

In accordance with the journal’s guidelines, we will provide our data for independent analysis by a team selected by the editorial team for the purposes of additional data analysis or for the reproducibility of this study in other centers, if such is requested.

RESULTS

Pre-Intervention Discordance

In a subset of 80 patients (April 2018), the average discordance rate between blinded independent centralized reviewers and investigator determination of progressive disease and non-progressive disease pre-intervention was 39% (n=31; figure 1). Among the 31 patients in whom investigators deemed progression and blinded independent centralized review reviewers did not, the most common sources of discordance were radiological findings of peritoneal carcinomatosis (n=11; 24% of all new lesions) and fluid collections (such as ascites and pleural effusion; n=8; 18% of all new lesions) arising from new non-target lesions (table 2).

Figure 1.

Figure 1

Open in a new tab

Summary of blinded independent centralized review vs investigator discordance in the PRIMA trial and changes implemented post-intervention. *Initial discordance was assessed in a subset of 80 patients; †Data cut-off for primary data analyses. Discordance rates were not evaluated statistically, and they have been shown to vary across other ovarian cancer studies. BICR, blinded independent central review; PD, progressive disease.

Table 2.

Pre-intervention discordance rate between investigators and blinded independent centralized review

Location of discordant progressive disease (n=31)* Location of concordant progressive disease (n=46)*
Target lesion progression (n=1)† Non-target lesion progression (n=9)† New lesion progression
(n=45)† 		New lesion progression (n=132)†

  • Peritoneum (peritoneal implant along the stomach, increase to 16 mm [from 10 mm at baseline])

  • Peritoneum, n=3 (33%)

  • Lymph nodes, n=2 (22%)

  • Pelvis, n=1 (11%)

  • Ascites, n=1 (11%)

  • Stomach, n=1 (11%)

  • Pleural effusion, n=1 (11%)

  • Peritoneum, n=11 (24%)

  • Pelvis, n=9 (20%)

  • Lymph nodes, n=7 (16%)

  • Ascites, n=6 (13%)

  • Stomach, n=3 (7%)

  • Liver, n=2 (4%)

  • Pleural effusion, n=2 (4%)

  • Breast, n=1 (2%)

  • Lung, n=1 (2%)

  • Other, n=3 (7%)

  • Peritoneum, n=50 (38%)

  • Pleura, n=29 (22%)

  • Lymph nodes, n=12 (9%)

  • Liver, n=11 (8%)

  • Ascites, n=5 (4%)

  • Colon, n=4 (3%)

  • Brain, n=2 (2%)

  • Pelvis, n=2 (2%)

  • Pleural effusion, n=2 (2%)

  • Stomach, n=1 (1%)

  • Other, n=14 (11%)
Open in a new tab

*Number of patients with discordant or concordant progressive disease where each patient could have progressive disease composed of multiple lesions across the categories listed.

†Number of lesions considered for progressive disease/non-progressive disease by investigator or blinded independent centralized review.

Intervention Training and Post-Intervention Discordance

While these metrics were being calculated, the reviewers underwent the previously described intervention training in July 2018. As a result, 10 of the most essential considerations in terms of best practices for radiologists to reduce reader variability were identified, and are described in table 3. Crucial changes to the PRIMA imaging charter were also identified (see table 3). These changes included evaluating coronal and sagittal scan reconstructions to improve identification of disease, such as peritoneal/omental lesions, or for enhanced demarcation from nearby anatomical structures (eg, bowel). All cases reviewed before May 2018 were re-reviewed after the training intervention, and all new cases were reviewed according to the updated PRIMA imaging charter. Due to high reader variability between pre-intervention and post-intervention, two out of five reviewers were removed from the PRIMA study and replaced by one additional reviewer who was hired post-intervention, but who also underwent the intervention training separately from the other reviewers. All PRIMA scans read by these two reviewers were re-read prior to the primary data lock for the PRIMA study.

Table 3.

Radiology best practices and changes to PRIMA imaging charter post-intervention

Item Radiology best practices14 Blinded independent centralized review charter changes post-intervention*
1 Readers should be familiar with the patterns of metastatic disease in ovarian cancer: direct extension, peritoneal, lymphatic, and hematogenous Included verbally in training
2 Peritoneal spread is the most common mode of metastatic disease in ovarian cancer, and knowledge of the flow and collection of peritoneal fluid can help guide a disease-specific search pattern Included verbally in training
3 Coronal and sagittal reconstructions should be routinely obtained and can aid in the detection of peritoneal metastases, particularly along the diaphragm Applied in multiple sections of charter, including:

  • If coronal or sagittal images are available, they will also be tracked and independently reviewed

  • All lesions visualized on CT/MRI will be marked or measured only in the axial plane. However, coronal and sagittal planes, if available, will be presented for independent review to assist in selection of disease like peritoneal/omental nodules, or for better demarcation from nearby anatomical structures like bowel, mesentery, etc
4 RECIST v1.1 should be applied in the context of ovarian cancer. If measurable, peritoneal lesions may serve as target lesions and the peritoneum should be treated as a single organ with a maximum of two target lesions Included verbally in training
5 Peritoneal and omental lesions need to be carefully selected and may be present only as non-measurable disease Omental (cake)/peritoneal (nodules) lesions

  • Evidence of omental (caking) or peritoneal (nodules) lesions present at baseline will be recorded as non-target lesions

  • Omental (cake) or peritoneal (nodules) lesions alone may drive an overall tumor assessment of progressive disease (eg, if presence ranges). The imaging appearance of omental disease may range from haziness to discrete nodules to omental cakes. Peritoneal disease may range from abnormal enhancement to thickening to discrete nodules


New:

  • Unequivocal new: if there is an unequivocal new appearance of omental or peritoneal (nodules) lesions, (resulting in increased thickened solid omentum/peritoneum) the radiologist will mark the new lesion as an unequivocal new lesion, and assign the overall tumor assessment as progressive disease

  • Equivocal new: if there is an equivocal new omental (cake) or peritoneal (nodules) lesion, the radiologist will mark the equivocal new lesion as an equivocal new lesion and assign the overall tumor assessment based on disease elsewhere. Comments should be entered describing the new equivocal lesion
6 Beware of choosing target lesions near the diaphragm, as they may change orientation with differences in respiration Included verbally in training
7 Ascites can drive progression if substantially increasing (eg, trace to large) and fits the overall disease burden As the presence of ascites can be common in assessing progression in ovarian cancer, follow the below guidance at follow-up time points:

  • In the case of (abnormal and) substantially enlarging ascites which has unequivocally increased, the radiologist will…

  • In the case of (minimal or moderately) enlarging ascites which has not unequivocally increased, the radiologist will…

  • In the case of substantial (and abnormal) new accumulation, the radiologist will mark the new ascites as…

  • …new lesion and update the (minimal or moderate) new accumulation at…

  • Ascites alone may drive an overall tumor assessment of progressive disease, for example, if (presence is) loculated, in multiple quadrants, in lesser sac and/or has a malignant appearance

  • In the case of (minimal or moderately) new accumulations which are considered equivocal, the radiologist will not assign progressive disease for the overall tumor assessment. The reviewer may capture this as an equivocal new lesion and update the (minimal or moderately) new accumulation at a follow-up time point if it becomes unequivocal


Global radiology review:

  • The independent radiologist may only update the time point assessments if clinical data is available and confirms the ascites is benign.

  • (Do not update an assessment to progressive disease based on equivocal new ascites becoming unequivocal.) In the case of equivocal ascites that becomes unequivocal for progressive disease, the time point of progressive disease should be “back dated” to (remain) the time point when equivocal ascites was first observed (became unequivocal). Cytology is not required for determination of progressive disease
8 Pleural fluid does not necessarily indicate progression unless there are unequivocal signs of thoracic metastatic disease or cytology As the presence of pleural effusions is not commonly used in assessing progression in ovarian cancer, follow the below guidance at follow-up time points:

  • Pleural effusion alone will not drive an overall tumor assessment of progressive disease without presence of unequivocal progression elsewhere. Evidence of unequivocal progression in the pleura could include but is not limited to (if new nodules are present) new pleural soft tissue nodules, an unequivocal increase of previously existing soft tissue nodules (in the pleura, and unequivocal increase or if there is) new pleural thickening/enhancement (this may be a cause for progression)


(As the presence of pleural effusions is not commonly used in assessing progression), follow the below guidance at follow-up time points:
Enlarging:

  • In the case of (abnormal and substantially) enlarging pleural effusions which increased unequivocally, for example, trace to large…

  • In the case of (minimal or moderately) enlarging pleural effusion which has not unequivocally increased…


Global Radiology Review:

  • (In the case of unequivocal pleural effusion that becomes equivocal, the time point of progressive disease should be “back dated” to the time point when equivocal pleural effusion was first observed.) The independent radiologist may update the time point assessments to progressive disease if in retrospect there are other sources of worsening, for example, presence of new nodular enhancement. Cytology is not required for determination of progressive disease

  • (The independent radiologist may update the time point assessments to progressive disease if clinical data are available and confirm any unequivocal pleural effusion as malignant)
9 Visceral peritoneal metastases should not be mistaken for parenchymal metastases. They can invade into the parenchyma but have a different prognosis than hematogenous metastases Included verbally in training
10 If a new finding is equivocal and could be explained by another process, then it is better to annotate/comment and review at the next time point rather than indicate progression. An assessment of progression can be backdated, if necessary, when a lesion becomes unequivocal If an equivocal lesion becomes unequivocal, the time point of progressive disease will be backdated on the Global Radiology Form. For certain exceptions, see Special Considerations in New and/or Enlarging Pleural/Pericardial Effusions and/or Ascites and Non-Measurable Disease
Open in a new tab

*Changes to the PRIMA imaging charter post-intervention are described, and text deletions are shown as (text).

CT/MRI, computerized tomography/magnetic resonance imaging.

PRIMA Outcomes

The median progression-free survival was 13.8 months with niraparib and 8.2 months with placebo in the overall patient population per blinded independent centralized reviewer assessment (HR 0.62, 95% confidence interval [CI] 0.50–0.76; p<0.001). A sensitivity analysis showed similarity with investigator-assessed progression-free survival: median progression-free survival was 13.8 months with niraparib and 8.2 months with placebo in the overall patient population (HR 0.63, 95% CI 0.51–0.76; p<0.001; figure 2). Across the overall and HRD-positive patient populations, there was a final concordance of 88% and 87%, respectively, between blinded independent centralized review and investigator-assessed progressive disease.

Figure 2.

Figure 2

Open in a new tab

Progression-free survival Kaplan–Meier curves by blinded independent centralized review and investigator assessment: (A) Overall; (B) HRD-positive; (C) HRD-positive BRCAm; (D) HRD-positive BRCAwt; (E) HRD-negative. BICR, blinded independent central review; BRCAm, breast cancer gene mutated; BRCAwt, breast cancer gene wild-type; CI, confidence interval; HRD, homologous recombination deficient.

In the overall population, 52% (n=255/487) of niraparib treated subjects and 37% (n=91/246) of placebo-treated subjects were censored per blinded independent centralized review assessment. The number of patients censored by investigators were similar (48% [n=232/487] of niraparib patients and 33% [n=80/246] of placebo patients). Reasons for censoring included: no evaluable baseline or no evaluable post-baseline radiological assessments; patients who have not progressed or died and have not started subsequent anti-cancer therapy; patients who have progressed or died after the start of subsequent anti-cancer therapy; and patients who have progressed or died after ≥2 missed radiological assessments (25 weeks allowing for visit window). Of the patients who had investigator-assessed progressive disease, 68 patients did not have progression confirmed by blinded central review.

The majority of these patients (71%, n=48) were censored on the same day that the investigator called progression; of the remaining 20 patients (29%), 11 patients had blinded central reviewer confirmed progressive disease, and 9 patients were censored. For those 11 patients with progression confirmed by blinded central review, the median time from investigator confirmed progressive disease to blinded central reviewer confirmed progressive disease was 2.8 months (range 0.7–7.3 months) for the niraparib arm (n=7) and 3.3 months (range 0.07–7.9 months) for the placebo arm (n=4). Event observations were also comparable between blinded independent centralized reviewers and investigators: 48% (n=232/487) versus 52% (255/487) for patients receiving niraparib, respectively, and 63% (n=155/246) versus 68% (n=166/246) for patients receiving placebo. In total, 20% (n=145/733) of patients enrolled in the PRIMA trial discontinued from the study (reasons included loss of follow-up, death, and withdrawal of consent), of which 62% (n=90/145) of patients were receiving niraparib.

At the primary data cut-off (May 2019), final discordance rates in the overall population decreased from 39% to 12% post-intervention training. Discordance rates between blinded independent centralized reviewers and investigators were also determined by biomarker subgroup, and ranged from 10–15% (HRD-positive 13% [n=48/373]; HRD-positive BRCAm 15% [n=33/223]; HRD-positive BRCAwt 10% [n=15/150]; HRD-negative 13% [n=32/249]). Similar observations in progression-free survival assessments were demonstrated across all biomarker subgroups (figure 2); the largest variation between blinded independent centralized reviewers and investigators was demonstrated in the HRD-positive BRCAwt subgroup (progression-free survival 19.6 months [HR 0.50; 95% CI 0.30–0.83] and 16.6 months [HR 0.60; 95% CI 0.37–0.95], respectively). In the final PRIMA study, of the 733 patients who were randomized, 360 (49%) patients were deemed to have progressive disease by both blinded independent centralized reviewers and investigators; 27 (4%) patients (16/487 patients receiving niraparib and 11/246 patients receiving placebo) were deemed to have progressive disease by blinded independent centralized review only.

DISCUSSION

Summary of Main Results

In our study, discrepant reads were identified during an early-phase assessment of PRIMA, with an average of 39% discordance between investigator assessment and blinded independent centralized review assessment. After the intervention, an improvement in the discordance rate was noted, with a decrease from 39% to 12%. When assessed by biomarker status, discordance rates were similar across subgroups, with the HRD-positive BRCAwt population, the smallest subgroup, having the highest discordance rate of 15%.

Results in the Context of Published Literature

Other ovarian cancer trials have reported high concordance between local and independent review committee assessments of progression status.1 16 In GOG-0218, concordance levels for both progressive disease and non-progressive disease determination ranged from 74% to 79% across the three study arms assessed.1 AGO-OVAR16 reported 84% and 86% concordance for assessment of disease progression between the treatment and placebo arms of the study, respectively.16 In a meta-analysis of phase 3 oncology trials,17 the mean proportion of discord on timing or occurrence of disease progression was just over 50% across 12 contributing studies. Importantly, the GOG-0218 analysis and the meta-analysis concluded that discrepancies were likely owing to variability in the process instead of systematic bias.1 17

However, there are negative trial consequences of high discordance rates between investigator and blinded independent centralized review. When the local investigator concludes progressive disease but blinded independent centralized review does not, enrolled patients are typically removed from the study without further scans, and the patient is censored at the time of the last negative evaluation, potentially biasing endpoint estimates if unbalanced across study arms, and reducing power to detect true treatment effects.9 18

Furthermore, discordance between blinded independent centralized reviewers may be problematic, and requires an adjudicator, who is an independent radiologist used to resolve discrepant reads between blinded independent centralized reviewers. Across 79 oncology trials, the proportion of cases in which a lack of consensus between reviewers required a third adjudicator was 42% overall, but 55% in ovarian cancer, making ovarian cancer the second-most-likely cancer to require adjudication.19 Furthermore, it has been emerging that the pattern of first recurrence during or after PARP inhibitor therapy is frequently oligometastatic disease.20 In the recurrent setting, approximately 30–50% of patients receiving PARP inhibitors may recur without significant increase in CA125 serum levels.21 22 Therefore, understanding the biology of the disease and recurrence patterns with specific agents is important for evaluation of disease response, and should be performed by a team of clinicians and radiologists with expertise in their specialized areas for optimal disease assessment.20–22

Strengths and Weaknesses

Our study demonstrates the effectiveness of implementing disease-specific training for radiologists in front-line oncology trials. We identified 10 best practices to reduce reader variability; however, there are some limitations to consider. Study investigators did not participate in the intervention training, as they were typically clinicians or radiologists with expertise in ovarian cancer and had access to a patient’s clinical data. A refresher course of RECIST v1.1 guidelines and PRIMA protocol definitions of clinical progression were available to study investigators, provided by the sponsor (Tesaro/GSK). There was particular emphasis on GCIG criteria for CA125 progression components and the importance of symptoms and other imaging.

The refresher training was not obligatory for study sites’ radiologists as they were not contractually bound by Tesaro/GSK. Additionally, in cases where there was no designated study radiologist, patient imaging data would be reviewed by the on-duty radiologist, who would also typically have experience in ovarian cancer imaging. Finally, recent data demonstrates agreement between local evaluation and blinded centralized review of progression-free survival endpoints in clinical trials, and therefore may obviate the need for the costly and time-consuming blinded independent central review for all patients in trials.23 This concordance has been demonstrated in a wide variety of hematologic and solid tumors, including ovarian cancer.16 24 25

Implications for Practice and Future Research

Several factors may contribute to the difficulty of determining radiologic disease progression in ovarian cancer, for example, the presence of surgical scarring and peritoneal spread occurring in the form of discrete measurable target lesions or non-target lesions (eg, ascites, omental nodularity/caking).1 16 This is reflected in our study, as target, non-target, and new lesion discordance was most often due to the presence of peritoneal carcinomatosis (table 2). Our study demonstrates that specialized disease-specific training can improve concordance among blinded independent centralized review reviewers, and this is particularly relevant when there may be more heterogeneity in reader experience with cancers that may spread via the peritoneum, as disease-specific expertise among blinded central reviewers cannot always be guaranteed in trials. Final results from the PRIMA trial showed consistency between investigator and blinded independent centralized review results in sensitivity analyses.11 Elements of this training highlight several best practice procedures (table 3) that can be implemented during radiology review, which may be useful in optimizing blinded independent centralized review assessments and thereby improve validity of endpoint assessment.

As a result of the training, several amendments were made to the PRIMA imaging charter. These include routinely obtaining coronal and sagittal reconstructions, as previously these additional planes would not be acquired or provided to blinded independent centralized review radiologists per the original imaging charter. While all lesions visualized by CT/MRI were marked or measured in the axial plane, coronal or sagittal reconstructions aid in the identification and selection of peritoneal nodules, or more clearly delineate lesions from nearby organs such as the bowel. Per the updated imaging charter, coronal and sagittal reconstructions would be reviewed and considered by blinded independent centralized review radiologists. Additionally, blinded independent centralized radiologists reviewed patient scans regardless of whether the scan was scheduled per the study protocol or was unscheduled i.e., due to clinical symptoms such as increased CA125. CA125 levels are routinely obtained in randomized clinical trials where progression-free survival is the primary endpoint.

Previously, investigators would declare progression and initiate second-line treatment based solely on CA125 levels regardless of clinical evidence of measurable disease, potentially lowering the threshold of defining progression and skewing final progression-free survival measurements in trials. Guidelines were then published by GCIG recommending that after first-line therapy, the doubling of CA125 from the upper limit of normal, or doubling from the post-treatment nadir, would be suggestive of disease progression.26 Revisions to the definition of progression include declaring a patient to have progressive disease based on either RECIST criteria or CA125 levels, and is defined as the date of the earlier of the two events if both are documented.26

A new modality in diagnostic imaging is radiomic analysis from CT or MRI data that, when applied to ovarian cancer, has provided valuable information on tumor biology.27 28 New applications enable the use of these data to identify radiomic features that correlate with specific genotypic and phenotypic features that may enable predications of disease progression or prognosis. This modality is especially advantageous for highly heterogenous tumors by enabling the evaluation of the entire tumor or tumor burden, which is not possible through biopsy, while still allowing the characterization and quantification of disease progression and drug resistance.27–29

CONCLUSION

Overall, for clinical trials using blinded independent centralized review assessment as a primary endpoint, we have demonstrated the importance of fully optimizing blinded independent centralized review. Furthermore, we have shown in PRIMA that initial blinded independent centralized review and investigator discordance was greatly improved, indicating a benefit in using specialized, disease-specific training, and highlighting the need to evaluate concordance early on in trials.

Acknowledgments

The authors would like to acknowledge Hasan H Jamal, MSc at GSK, for their review and coordination of the manuscript. Medical writing support was provided by Claire Kelly, PhD, of Fishawack Indicia Ltd, UK, part of Avalere Health, funded by GSK. The authors would like to acknowledge the contributions made to this publication by the teams at Paraxel and Calyx. Data from this study were previously presented, in part, as a poster presentation at the 2022 International Gynecologic Cancer Society (IGCS) annual meeting by Herzog, T.J. et al, EP224/#531 Optimization of Assessment of Disease Progression Between Blinded Central Independent Review and Investigator Assessment in the PRIMA/ENGOT-ov26/GOG-3012 Trial (abstract Herzog, T.J et al, IJGC. 2022;32: suppl 3).

Footnotes

Twitter: @Thomas Herzog

Contributors: TJH: Conceptualization; Data curation; Roles/Writing - original draft; Writing - review and editing.; SAW: Conceptualization; Data curation; Roles/Writing - original draft; Writing - review and editing; MRM: Conceptualization; Data curation; Roles/Writing - original draft; Writing - review and editing; BP: Conceptualization; Data curation; Roles/Writing - original draft; Writing - review and editing; IV: Conceptualization; Data curation; Roles/Writing - original draft; Writing - review and editing; WSG: Conceptualization; Data curation; Roles/Writing - original draft; Writing - review and editing; IAM: Conceptualization; Data curation; Validation; Data Curation; Roles/Writing - original draft; Writing - review and editing; WY: Data Curation; Roles/Writing - original draft; Writing - review and editing; JAH: Conceptualization; Data Curation; Roles/Writing - original draft; Writing - review and editing; DG: Conceptualization; Data curation; Validation; Data Curation; Roles/Writing - original draft; Writing - review and editing; AGM: Conceptualization; Data curation; Roles/Writing - original draft; Writing - review and editing; BJM: Conceptualization; Data curation; Roles/Writing - original draft; Writing - review and editing. TJH is guarantor for this manuscript.

Funding: This study was funded by GSK (GSK study 213359). GSK contributed to study design, implementation, data collection, interpretation, and analysis.

Competing interests: TJH has served on advisory boards (Aravive, AstraZeneca, Caris, Clovis Oncology, Eisai, Epsilogen, GSK, Immunogen, Johnson & Johnson, Merck, Roche Genentech, Seagen) and as a consultant for Abbvie. SAW reports consulting fees (GSK and BioClinica). MRM reports personal fees and other (Karyopharm Therapeutics, Sera Prognostics, Roche); institutional grants and no financial interest (Apexigen, AstraZeneca, Deciphera, GSK, Ultimovacs); personal fees and invited speaker (AstraZeneca, GSK); personal fees and advisory boards (AstraZeneca, Biocad, Boehringer Ingelheim, GSK, Karyopharm, Merck, Mersana, ImmunoGen, Clovis Oncology, Roche, Zailab); personal fees, stocks and a member of board of directors (Karyopharm). BP reports research funding (AstraZeneca, Celgene, Celsion, Clovis, Genentech, SeaGen, GSK, Merck, Mersana, SeaGen, Takada, Toray) and consulting fees (AstraZeneca, Celsion, Eisai, GSK, GOG Foundation, Immunogen, Inxmed, Lily, Merck, Mersana, SeaGen, Toray). IV reports consulting fees (Agenus, Akesobio, AstraZeneca, Bristol Myers Squibb, Deciphera Pharmaceuticals, Eisai, Elevar Therapeutics, Exelixis, F. Hoffmann-La Roche, Genmab, GSK, Immunogen, Jazzpharma, Karyopharm, Mersana, MSD, Novocure, Novartis, Oncoinvent, OncoXerna, Regeneron, Sanofi, Seagen, Sotio, Verastem Oncology, Zentalis), travel grants (Karyopharm, Genmab, Novocure), and contracted research grants (Amgen, Roche, Oncoinvent AS). WSG reports advisory board and speaker fees (GSK). IAM, WY, and DG are employees of GSK. JAH is an employee of GSK and reports stocks in GSK. AGM reports personal fees for educational/advisory-related activities (Alkermes, Amgen, AstraZeneca, Clovis, Genmab, GSK, Immunogen, Mersana, MSD, Novocure, Oncoinvent, PharmaMar, Roche, SOTIO, Takeda). BJM reports consulting fees (Acrivon, Adaptimmune, Agenus, Akeso Bio, Amgen, Aravive, AstraZeneca, Bayer, Clovis, Eisai, Elevar, EMD Merck, Genmab/Seagen, GOG Foundation, Gradalis, Heng Rui, ImmunoGen, Karyoparm, Iovance, Laekna, Macrogenics, Merck, Mersana, Myriad, Novartis, Novocure, OncoC4, Panavance, Pieris, Pfizer, Puma, Regeneron, Roche/Genentech, Sorrento, GSK, US Oncology Research, VBL, Verstem, Zentalis); speakers honoraria (AstraZeneca, Clovis, Eisai, Merck, Myriad, Roche/Genentech, GSK); investigator honoraria (Gradalis).

Provenance and peer review: Not commissioned; externally peer reviewed.

Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Data availability statement

Data are available upon reasonable request. GSK makes available anonymized individual participant data and associated documents from interventional clinical studies that evaluate medicines, upon approval of proposals submitted to https://www.gsk-studyregister.com/en/.

Ethics statements

Patient consent for publication

Not applicable.

References

  • 1. Burger RA, Brady MF, Bookman MA, et al. Incorporation of bevacizumab in the primary treatment of ovarian cancer. N Engl J Med 2011;365:2473–83. 10.1056/NEJMoa1104390 [DOI] [PubMed] [Google Scholar]
  • 2. Moore K, Colombo N, Scambia G, et al. Maintenance olaparib in patients with newly diagnosed advanced ovarian cancer. N Engl J Med 2018;379:2495–505. 10.1056/NEJMoa1810858 [DOI] [PubMed] [Google Scholar]
  • 3. Perren TJ, Swart AM, Pfisterer J, et al. A phase 3 trial of bevacizumab in ovarian cancer. N Engl J Med 2011;365:2484–96. 10.1056/NEJMoa1103799 [DOI] [PubMed] [Google Scholar]
  • 4. Vergote I, Gonzalez-Martin A, Lorusso D, et al. Clinical research in ovarian cancer: consensus recommendations from the gynecologic cancer intergroup. Lancet Oncol 2022;23:e374–84. 10.1016/S1470-2045(22)00139-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5. Loreen A, Polen-De C, Monk BJ, et al. The role of blinded independent radiologic review in ovarian cancer clinical trials: discerning the value. Gynecol Oncol 2021;161:491–5. 10.1016/j.ygyno.2021.02.031 [DOI] [PubMed] [Google Scholar]
  • 6. O’Malley D. Ovarian cancer in focus: update on drugs in advanced ovarian cancer. Clin Adv Hematol Oncol 2020;18:308–11. [PubMed] [Google Scholar]
  • 7. Herzog TJ, Ison G, Alvarez RD, et al. FDA ovarian cancer clinical trial endpoints workshop: a society of gynecologic oncology white paper. Gynecol Oncol 2017;147:3–10. 10.1016/j.ygyno.2017.08.012 [DOI] [PubMed] [Google Scholar]
  • 8. Stone A, Gebski V, Davidson R, et al. Exaggeration of PFS by blinded, independent, central review (BICR). Ann Oncol 2019;30:332–8. 10.1093/annonc/mdy514 [DOI] [PubMed] [Google Scholar]
  • 9. Ford R, Schwartz L, Dancey J, et al. Lessons learned from independent central review. Eur J Cancer 2009;45:268–74. 10.1016/j.ejca.2008.10.031 [DOI] [PubMed] [Google Scholar]
  • 10. King M. Ovarian cancer — difficulties in monitoring response. Cancer Imaging 2015;2:34–6. [Google Scholar]
  • 11. González-Martín A, Pothuri B, Vergote I, et al. Niraparib in patients with newly diagnosed advanced ovarian cancer. N Engl J Med 2019;381:2391–402. 10.1056/NEJMoa1910962 [DOI] [PubMed] [Google Scholar]
  • 12. U.S.Food and Drug Administration . FDA approves niraparib for first-line maintenance of advanced ovarian cancer. 2020.
  • 13. Agency EM. GSK: Niraparib summary of product characteristics. 2021.
  • 14. Eisenhauer EA, Therasse P, Bogaerts J, et al. New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1). Eur J Cancer 2009;45:228–47. 10.1016/j.ejca.2008.10.026 [DOI] [PubMed] [Google Scholar]
  • 15. Rustin GJS, Vergote I, Eisenhauer E, et al. Definitions for response and progression in ovarian cancer clinical trials incorporating RECIST 1.1 and CA 125 agreed by the gynecological cancer intergroup (GCIG). Int J Gynecol Cancer 2011;21:419–23. 10.1097/IGC.0b013e3182070f17 [DOI] [PubMed] [Google Scholar]
  • 16. Floquet A, Vergote I, Colombo N, et al. Progression-free survival by local investigator versus independent central review: comparative analysis of the AGO-Ovar16 trial. Gynecol Oncol 2015;136:37–42. 10.1016/j.ygyno.2014.11.074 [DOI] [PubMed] [Google Scholar]
  • 17. Amit O, Mannino F, Stone AM, et al. Blinded independent central review of progression in cancer clinical trials: results from a meta-analysis. Eur J Cancer 2011;47:1772–8. 10.1016/j.ejca.2011.02.013 [DOI] [PubMed] [Google Scholar]
  • 18. Dodd LE, Korn EL, Freidlin B, et al. Blinded independent central review of progression-free survival in phase III clinical trials: important design element or unnecessary expense J Clin Oncol 2008;26:3791–6. 10.1200/JCO.2008.16.1711 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19. Ford RON M, Moskowitz SC, Fraunberger J. Adjudication rates between readers in blinded independent central review of oncology studies. J Clin Trials 2016;06:1–9. 10.4172/2167-0870.1000289 [DOI] [Google Scholar]
  • 20. Kamavra M, Gonzalez-Martin A, Pothuri B, et al. EP233/#541 patterns of initial ovarian cancer recurrence on niraparib maintenance monotherapy in patients with no baseline evidence of disease following first-line chemotherapy: PRIMA/ENGOT-ov26/GOG-3012 post-hoc subgroup analysis. IGCS 2022 Annual Meeting Abstracts; December 2022. 10.1136/ijgc-2022-igcs.324 [DOI] [Google Scholar]
  • 21. Boccia SM, Marchetti C, Chiamenti C, et al. 2022-RA-1414-ESGO pattern of recurrence and CA125 monitoring in BRCA wild-typerecurrent ovarian cancer patients under maintenance with niraparib. ESGO 2022 Congress; October 2022. 10.1136/ijgc-2022-ESGO.711 [DOI] [Google Scholar]
  • 22. Tjokrowidjaja A, Lee CK, Friedlander M, et al. Concordance between CA-125 and RECIST progression in patients with germline BRCA-mutated platinum-sensitive relapsed ovarian cancer treated in the Solo2 trial with olaparib as maintenance therapy after response to chemotherapy. Eur J Cancer 2020;139:59–67. 10.1016/j.ejca.2020.08.021 [DOI] [PubMed] [Google Scholar]
  • 23. Zhang JJ, Chen H, He K, et al. Evaluation of blinded independent central review of tumor progression in oncology clinical trials: a meta-analysis. Ther Innov Regul Sci 2013;47:167–74. 10.1177/0092861512459733 [DOI] [PubMed] [Google Scholar]
  • 24. Lian Q, Fredrickson J, Boudier K, et al. Meta-analysis of 49 roche oncology trials comparing blinded independent central review (BICR) and local evaluation to assess the value of BICR. Oncologist 2023:oyad012. 10.1093/oncolo/oyad012 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25. Zhang J, Zhang Y, Tang S, et al. Systematic bias between blinded independent central review and local assessment: literature review and analyses of 76 phase III randomised controlled trials in 45 688 patients with advanced solid tumour. BMJ Open 2018;8:e017240. 10.1136/bmjopen-2017-017240 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26. Vergote I, Rustin GJ, Eisenhauer EA, et al. Re: new guidelines to evaluate the response to treatment in solid tumors[ovarian cancer]. J Natl Cancer Inst 2000;92:1534–5. 10.1093/jnci/92.18.1534 [DOI] [PubMed] [Google Scholar]
  • 27. Rizzo S, Botta F, Raimondi S, et al. Radiomics of high-grade serous ovarian cancer: association between quantitative CT features, residual tumour and disease progression within 12 months. Eur Radiol 2018;28:4849–59. 10.1007/s00330-018-5389-z [DOI] [PubMed] [Google Scholar]
  • 28. Vargas HA, Veeraraghavan H, Micco M, et al. A novel representation of inter-site tumour heterogeneity from pre-treatment computed tomography textures classifies ovarian cancers by clinical outcome. Eur Radiol 2017;27:3991–4001. 10.1007/s00330-017-4779-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29. Forstner R. Early detection of ovarian cancer. Eur Radiol 2020;30:5370–3. 10.1007/s00330-020-06937-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30. U.S. Food and Drug Administration . Clinical trial endpoints for the approval of cancer drugs and biologics. Guidance for Industry 2018. [Google Scholar]
  • 31. European medicines agency . Appendix 1 to the guideline on the evaluation of anticancer medicinal products in man-methodological consideration for using progression-free survival (PFS) or disease-free survival (DFS) in Confirmatory trials. 2012.

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary data

ijgc-2023-004605supp001.pdf (245.6KB, pdf)

Data Availability Statement

Data are available upon reasonable request. GSK makes available anonymized individual participant data and associated documents from interventional clinical studies that evaluate medicines, upon approval of proposals submitted to https://www.gsk-studyregister.com/en/.

Articles from International Journal of Gynecological Cancer are provided here courtesy of BMJ Publishing Group

RESOURCES