Publication of Tumor Marker Research Results: The Necessity for Complete and Transparent Reporting

Abstract

Clinical management decisions for patients with cancer are increasingly being guided by prognostic and predictive markers. Use of these markers should be based on a sufficiently comprehensive body of unbiased evidence to establish that benefits to patients outweigh harms and to justify expenditure of health care dollars. Careful assessments of the clinical utility of markers by using comparative effectiveness research methods are urgently needed to more rigorously summarize and evaluate the evidence, but multiple factors have made such assessments difficult. The literature on tumor markers is plagued by nonpublication bias, selective reporting, and incomplete reporting. Several measures to address these problems are discussed, including development of a tumor marker study registry, greater attention to assay analytic performance and specimen quality, use of more rigorous study designs and analysis plans to establish clinical utility, and adherence to higher standards for reporting tumor marker studies. More complete and transparent reporting by adhering to criteria such as BRISQ [Biospecimen Reporting for Improved Study Quality] criteria for reporting details about specimens and REMARK [Reporting Recommendations for Tumor Marker Prognostic Studies] criteria for reporting a multitude of aspects relating to study design, analysis, and results, is essential for reliable assessment of study quality, detection of potential biases, and proper interpretation of study findings. Adopting these measures will improve the quality of the body of evidence available for comparative effectiveness research and enhance the ability to establish the clinical utility of prognostic and predictive tumor markers.

INTRODUCTION

Predictive and prognostic tumor markers are playing an increasingly important role in personalized oncologic patient care.¹ These markers range from conventional single-protein-, RNA-, or DNA-based markers to molecular signatures based on multiplex assays. As the number of available markers continues to increase and to result in substantial expenditure of health care dollars, there is a pressing need to perform critical reviews of the body of evidence that supports claims of the clinical utility of these markers. Prognostic and predictive markers are used to guide clinical management of patients with established cancer diagnoses. Pure prognostic markers distinguish the clinical outcomes of subgroups of patients (eg, those who are positive v negative for the marker) in the absence of a future, considered therapy, assuming the patients will receive either no treatment or some selected base treatment (eg, local therapies such as surgery and/or radiation). A strong prognostic factor may be able to identify patients with cancers that are so likely to be cured with the base treatment that additional therapy is not needed, even if it has activity. When further therapy is deemed necessary, predictive markers identify patient subpopulations that will or will not derive substantial benefit from promising new targeted therapies. For example, estrogen receptor and human epidermal growth factor receptor 2 (HER2) status in breast cancer predict benefit or resistance to endocrine and anti-HER2 therapies, respectively.^2,3 More recent investigations have demonstrated that ALK translocations in lung cancer and the absence of KRAS mutations in colorectal cancers indicate benefit from crizotinib⁴ and anti–epidermal growth factor receptor antibodies,^5,6 respectively.

Given their critical importance in making clinical decisions, prognostic and predictive tumor markers should be subject to the same evidence-based medicine standards as other types of medical interventions and practices. Evidence-based medicine relies on access to complete and accurate information to draw reliable conclusions. We review the current state of efforts to enhance the quality and transparency of reporting of tumor marker studies.

TUMOR MARKER RESEARCH AND TRANSLATION TO THE CLINIC: IMPORTANT SEMANTICS

A clear consensus on definitions of terms is essential to understanding how to translate tumor marker research to standard clinical practice. First, it is important to delineate the intended clinical use of the marker; for example, distinguishing between prognostic and predictive roles. Other uses include risk categorization in unaffected individuals, screening for occult malignancy, differential diagnosis, and monitoring. Moreover, for a tumor marker to be used in making clinical management decisions, issues related to analytic and clinical validity, clinical utility, study designs and analysis, and comparative effectiveness research must be fully understood. Improved reporting strategies are critical to achieving this necessary level of understanding.

Validity Versus Utility

The term “validation” is widely used, but it means different things in different contexts.⁷ Recently, the Evaluation of Genomic Applications in Practice and Prevention (EGAPP) working group, convened by the Centers for Disease Control and Prevention, has designated three important terms that describe necessary steps in developing genetic and other types of markers: analytic validity, clinical validity, and clinical utility.⁸ Analytic validity refers to analytic accuracy, reliability, and reproducibility related to the marker assay or test in hand. A report on the analytic evaluation of the Oncotype DX assay⁹ provides an example of the types of performance characteristics that should be assessed to establish analytic validity. The National Cancer Institute's Cancer Diagnosis Program has also proposed guidelines for factors to be considered in the evaluation of assay performance¹⁰. Clinical validity is the demonstration that the test has a suitably strong association with a clinical outcome of interest. The clinical validity of the proliferation marker Ki67 as a prognostic indicator in breast cancer has been demonstrated in many studies, but clinical utility has not yet been established because of lack of harmonization of assay methodology, uncertain reproducibility, and lack of consensus on specific situations in which the marker might reliably inform clinical decisions.¹¹ We have added an additional term, biologic validity which, although not proposed by the EGAPP working group, implies that the marker might also be associated with a biologic feature or end point, perhaps but not necessarily associated with clinical outcomes. Gene expression profiling studies have identified biologically distinct subgroups of diffuse large B-cell lymphoma that include germinal center B-cell–like and activated B-cell–like diffuse large B-cell lymphoma.¹² These subgroups have been identified consistently across multiple studies and have been shown to associate with survival, even with changes in the assay platform¹³ and in a new treatment era with the addition of rituximab to standard chemotherapy.¹⁴ Although biologic validity for these subgroups has been established, potential clinical utility remains under active investigation.¹⁵ Regardless, although both are important components of tumor marker development, neither clinical nor biologic validity alone implies that the marker should be used to direct patient care. Most importantly, clinical utility implies that use of the marker test to direct patient care has been shown to result in a favorable balance of benefits to harm, leading to improved outcomes compared with nonuse of the marker test. Improvement in outcome may relate to overall survival, disease-free survival, quality of life, or cost of care.^7,16

Prospective Versus Prospective-Retrospective Tumor Marker Studies

Ideally, as with new therapeutics, the clinical utility of a predictive or prognostic tumor marker would be established with a high level of evidence generated in large, prospective trials. Several possible trial designs for prospectively testing tumor marker utility have been proposed.^17,18 In many respects, establishing the clinical utility of a marker can be a greater challenge than establishing efficacy of a drug. Such trials are large, time consuming, and expensive, and only a few have been or are being conducted, such as TAILORx [Trial Assigning Individualized Options for Treatment (Rx)],¹⁹ MINDACT [Microarray in Node-Negative and 1-3 Node-Positive Disease May Avoid Chemotherapy],²⁰ and RxPONDER [Rx for Positive Node, Endocrine Responsive Breast Cancer],²¹ to test multianalyte assays in breast cancer.

One advantage of tumor marker research compared with drug investigations is the ability to perform studies by using archived specimens linked to clinical annotation. However, this approach is a double-edged sword, since it also makes it easier to perform poorly designed and improperly controlled studies. If not properly planned, conducted, analyzed, and reported, such studies may demonstrate clinical/biologic validity, but they usually provide low levels of evidence to support clinical utility. Even worse, they may produce completely spurious false-positive or false-negative results. Recently, Simon et al²² have proposed a hierarchy of studies using archived specimens that produce varying levels of evidence. For studies using archived specimens, they propose that the best option is what they have termed a “prospective-retrospective” study. In this type of study, specimens that are collected, processed, and archived during the course of a prospective trial are analyzed retrospectively to test the clinical utility of a tumor marker. Such a trial can be a prospective registry of patients treated and observed uniformly to evaluate prognosis or a prospective, randomized treatment trial to address the predictive role of a tumor marker with the therapeutic strategy under investigation. These registries and trials collect clinical data prospectively by using trial-quality methodology. They also prospectively collect, process, and store tissue or other samples, which usually results in a higher percentage of available, high-quality specimens than in studies using specimen banks for which the specimens have been collected ad hoc (so-called “studies of convenience”).

Taken together, these conditions maximize the chances that tumor marker studies conducted by using prospectively collected, trial-quality specimens provide more accurate results and approximate high levels of evidence for clinical utility. Nonetheless, a critical requirement of a prospective-retrospective study is that the design and analytic approach proposed for the tumor marker investigation must be prospectively planned in writing. If planned, performed, and analyzed properly, prospective-retrospective studies can provide a high level of evidence for the utility of a marker (Level IB in the terminology of Simon et al²²) under certain conditions (Table 1). The requirement for multiple validating studies is uncertain, even for therapeutic agents. For example, although the US Food and Drug Administration and guidelines bodies have on occasion accepted a single prospective, randomized controlled trial to introduce a new drug into clinical practice, meta-analyses of multiple clinical trials are preferable for determining true clinical utility.^23–26 Simon et al have proposed the requirement that a prospective-retrospective study be confirmed by at least one additional prospective-retrospective study (with similar results) to establish clinical utility of a marker, compared with the need to conduct only a single truly prospective clinical trial that directly addresses clinical utility of the marker. This requirement is necessary because often there is limited statistical power to evaluate a marker's clinical utility within a single trial originally designed to answer a treatment efficacy question, and there might be heterogeneity among the patient populations or marker assays used in different studies. For example, if a single, properly powered, prospective clinical trial is considered sufficient for recommending use of a marker in clinical practice, Simon et al suggest that at least two prospective-retrospective studies with consistent results, that use the same or a similar assay for the same marker, and that are conducted by using specimens from similar trials or trial arms, be required to demonstrate clinical utility of a marker.

Table 1.

Requirements for a Marker-Based Test to Reach Level IB Evidence of Clinical Utility Based on Prospective-Retrospective Studies

1. Adequate amounts of archived specimen must be available from enough patients from a prospective trial (which for predictive factors should generally be a randomized design) for analyses to have adequate statistical power and for the patients included in the evaluation to be clearly representative of the patients in the trial. 2. The marker-based test should be analytically and preanalytically validated for use with archived specimens. 3. The plan for marker evaluation should be completely specified in writing before the performance of marker assays on archived specimens and should be focused on evaluation of a single completely defined marker-based test. 4. The results from archived specimens should be validated using specimens from one or more similar, but separate, studies.

NOTE. Guidelines adapted.²²

Comparative Effectiveness

Recently, the generalizability of results from prospective randomized clinical trials for standard clinical care has come into question.²⁷ Prospective trials are generally conducted under idealized conditions, for example, with potentially restrictive eligibility criteria that would exclude patients who are at higher risk of complications or adverse outcomes because of comorbid conditions. The term “comparative effectiveness research” has been proposed to designate whether results generated in such a pristine setting are applicable to the real world of typical patients seen in everyday practice.²⁸

There is some ambiguity in how the terms “comparative effectiveness research” and “evidence-based medicine” are used. We view evidence-based medicine as referring to the process of synthesizing available research evidence to draw conclusions about how to address specific clinical problems. We view comparative effectiveness research as encompassing broader considerations such as cost and feasibility,²⁹ thereby placing a higher emphasis on real-world settings and recognizing the impact of health care decisions at both the individual and population levels. Regardless of whether one is doing comparative effectiveness research or practicing evidence-based medicine, the quality of the evidence is of paramount importance. We hold the view that without complete and transparent reporting of studies, one can neither assess the quality of the evidence nor properly interpret it. Therefore, the ambiguity of these terms does not affect our conclusions about the importance of good reporting.

Comparative effectiveness considerations apply to all aspects of clinical care, including both therapeutics and diagnostics. Indeed, for real-world comparative effectiveness, one could argue that the tumor marker arena offers a better opportunity than that for therapeutic agents, since many specimen banks and associated clinical data annotation used to study tumor markers are collected within standard-of-care situations. However, this same circumstance also produces considerable risk of bias. Such specimens are often collected, processed, and stored without regard to careful preanalytic quality assurance and quality control, and the clinical data are usually gathered retrospectively and without the careful scrutiny and auditing that is used for clinical trials. Thus, although such banks may provide useful analytic and clinical/biologic validity during tumor marker development, they are rarely useful for determining true clinical utility or comparative effectiveness.

THE PROBLEM: THERE ARE FEW TUMOR MARKERS WITH CLINICAL UTILITY

Regrettably, most tumor marker studies fail to rigorously address analytic validity or clinical utility. Rather, most publications simply demonstrate clinical validity with a marker assay that is often poorly described and not demonstrated to be accurate, reliable, and reproducible outside of the respective research laboratory that developed it. Few tumor marker tests have been studied with sufficient rigor to generate the kind of high-level evidence needed to determine whether they have clinical utility.²² Indeed, 40 years into the remarkable biologic observations from the revolution in molecular biology and a decade after initiation of the “-omics” era,^29a only a few tumor markers are supported by evidence that would be considered Level 1. For example, the only tissue-based markers recommended for breast cancer by the Tumor Marker Guidelines Committee of the American Society of Clinical Oncology (ASCO) are estrogen and progesterone receptor testing for decisions about delivery of endocrine therapy, HER2 testing for anti-HER2 therapy, and the 21-gene recurrence score and urinary plasminogen activator and plasminogen activator inhibitor 1 assay to determine prognosis.³⁰ The situation in other solid tumors is even more dismal. In colorectal cancer, testing for KRAS mutations is recommended to select patients for treatment with antibodies against epidermal growth factor receptor.^5,6 In lung cancer, testing for ALK translocations, which signify likelihood of sensitivity to crizotinib,⁴ and testing for EGFR mutations, which are associated with benefit from tyrosine kinase inhibitor therapy, are also recommended.³¹ Although many other promising markers have been reported for prognosis and prediction of benefit from both targeted and routine therapies, few have advanced beyond the clinical validity phase.

OBSTACLES IN TRANSLATING TUMOR MARKER RESEARCH TO THE CLINIC

Several factors have impeded translation of findings from tumor marker research into clinically useful tests that meet the requirements of comparative effectiveness research for demonstrating clinical utility. These factors include an uncertain regulatory environment, inadequate reimbursement incentives, and a lack of structure for design and conduct of the studies necessary to achieve the high levels of evidence demanded by guidelines bodies, third-party payers, and patients and clinicians. These issues have been discussed in several other publications.^{16,22,32–40} However, a little recognized but major problem is the disorganization, incompleteness, and lack of transparency that characterize the publication environment for tumor marker studies. If tumor markers are to achieve the kind of clinical utility associated with new therapeutics, and if they are to be scrutinized with the tools of comparative effectiveness research, it is essential that researchers, journal editors and reviewers, guidelines and technical assessment panels, and clinicians and their patients have a clear understanding of what has not been reported or published, and even within published papers, what factors may have led to biases that could affect the results or interpretation of the reported findings.

Reporting Biases

Biased reporting of studies is a major threat to the reliability of comparative effectiveness research to determine the clinical utility of tumor markers. There is ample evidence for medical study reporting bias and outcome reporting bias⁴¹ and strong evidence for publication bias in tumor marker studies in particular.^42–44 Therefore, it is important to recognize these types of biases and for authors, editors, and reviewers to insist that these biases be minimized as much as possible.

We distinguish between three main types of reporting bias. The first arises from submission and acceptance bias, which is commonly called “publication bias,” or more accurately, “nonpublication” bias.⁴⁵ In this case, authors may decide not to report negative studies at all; if they do, those negative studies are much less likely to be published in highly regarded, high-impact journals. For example, in a meta-analysis of the tumor suppressor protein TP53 as a prognostic marker in head and neck cancer, substantial differences in the magnitude and statistical significance of observed prognostic effects were noted when comparing published studies to unpublished studies with retrievable data.⁴⁶

The second type of reporting bias has been termed “within-publication selective reporting.”^47–49 This type of bias occurs when an author elects to selectively report results for only a subset of study outcomes that were actually analyzed. Just as in the case of nonpublication bias, this practice is particularly misleading if the decision to report the result for a particular outcome is influenced by the statistical significance of or consistency with prior studies or expectations. Within-publication selective reporting can take many forms: reporting results only for time-to-event end points for which statistically significant associations with the marker are observed rather than reporting results for the most clinically relevant end point, failure to report results of multivariable analyses adjusting for standard variables unless the marker maintains its statistical significance, and reporting results based on optimized cut points that are selected to minimize the P value for the test of association between the dichotomized marker value and outcome.³³ In a meta-analysis of studies that examined proliferation markers in early-stage breast cancer,⁵⁰ more than a dozen different cut points ranging from 0% to 30% were applied among 26 studies that used a Ki67 assay based on the MIB1 antibody alone, which raises questions about whether some of the studies might have reported results for optimized cut points. Unfortunately, authors often fail to provide a rationale for their selection of cut point, so this potential bias often goes undetected.

A third type of bias results from a practice designated as “incomplete study reporting.”^47,48 In this case, the authors fail to provide sufficient detail regarding study design, conduct, and analysis within the papers accepted for publication. Reproducibility is one of the hallmarks of the scientific method, and complete and transparent reporting of investigational design and methodology is universally required for basic and clinical trial publications. However, for a variety of reasons, crucial factors in design and methodology of tumor marker studies, including preanalytic issues related to collection, processing, and storage of specimens, assay accuracy and variability, selection of patients for study, and critical details about their treatment and follow-up are often poorly documented, if at all. Henry and Hayes⁵¹ provide an example of how incomplete reporting of specific types of chemotherapies received by patients with breast cancer could lead to different conclusions about the direction of the association between HER2 status and efficacy of chemotherapy. Incomplete reporting that omits certain details might occur independently of any knowledge of results; nonetheless, missing details may lead to incorrect interpretation of study results which may lead to biased analyses and would prevent other researchers from reproducing the study findings.

STRATEGIES TO IMPROVE REPORTING OF TUMOR MARKER STUDIES

Biases due to nonpublication, selective reporting, and incomplete reporting of tumor marker studies are common and insidious.⁵² Collectively, these practices result in exaggerated claims of the significance of findings, complicate efforts to perform systematic reviews, confound analysis of clinical utility, and prevent estimates of comparative effectiveness.^{42–44,46,53,54} However, several strategies for avoiding selective reporting have been proposed, and these should be widely understood and followed by authors, editors, and reviewers so that tumor marker research gains the same legitimacy as basic and clinical therapeutic research. We maintain that acceptance of these simple steps will lead to better and more personalized oncologic patient care.

Establishment of a Prospective Registry of Tumor Marker Studies to Decrease Nonpublication Bias

Federal legislation was passed in late 1977 to mandate a registry for both federally and privately funded clinical trials of experimental treatments so that information about treatment options under investigation in clinical trials would be widely accessible in a form understandable to the general public.⁵⁵ In response to that mandate, the ClinicalTrials.gov searchable database was developed and launched online in February 2000. Each trial entered into the database is assigned a unique identifier, and essential information is captured, including a trial summary, eligibility criteria, interventions, study design, recruitment status, and sponsors. Usage of the system rose sharply beginning in 2004 when the International Committee of Medical Journal Editors (ICMJE) instituted a policy requiring that a clinical trial would have to be registered in ClinicalTrials.gov before the first participant was accrued for it to be considered for publication.⁵⁶ Updated legislation resulting from the 2007 US Food and Drug Administration Amendments Act requires that investigators enter results for the primary and secondary trial end points when they become available. A recent cross-sectional analysis of trial publication after registration in ClinicalTrials.gov provided evidence that the trials database could help identify a substantial number of unpublished studies.⁵⁷

A public registry specifically for tumor marker studies does not currently exist. Indeed, because archived specimens are often collected and stored for unspecified future use outside a specific written study protocol, it has been difficult to know what tumor marker studies are planned, in progress, or completed. In an effort to provide some transparency to this situation, an international panel of researchers has called for initiation of a comprehensive marker study registry in oncology, and a prototype web-based system is currently under development.^58,59 At the very least, this registry will incorporate prospectively conducted tumor marker trials, such as MINDACT,²⁰ TAILORx,¹⁹ and RxPONDER.²¹ A study summary and identifier, patient characteristics, a listing of all markers tested and assay technologies used, and a study contact are some of the data elements initially planned for inclusion in the database. Over time, it is hoped that prospective-retrospective studies would be listed in the registry, and perhaps eventually registration at time of study inception would become a requirement for publication of the study results. If implemented, such a registration system could substantially improve the ease of identification of unpublished marker studies.

Standards for Reporting Within Published Manuscripts to Decrease Biased and Inadequate Reporting

In addition to trial registries, reporting guidelines have been developed for several major types of health research studies to address the problems of incomplete and selective reporting within publications.^49,60 In 1994 independent efforts led to two proposals for standards for reporting of randomized controlled trials.^61,62 In 1996, representatives from both working groups reconciled these efforts to generate the now widely used Consolidated Standards of Reporting Trials (CONSORT) Statement.^63,64 The CONSORT Statement comprises a flow diagram and checklist of items to report that describe the flow of patients through the study, primary and secondary end points, prespecified hypotheses, key aspects of the study design and methods, the prespecified statistical analysis plan, and results. The CONSORT Statement has subsequently undergone revision^65–69 and elaboration.⁷⁰ Efforts to develop similar reporting guidelines for several other types of health research studies have followed the successful CONSORT model.^49,60

Complete and transparent reporting for prognostic and predictive marker studies requires attention to the relevant elements of the CONSORT guidelines as well as other aspects of these studies, including issues surrounding specimens and marker assays. Two reporting guidelines highly relevant to prognostic and predictive tumor marker studies are the Biospecimen Reporting for Improved Study Quality (BRISQ)^71–73 guidelines and the Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK) guidelines.⁷⁴

BRISQ

Unrecognized preanalytic issues can fundamentally alter the results obtained for a tumor marker. Thus, translating study results into reliable clinical care can be challenging, and comparative effectiveness research demands that preanalytic issues be considered and addressed appropriately. These factors can include the organ or tissue site from which a specimen was obtained (eg, tissue, blood, secretions), the type of specimen (eg, core biopsy, fine-needle biopsy, excision), and whether the specimen was stored frozen or fixed, and if so, the type of fixative, time and duration of fixation, how it was stored, duration of storage, and whether it was manipulated (eg, freezing and thawing). These elements are critical to the analytic validity of any tumor marker test, and one cannot assume that an assay developed under one set of preanalytic conditions will pertain to or remain valid in another. Thus, it is essential that these issues are considered and reported so that other investigators can reproduce exciting new exploratory data (the hallmark of scientific endeavor), and more important, so that the results can be translated into clinical utility with a high degree of confidence.

Recently, a group of experts has developed the BRISQ criteria—a check list that authors can submit either as part of their original manuscript or as supplementary material to be made available online.^71–73 BRISQ consists of three tiers of elements. Tier 1 items are necessary to report and include essential issues such as the organ(s) and/or tissues from which the biospecimens were derived, the manner in which the biospecimens were stabilized and preserved, the disease status (or for controls, the lack of it), and other critical preanalytic features that may fundamentally affect technical conduct of the assay in question (Table 2). Tier 2 items are advisable to report and include data elements that are slightly less crucial or less likely to be available in the biospecimens' annotation (eg, the demographics of the patient population and the method of enrichment for relevant components). Tier 3 includes additional factors that are not as likely to influence research results or are unlikely to be available (eg, environmental factors to which patients were exposed or the type of storage container in which the biospecimens were kept). Adhering to the BRISQ guidelines when reporting marker studies will allow for better assessment of any biases inherent in the types of specimens used in the study and will provide a more realistic sense of the feasibility of collecting the specimens required to successfully perform the marker assay.

Table 2.

BRISQ Tier 1 Reporting Items

Data Elements	Examples
□ Biospecimen type	Serum, urine
Solid tissue, whole blood, or another product derived from a human being
□ Anatomical site	Liver, antecubital area of the arm
Organ of origin or site of blood draw
□ Disease status of patients	Diabetic, healthy control
Controls or individuals with the disease of interest
□ Clinical characteristics of patients	Premenopausal patients with breast cancer
Available medical information known or believed to be pertinent to the condition of the biospecimens
□ Vital state of patients	Postmortem
Alive or deceased patient when biospecimens were obtained
□ Clinical diagnosis of patients	Breast cancer
Patient clinical diagnoses (determined by medical history, physical examination, and analyses of the biospecimen) pertinent to the study
□ Pathology diagnosis	HER2-negative intraductal carcinoma
Patient pathology diagnoses (determined by macroscopic and/or microscopic evaluation of the biospecimen at the time of diagnosis and/or prior to research use) pertinent to the study
□ Collection mechanism	Fine-needle aspiration, preoperative blood draw
How the biospecimens were obtained
□ Type of stabilization	Heparin, on ice
The initial process by which biospecimens were stabilized during collection
□ Type of long-term preservation	Formalin fixation, freezing
The process by which the biospecimens were sustained after collection
□ Constitution of preservative	10% neutral-buffered formalin, 10 USP heparin U/mL
The makeup of any formulation used to maintain the biospecimens in a nonreactive state
□ Storage temperature	−80°C, 20 to 25°C
The temperature or range thereof at which the biospecimens were kept until distribution or analysis
□ Storage duration	8 days, 5 to 7 years
The time or range thereof between biospecimen acquisition and distribution or analysis
□ Shipping temperature	−170°C to −190°C
The temperature or range thereof at which biospecimens were kept during shipment or relocation
□ Composition assessment and selection	Minimum 80% tumor nuclei and maximum 50% necrosis
Parameters used to choose biospecimens for the study

NOTE. This checklist is reproduced with permission.^71–73

Abbreviations: BRISQ, Biospecimen Reporting for Improved Study Quality; HER2, human epidermal growth factor receptor 2; USP, United States Pharmacopeia.

REMARK

In 2005, a set of guidelines was proposed by another international committee of experts in the tumor marker field. At this writing, these REMARK guidelines have been published and endorsed by seven highly respected journals.^74–80 Indeed, the Journal of Clinical Oncology has written instructions for potential authors that “submissions [to the Journal] need to be REMARK compliant … JCO will assign higher priority to those biomarker papers that, in addition to satisfying REMARK criteria, have validated the prognostic or predictive value of the biomarker in an independent data set that was not used for initial biomarker identification and characterization. Without such independent biomarker validation, the reproducibility of the findings may be difficult to assess. This is especially important for those papers that suggest clinical utility for a given biomarker. In select circumstances, discovery of a novel biomarker not previously reported, especially if derived from a prospective analysis, might not always require independent validation if the research is hypothesis generating and the marker has convincing biologic relevance.”⁸¹

The REMARK guidelines are modeled after the CONSORT criteria but were developed to be specific to tumor marker studies, with the goal of increasing the transparency and completeness of information provided by those studies. They list 20 reporting elements under four main headings: Introduction (one), Materials and Methods (10), Results (seven), and Discussion (two)— that include relevant information concerning study design, prespecified hypotheses, patient and specimen characteristics, assay methods, statistical analysis methods, and recommendations for specific types of results that are useful to report (Table 3). Taken together, the REMARK criteria are designed to permit an accurate understanding of the analytic validity, clinical/biologic validities, and clinical utility, if any, of a tumor marker.

Table 3.

REMARK Checklist

Introduction

1. State the marker examined, the study objectives, and any pre-specified hypotheses.

Materials and Methods

Patients

2. Describe the characteristics (for example, disease stage or co-morbidities) of the study patients, including their source and inclusion and exclusion criteria.

3. Describe treatments received and how chosen (for example, randomized or rule-based).

Specimen characteristics

4. Describe type of biological material used (including control samples) and methods of preservation and storage.

Assay methods

5. Specify the assay method used and provide (or reference) a detailed protocol, including specific reagents or kits used, quality control procedures, reproducibility assessments, quantitation methods, and scoring and reporting protocols. Specify whether and how assays were performed blinded to the study endpoint.

Study design

6. State the method of case selection, including whether prospective or retrospective and whether stratification or matching (for example, by stage of disease or age) was used. Specify the time period from which cases were taken, the end of the follow-up period, and the median follow-up time.

7. Precisely define all clinical endpoints examined.

8. List all candidate variables initially examined or considered for inclusion in models.

9. Give rationale for sample size; if the study was designed to detect a specified effect size, give the target power and effect size.

Statistical analysis methods

10. Specify all statistical methods, including details of any variable selection procedures and other model-building issues, how model assumptions were verified, and how missing data were handled.

11. Clarify how marker values were handled in the analyses; if relevant, describe methods used for cutpoint determination.

Results

Data

12. Describe the flow of patients through the study, including the number of patients included in each stage of the analysis (a diagram may be helpful) and reasons for dropout. Specifically, both overall and for each subgroup extensively examined report the number of patients and the number of events.

13. Report distributions of basic demographic characteristics (at least age and sex), standard (disease-specific) prognostic variables, and tumor marker, including numbers of missing values.

Analysis and presentation

14. Show the relation of the marker to standard prognostic variables.

15. Present univariable analyses showing the relation between the marker and outcome, with the estimated effect (eg, hazard ratio and survival probability). Preferably provide similar analyses for all other variables being analyzed. For the effect of a tumor marker on a time-to-event outcome, a Kaplan-Meier plot is recommended.

16. For key multivariable analyses, report estimated effects (eg, hazard ratio) with confidence intervals for the marker and, at least for the final model, all other variables in the model.

17. Among reported results, provide estimated effects with confidence intervals from an analysis in which the marker and standard prognostic variables are included, regardless of their statistical significance.

18. If done, report results of further investigations, such as checking assumptions, sensitivity analyses, and internal validation.

Discussion

19. Interpret the results in the context of the pre-specified hypotheses and other relevant studies; include a discussion of limitations of the study.

20. Discuss implications for future research and clinical value.

NOTE. Reproduced with minor stylistic edits.^74–80 Downloadable version.⁸²

Abbreviation: REMARK, Reporting Recommendations for Tumor Marker Prognostic Studies.

Regarding analytic issues, the REMARK guidelines call for description of the types of specimens used and preanalytic variables affecting those specimens (which are covered in more detail in BRISQ) and precise descriptions of the laboratory methods used to perform the assays of interest. Information about the required specimen collection and processing procedures for reliable performance of the assay and difficulty and robustness (sensitivity, specificity) of the assay method is critical for determining the feasibly of using the marker in real-world clinical settings. To put the study in its proper clinical context, REMARK guidelines request patient clinical characteristics, pathologic diagnoses, and the treatments received (necessary to distinguish prognostic and predictive markers), and the clinical setting in which they are delivered.

The REMARK criteria and recommended study profile^47,48,52 stress the importance of thorough and transparent reporting of statistical analysis methods to allow for assessment of appropriateness of approaches used and to facilitate efforts to reproduce the results. This emphasis on reporting statistical analysis methods is also motivated in part by the observational nature of many tumor marker studies and their frequent inclusions of multiple exploratory analyses that increase the chances of generating spurious findings. Therefore, REMARK criteria specify that reports should explicitly identify all the markers and end points that were initially examined in the study, state any prespecified hypotheses, and clearly describe all analysis methods used to produce the results.

REMARK criteria also emphasize the need for comprehensive reporting of analysis results, including reporting of estimated effects accompanied by measures of their uncertainty, relationships between the markers and standard clinical and pathologic variables, and estimated marker effects after adjustment for these standard variables. These factors permit assessment of the contribution of prognostic and predictive markers above and beyond information already available for use in clinical decision making, an important goal of comparative effectiveness research.

Although the original scope of the REMARK recommendations was primarily the reporting of studies that evaluated the prognostic value of a single marker, the guidelines are relevant to predictive studies, to studies investigating more than one marker (eg, multivariable classification functions or indices), and to studies evaluating prognostic or predictive factors other than those that are marker-based. Not only is the REMARK checklist⁸² useful for evaluating an individual tumor marker study report, it can be a helpful tool for collecting information from multiple studies for the purpose of a comparative effectiveness analysis to ultimately determine clinical utility.

The REMARK working group has recently published an elaboration of the REMARK guidelines,^47,48 and readers are referred to that publication for a more comprehensive discussion of the REMARK reporting elements and specific examples of good reporting. The REMARK criteria elaborations more thoroughly describe the scope and level of detail recommended for each of the REMARK reporting items, explain the rationale behind the need to report each item, and encourage use of a study profile as a format for presenting the key information that fulfills the checklist reporting elements.

SUMMARY

Taken together, the use of a tumor marker study registry, and adherence to BRISQ and REMARK guidelines will result in more complete and transparent reporting of tumor marker studies, thus making it easier to assess study quality, inherent biases, and relevance to a given clinical setting. Table 4⁶⁰ summarizes the multiple challenges associated with conducting comparative effectiveness research for prognostic and predictive markers in oncology. Many are related to deficient reporting of tumor marker studies, and we believe that these challenges can be minimized with the described strategies. Institution of these measures is feasible but will require concerted effort and support from members of the research community, study funders, regulatory agencies, payers, and patients. Support will arise from recognition of the potential for prognostic and predictive markers to improve patient care and outcomes as well as from an understanding of the need for more informed overviews on which evidence-based conclusions about the real-world utility of prognostic and predictive markers can be based. We strongly urge editors of major journals that publish oncologic research to mandate that these criteria be fulfilled and transparently included in all manuscripts at the time of submission.

Table 4.

Strategies to Facilitate Comparative Effectiveness Studies of Prognostic and Predictive Markers in Oncology

Challenge	Strategy
Lack of resources to conduct prospective clinical trial to specifically address clinical utility of prognostic and predictive markers	Collect high quality, well-annotated specimens from randomized treatment trials and well-designed prospective cohort studies
Nonpublication bias	Register protocol for marker study at study inception as requirement for journal publication
Uncertain specimen quality and preanalytical conditions	Adhere to BRISQ guidelines
Incomplete and selective within-publication reporting	Adhere to CONSORT, REMARK, and other applicable health research reporting guidelines60
Lack of requirement for evidence-based consensus for clinical utility for many markers in clinical use	Align regulatory requirements, third-party payer decision making, and publication requirements

Abbreviations: BRISQ, Biospecimen Reporting for Improved Study Quality; CONSORT, Consolidated Standards of Reporting Trials; REMARK, Reporting Recommendations for Tumor Marker Prognostic Studies.

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

Although all authors completed the disclosure declaration, the following author(s) indicated a financial or other interest that is relevant to the subject matter under consideration in this article. Certain relationships marked with a “U” are those for which no compensation was received; those relationships marked with a “C” were compensated. For a detailed description of the disclosure categories, or for more information about ASCO's conflict of interest policy, please refer to the Author Disclosure Declaration and the Disclosures of Potential Conflicts of Interest section in Information for Contributors.

Employment or Leadership Position: None Consultant or Advisory Role: None Stock Ownership: Daniel F. Hayes, Oncimmune Honoraria: None Research Funding: Daniel F. Hayes, Pfizer, Novartis, Veridex, Janssen Pharmaceutica Expert Testimony: None Other Remuneration: None

AUTHOR CONTRIBUTIONS

Conception and design: All authors

Collection and assembly of data: All authors

Data analysis and interpretation: All authors

Manuscript writing: All authors

Final approval of manuscript: All authors