Translational Research Working Group Developmental Pathway for Biospecimen-Based Assessment Modalities

Abstract

The Translational Research Working Group (TRWG) was created as a national initiative to evaluate the current status of National Cancer Institute's investment in translational research and envision its future. The TRWG conceptualized translational research as a set of six developmental processes or pathways focused on various clinical goals. One of those pathways describes the development of biospecimen-based assays that use biomarkers for the detection, diagnosis, and prognosis of cancer and the assessment of response to cancer treatment. The biospecimen-based assessment modality pathway was conceived not as comprehensive description of the corresponding real-world processes but rather as a tool designed to facilitate movement of a candidate assay through the translational process to the point where it can be handed off for definitive clinical testing. This paper introduces the pathway in the context of prior work and discusses key challenges associated with the biomarker development process in light of the pathway.

Molecular biomarkers are at the heart of our aspirations for a new era of cancer prevention and treatment. Novel biomarkers offer the potential for improved management of the disease at every point from screening and detection, through diagnosis, staging, and prognosis, to assessment of treatment response.

Large-scale assays and bibliometric searches have identified hundreds of candidate biomarkers for various cancers. To date, however, the successful translation of a candidate biomarker from discovery to routine clinical application remains relatively rare (1). Even as the research community wrestles with the methodologic challenges of biomarker development, conditions for bringing biomarker-based tests to market are becoming more stringent, with both regulators and payors moving to apply more rigorous standards for analytic and clinical validation. To assure that scarce resources are invested wisely, there is an urgent need to develop and consistently apply more systematic and effective approaches to the development of cancer biomarkers.

The combination of clinical need, scientific promise, and methodologic challenge made biomarker development a focus of the Translational Research Working Group (TRWG). As with other key areas of cancer translational research, the TRWG sketched out a flowchart of steps in biomarker translational research to facilitate identification of challenges and bottlenecks and stimulate and focus discussion about how best to address them in any given developmental project. An introduction and overview of the TRWG Developmental Pathways to Clinical Goals is found in Hawk et al. (2). This article is intended to explain the purpose of the biospecimen-based assessment modality (BM) pathway depicted in Fig. 1 in the context of prior efforts to systematize the approach to biomarker development, and to highlight key aspects of the process that warrant special attention.

Fig. 1.

Biospecimen-Based Assessment Modality (BM) Pathway. The BM pathway is depicted as a flowchart, a schematic process representation widely used in engineering. Rounded rectangle at the top, the origin of the process. Square-cornered rectangles, activity steps. Diamonds, conditional tests or decision steps. Unidirectional arrows, the direction of the activity sequence, and the direction of transfer of supporting tools from their parallel development paths to the main path of modality development. The three diamonds in the initial steps of the pathway (blue) are decisions required to proceed through the pathway and represent the credentialing step. Subsequent steps include the development of supporting tools (red), the creation of the modality (green), preclinical development (purple), and early stage clinical trials (yellow). For each activity or decision point, it is understood that there are many more variations that can occur, and that not all steps may occur in each instance. The pathway does not address the ways in which insights gained from late-stage clinical trials can influence the development process. Biospecimen-based assessment devices can be used for screening, early detection, diagnosis, prediction, prognosis, or response assessment. The pathways are conceived not as comprehensive descriptions of the corresponding real-world processes but as tools designed to serve specific purposes, including research program and project management, coordination of research efforts, and professional and lay education and communication.

Insights from Previous Work

In drafting the developmental pathway, the TRWG followed the pioneering work of Pepe et al. (3), who addressed the phases of development of biomarker-based screening tools for early detection of cancer in a seminal article. Pepe et al. defined these phases as follows, by analogy with the process by which new drugs are developed:

• Phase 1, “Preclinical Exploratory,” in which promising directions are identified;

• Phase 2, “Clinical Assay and Validation,” in which the ability of the clinical assay to detect established disease is shown;

• Phase 3, “Retrospective Longitudinal,” in which the ability of a biomarker to detect disease before it becomes clinically evident is shown, and rule for judging a result as “positive” is defined;

• Phase 4, “Prospective Screening,” in which the extent and characteristics of disease detected by the test and the false referral rate are identified; and

• Phase 5, “Cancer Control,” in which the effect of screening on reducing the burden of disease on the population is identified.

In describing these phases, Pepe et al. (3) focus especially on the question of what kinds of evidence are needed to establish the clinical validity and utility of a new biomarker.

The TRWG also gained important insights from a key element of the current biomarker research portfolio of the National Cancer Institute, the Early Detection Research Network (EDRN).⁷ In the context of the work of TRWG, EDRN is noteworthy for taking a systems view of the translational research process, defining key functional elements and implementing them in an explicitly structured and choreographed way that reflects and addresses the evidentiary framework specified by Pepe et al. The EDRN approach recognizes that the culture and working methods of the fundamental science laboratories from which biomarker candidates typically emerge are not well-matched to the requirements of product development and of analytic and clinical validation, and offers an alternative path better suited to the task.

Building on these efforts, the TRWG drafted a Developmental Pathway for Biospecimen-Based Risk Assessment Devices that lays out the biomarker translational research process from a systems perspective, describing it in terms of key activities and decision points along the path from concept through assay development to clinical testing. Compared with the frameworks created by the earlier efforts, the TRWG developmental pathway differs in important respects. Its focus on the phases of development is narrower, excluding fundamental discovery research to concentrate on the process by which emerging concepts are translated into a tangible form ready for definitive clinical testing. Thus, the TRWG biospecimen/biomarker developmental pathway overlaps with phases 2 through 4 of the schema of Pepe et al. However, although Pepe et al. use the phases as a framework for clarifying the kinds of evidence needed to establish the clinical validity and utility of a new biomarker, the TRWG developmental pathway parses the development process from the perspective of a scientistmanager, applying a programmatic and operational perspective to the systematic assessment of translational research activity with the objective of enhancing the efficiency and effectiveness of that activity. In addition, compared with both the analysis of Pepe et al. and the programmatic focus of the EDRN, the TRWG developmental pathway encompasses a broader range of biomarker applications, extending beyond screening and early detection to encompass uses in the therapeutic setting as well.

KEY POINTS.

• The Biospecimen Modalities Pathway heightens awareness of the elements of the development path and provides a framework for understanding key scientific and regulatory challenges in bringing new biomarker-based assessment modalities to the clinic.

• The Biospecimen Modalities Pathway highlights the central role of validation throughout the development of biomarker-based assessment modalities.

• The Biospecimen Modalities Pathway highlights the need for biospecimen repositories and other supporting tools.

• The Biospecimen Modalities Pathway can lead to improved communication and effective choreography of the relationships among academia, government, and industry.

Validation: The Central Challenge

The BM pathway is distinctive in that the greatest challenges associated with translation revolve not around creation of the modality (that is, development of the practical laboratory procedures or kits needed to implement tests based on the marker) but rather around its validation. The TRWG used the term validation broadly, to cover all of the many different activities designed to verify that the characteristics of the modality are as expected or desired. With respect to biomarker-based assessment modalities, this includes especially analytic validity—“a test's ability to measure the analyte or genotype of interest accurately and reliably” and clinical validity—“a test's ability to detect or predict the associated disorder (phenotype).”⁸

Of these two key dimensions of performance, assuring clinical validity poses the greatest conceptual and methodologic challenges throughout the developmental pathway, from credentialing of the initial discovery through clinical trials. Success requires rigorous adherence to careful study design and valid statistical methodology to avoid the trap of spurious correlation.

Important methodologic considerations that have often not been addressed in the past in the development of biomarker-based diagnostics include attention to data accuracy, reproducibility, or standardization beyond the laboratory in which the markers were discovered; blinding of the laboratory researchers who perform the assays with respect to the status of the samples (whether it is a case or a control, whether they are replicates from the same specimen); and randomization of samples and their replicates to the assay allocations (e.g., spots on the chip, chips, assay dates, etc). Evaluation of cancer therapeutic modalities is usually conducted via multicenter clinical trials operating under clearly specified and strictly enforced investigational protocols; the same discipline needs to be applied to biomarker validation.

The biostatistical challenges of validating associations and assessing their predictive value in real-world populations are, if anything, more subtle and difficult than those faced in typical randomized trials of therapeutics (4, 5). In assessing the robustness of the correlation with the clinical phenomena of interest that define the potential value of a biomarker, it is essential that successive rounds of testing be done using truly independent sample sets, and that the specificity, sensitivity, and predictive value of the assay be quantified in study populations where the prevalence of the marker reflects what is likely to be observed in clinical practice. Development of profile-based tests introduces a new layer of methodologic traps for the unwary. Seemingly small errors in the specification of statistical models, failure to replicate results using truly independent sample sets, or any bias introduced by failure to incorporate careful blinding and randomization will have an even larger effect.

The BM Pathway Domains

During the credentialing phase of the BM pathway, the questions of clinical validation, clinical need, and feasibility are addressed. Is available exploratory data sufficiently convincing to justify the expenditure of resources in a focused effort to develop a practical assay? In sifting through the vast amounts of available information to evaluate and prioritize biomarker candidates for translation, the key requirement in addition to valid statistical methodology is care in identifying clinical scenarios in which the availability of a robust biomarker is likely to provide meaningful clinical benefit by enabling strategies for prevention or treatment that are measurably more effective.

A key hurdle that biomarker developers face is access to a sufficient quantity of properly preserved, clinically relevant, well-annotated biospecimens, influencing the supporting tools, creation of modality, and preclinical development domains of the BM pathway. As an example, development of the Oncotype DX breast cancer assay, one of the first of a new generation of genomics-based tests to reach the market, relied in large part on biospecimens from National Surgical Adjuvant Breast and Bowel Project studies (6, 7). In the absence of an established network of biospecimen repositories addressing a range of tumor types, successful translation of candidate biomarkers is subject to the chance availability of the required samples.

The development of tests based on profiles of markers underscores the importance of systematically cataloged knowledge on a broad range of markers, even those that do not seem to show a robust association with clinical phenomena of interest when assessed individually.

Researchers are pursuing a wide range of genomic, proteomic and metabolomic species, and analytic methods for use as biomarker-based assessment modalities; each method poses its own distinct challenges and potential pitfalls in the areas of implementation requirements, sensitivity, specificity, reproducibility, and interpretation. Particularly where the analytic approaches to be applied are novel, special attention is required to standardization of methods and reproducibility of results, at all stages from creation of the modality and proof of concept in the laboratory through implementation of products or protocols intended for definitive clinical trials and implementation in routine clinical settings.

Regulatory Considerations

To be successful, the development process must be organized to cope effectively with the regulatory system under which diagnostic products are brought to market in the United States.

Two parallel regulatory regimens are involved in the regulation of in vitro diagnostics—clinical laboratories are regulated by the Centers for Medicare and Medicaid Services under the Clinical Laboratory Improvement Amendments (CLIA) of 1988,⁹ whereas medical devices, including in vitro diagnostic products, are regulated by the Food and Drug Administration (FDA).¹⁰ For a number of years a status quo prevailed, under which “home brew” tests assembled by individual laboratories from general-purpose “analyte-specific reagents” were regulated for analytic validity and proficiency in laboratory implementation under CLIA, whereas “kits,” or complete, packaged tests marketed to laboratories by a manufacturer, were regulated by FDA, with a somewhat more stringent requirement for evidence of clinical validity and utility, as well as requirements for quality control in manufacture.

As a general matter, laboratory procedures used to generate data in nonclinical studies that will be used to support a product, submission to FDA must meet standards for Good Laboratory Practices;¹¹ manufacturing processes for test components must meet standards of composition, stability, and consistency, as specified by Good Manufacturing Practices;¹² and well-defined, standardized protocols must be created for use of the assay in clinical laboratories and for quality assurance and verification of proficiency in such routine use.

With rapid innovation in genomic technologies leading to the emergence of new assays based on the association of genomic biomarkers with clinical conditions, regulators have begun to focus greater attention on the adequacy of existing approaches to assure the safety and efficacy of these new products. As a result, the regulatory system for in vitro diagnostic tests is in transition.

A key milestone in this transition has been the recent release of a draft guidance on multivariate index assays by the FDA, presenting its views of the technical issues involved and explaining its proposed approach to regulating these new tests.¹³ This draft guidance has been the focus of some controversy, and certain elements of the proposed approach by the FDA have not been finalized as of this writing. However, the strategy adopted by some developers—to implement new genomics-based diagnostic tests as centralized laboratory services to bring then to market under CLIA rather than more stringent FDA regulation—is likely to be restricted or eliminated.

Changes in the regulatory regimen for in vitro diagnostics will reinforce the scientific and clinical imperative to define and adhere consistently to more robust standards for both analytic and clinical validation. The developmental pathway reflects this more rigorous approach, conceptualizing translational research on biomarkers as extending through validation in prospective clinical studies.

Implications of Trends in Insurance Coverage for New Clinical Product and Services

With respect to insurance coverage of new clinical products and services, there are two distinctive aspects of diagnostic or screening tests that increase concerns among payors about both efficacy and appropriate use. First, medical officers at the major health insurers are aware of the methodologic challenges of developing robust and valid biomarkers, and the risk that apparent correlations will prove illusory on more rigorous analysis. A fairly high threshold of skepticism is usually applied to claimed advances in this field because of concern that faulty tests will reach the market, consuming resources unproductively or even placing patients at risk of inappropriate care and adverse outcomes. Payors also have strong concerns that the availability of a new, expensive assay will lead to a wave of costly, inappropriate usage because of a widespread perception that diagnostic tests, especially those based on blood samples or on noninvasive imaging, impose relatively little risk for a patient compared with therapeutic interventions.

The consequence of these concerns is that payors demand more extensive data on diagnostic tests than is required to gain FDA approval, to validate their clinical benefits in real-world practice.¹⁴ The implication for biomarker developers is that rigorous attention to clinical value is required throughout the development process. Reliable detection of a biomarker— analytic validity—is not by itself sufficient to gain market acceptance for a new product.

Coupling Biomarkers with Treatments

Interactions between this developmental pathway and the developmental pathway for new targeted therapeutic agents must be considered as well. The role of diagnostic tests for HER2 overexpression in defining the population of metastatic breast cancer patients for whom trastuzumab is an effective treatment is a model for targeted agents of the future. However, development of such diagnostic/therapeutic pairs will likely be more effective—and more cost-effective—when the parallel development paths are coordinated from earlier in the development process than was the case with HER2 and trastuzumab.

Such coordination can be quite complex, with logistical challenges further exacerbated by the fact that, in most cases, the diagnostic and the therapeutic are being developed by different companies or organizations. The FDA has issued a draft concept paper on drug-diagnostic codevelopment that addresses aspects of the codevelopment process, with the objective of facilitating a shared understanding with academia and industry of approaches that are likely to produce results sufficiently robust to support regulatory decision making.¹⁵ The TRWG developmental pathways can facilitate coordination by specifying developmental steps and clarifying dependencies between the developmental steps for therapeutics and for their associated biomarkers.

Example of the Use of the BM Pathway

Although the developmental pathway was created by the TRWG and, thus, has not been used to guide previous development efforts, it is instructive to review prior efforts in light of the pathway.

The development of a fluorescent in situ hybridization assay (FISH) in urine samples for the detection of bladder cancer followed the biomarkers pathway closely. The technique was originally developed at Lawrence Livermore National Laboratory (8), and academic researchers did the fundamental research that indicated the FISH technology could be applied to early detection of bladder cancer (9). Vysis, Inc. provided the supporting tool of a reproducible assay by further developing the FISH technology for use in clinical tests and showed the FDA that assays based on FISH were sufficiently robust and reproducible to be used for clinical purposes. Vysis credentialed the use of FISH for early detection of bladder cancer as a commercial target based on the combination of clinical need (existing tests had limited sensitivity/specificity) and the assessment that the FISH technology of Vysis could improve upon those tests (10). The company collaborated with the University of Basel and Mayo Clinic to obtain the required supporting tools of samples and clinical data sets. They developed the modality by validating the technology using these data sets and proceeded through the preclinical development and clinical trial steps by pursuing large-scale prospective studies. FDA approval for the UroVysion test was granted in 2005.¹⁶

Looking to the Future

The value of the developmental pathway as a tool for project and program planning, for training, and for heightening general awareness of the optimal approach to biomarker development can be enhanced through further development of the pathway to reflect the activities, decision points, and interactions associated with the regulatory process and with codevelopment of drugs and therapeutics. Continued investment in strong analytic technology, informatics, statistics, epidemiology, and in biosample management will pay dividends through high quality data that will meet regulatory requirements.

Review of the developmental pathway reminds us once more of the importance of the efforts of National Cancer Institute to develop biospecimen repositories as well as management approaches for prioritizing and facilitating access to these essential resources.

Finally, academic culture emphasizes individual achievement over collaborative work. However, realizing the full potential of the nation's investment in cancer research requires collaboration that crosses disciplinary boundaries and integrates complementary activities in government, academia, and industry to achieve priority objectives. We must find ways to incentivize the kind of creativity and intellectual leadership that not only creates new concepts but also advances them to fruition.

Conclusion

Despite the central role of biomarkers in current thinking about cancer screening, diagnosis, and therapeutics, progress in bringing biomarker-based assessment modalities to the clinic has been disappointing. The substantial challenges posed by biomarker development can be met only through rigorous adherence to high methodologic standards and close attention to the requirements of regulators and payors. The BM Pathway clarifies the elements of the development process, provides a framework for understanding key scientific and regulatory challenges in the development process, and facilitates coordination of the diverse, crossdisciplinary efforts required to meet those challenges.