Workshop Report and Follow-Up—AAPS Workshop on Current Topics in GLP Bioanalysis: Assay Reproducibility for Incurred Samples—Implications of Crystal City Recommendations

Abstract

The Conference Report of the 3rd AAPS/FDA Bioanalytical Workshop (Crystal City III) endorsed the concept that assay methods supporting bioanalytical data in submissions must demonstrate assay reproducibility by using incurred samples. The present Workshop was convened to provide a forum for discussion and consensus building about incurred sample assay reproducibility for both nonclinical and clinical studies. Information about current regulatory perspectives on incurred sample reanalysis (ISR) was presented, implications of ISR for both large and small molecules were discussed, and the steering committee put forth recommendations for performing ISR. These recommendations from the Workshop, along with the subsequent evolution of approaches leading to a robust ISR program, may be used by scientists performing bioanalytical assays for regulated studies to provide additional confirmation of assay reproducibility for incurred samples.

INTRODUCTION

The Conference Report of the third AAPS/FDA Bioanalytical Workshop (1) held in May, 2006, called for incurred sample reanalysis (ISR) to be conducted for both nonclinical and clinical studies. While providing a general framework and a rationale for ISR, the Conference Report did not provide detail about how to conduct ISR or what would be considered acceptable ISR performance. Before and since the publication of the Conference Report, several methods for conducting ISR for bioanalytical assays have been proposed (2–7), but until the time of this Workshop, there had not been a consensus on recommendations for implementation. This report summarizes recommendations made at the Workshop on Incurred Sample Reanalysis held in February, 2008. Specific recommendations presented on ISR at that meeting are discussed, and additional recommendations for implementing a valid ISR program are provided. Issues not covered by this paper should be addressed by the scientist in the bioanalytical laboratory.

GOALS AND OBJECTIVES

The Workshop was planned to present a forum for discussing what laboratories have done to implement the recommendations of the Crystal City III Conference Report on assay reproducibility assessment for both small and large molecules (1). Speakers gave an overview of various practices, and breakout sessions allowed the sharing and compiling of various techniques that have been used for both large and small molecules.

CLARIFICATIONS AND RECOMMENDATIONS

At the conclusion of the workshop the following items were clarified or recommended.

Basis for ISR

A carefully designed ISR program provides additional data to improve confidence in the reliability and reproducibility of a validated method for nonclinical and clinical study samples. In addition, a well-constructed ISR program should lead to continuous review and improvement practices for the laboratory that is conducting ISR experiments.

Central to the concept of ISR is the understanding that failed assessments must lead to a careful and thorough investigation of the assay. Review of the circumstances around the assay should be documented along with conclusions about remediation and recommendations about the validity of the original results. A failed ISR assessment does not immediately invalidate the entire study, but it does call for suspension of the bioanalytical portion of the study until an investigation is completed, documented, and appropriate follow-up actions are in place.

General Principles

If a pharmacokinetic (PK) determination is an outcome of a study, then an ISR assessment should be considered in the bioanalytical portion of the study. Generally, ISR will be conducted for plasma and serum samples. Because the assessment of exposure is critical to safety considerations, bioanalysts should provide as much confidence as possible in the reproducibility of the assays employed to generate drug concentrations used for the PK calculations.

Results from or reference to ISR assessments should be included in the report of the study from which the samples were taken. This will assist agencies that review the marketing submission to determine that ISR was conducted and the outcome of the experiment. ISR may be conducted as a component of the validation, particularly in nonclinical studies where animal populations are often quite homogenous, and diet and conditions are relatively constant among animals, but it is not required to be a part of the validation. If the ISR is conducted as part of the validation, the results of the validation experiments should be referenced in the bioanalytical report for each study using that validated method.

A SOP or study plan is critical to the proper conduct of ISR assessments. The SOP or plan should detail the method of conducting ISR, how differences between original and reanalyzed results are computed, what acceptance criteria will be used, how an investigation of a failed ISR assessment will be conducted, documented, reported, and archived, and where assessment results will be reported and archived.

ISR Assessment Timing and Scope

ISR assessments should be conducted for both clinical and nonclinical assays to inform the bioanalyst of possible reproducibility issues as early in the program as feasible. However, laboratory efficiencies and operations may dictate that the assessment be conducted at the end of small studies (e.g., short-term toxicokinetic studies).

For nonclinical studies, the ISR assessment should be conducted using samples from the first subchronic toxicology study for each compound. If performed on an earlier non-GLP study, the bioanalyst must assure relevance to the first regulated study. The assessment need only be conducted one time for each species, assay method, and laboratory because animals are considered to be more homogeneous in genetics, diet, and housing than humans.

For clinical studies, the assessment should be conducted for all bioequivalence studies. The bioanalyst should determine on a study-by-study basis whether to conduct ISR assessments for studies in healthy volunteers, in unusual patient populations, for testing small molecule drug–drug interactions, or for evaluating disease state changes in patient populations.

First-in-human oncology studies are an example of situations where a thoughtful design of the ISR assessment is necessary. Often, these studies have intermittent and slow enrollment of patients into the study. Given the multiple medications most patients receive and the changes to metabolism and endogenous compounds that oncology patients experience, the need to demonstrate ISR must be balanced with timing and sample stability issues to achieve a scientifically sound conclusion.

Selection of ISR Samples

ISR assessments should be conducted on aliquots of individual samples not on pools. While pooled samples may be used to assess stability in incurred samples (stability assessments were not addressed by the Workshop), individual samples provide the optimum probability of providing conditions that will best test the reproducibility of the assay. Selecting fewer samples from more subjects also improves the probability of finding anomalous samples or subjects. Sample selection regions of the plasma time-concentration profile should include the vicinity of t_max and near the end of the terminal or elimination phase of the plasma concentration versus time curve.

If the original result from the assay is determined by data from a single replicate of the unknown sample, then the ISR should be conducted using a single replicate. If original results are obtained by taking the mean of multiple replicates (typical for ligand binding assays), the same number of replicates should be used for the ISR determinations.

Sample size considerations are critical, and the number of samples repeated for ISR should be representative of the study conduct in its entirety and method performance overall. To eliminate confusion and facilitate a straightforward process for conducting ISR and, based on discussions since the Workshop, a sample scheme that uses a fixed percentage of the total sample size is recommended. While there are other proposed methods for calculating the number of ISR samples, the number of samples repeated should equal 5–10% of the total sample size, with 5% as the minimum for larger studies.

Acceptance Criteria for ISR

While an ISR assessment that passes acceptance criteria may bolster confidence in the overall validity of an assay, bioanalysts must clearly understand that this assessment alone is not a sufficient reason to accept or reject the results from any study for analytical reasons. ISR is just one aspect, but a significant part, of the assay performance. Many other factors enter into the decision that sample results are reportable, and good scientific judgment must be applied to the entire body of work in a specific study before accepting or rejecting results.

A failed ISR requires an investigation to determine if the assay is performing adequately for the purpose intended. The results of the investigation should be documented, and a conclusion should be made regarding the reliability of the assay before accepting or rejecting those results. No attempt was made to define the investigation process—it is the responsibility of the bioanalyst to evaluate the method using logical, scientific steps and tools necessary to support the acceptance or rejection of the study results. While an investigation is in progress, the bioanalytical portion of the study should not proceed unless it is conducted “at risk.” Storage stability already established should not deter or limit the pursuit of ISR resolution.

A fixed error limit method is recommended and a model similar to the familiar 4-6-X QC criteria can be applied. For small molecules (non-ligand binding) two thirds of the repeat samples should agree within 20%, and for ligand-binding assays, two thirds of the repeat samples should agree within 30%. The variability (% difference) should be calculated using the mean of the original and repeat results as described by the following formula:

TOPICS WITH NO CONSENSUS OR NOT ADDRESSED

Although the workshop conferees did not reach consensus on or address a number of issues, the following aspects should also be considered in implementing a robust ISR program. It is the responsibility of the bioanalyst to use sound scientific judgment and assure that the approaches used are appropriate to and adequate for the intended purpose. Documentation must be sufficient to reconstruct all aspects of the ISR assessment and any investigation of a failed ISR. Since ISR evaluations are specific to the study of interest, the bioanalyst must report or reference all pertinent information in the corresponding bioanalytical report.

• The use of random or nonrandom selection of samples for ISR assessment. The recommendation that samples be selected near the t_max and near the end of the terminal or elimination phase of the plasma concentration versus time curve indicates that nonrandom sampling is appropriate for those areas of the PK curve. Additional samples selected for ISR should be randomly chosen. Samples with concentrations below the limit of quantitation should not be selected.

• The use of diluted versus undiluted samples. It is expected that repeat analysis for ISR will be conducted in the same manner as the original sample analysis. Because original results are reported after correcting for dilution if undiluted study samples have analyte concentrations greater than the upper calibration standard, it is recognized that limiting ISR assessments to only undiluted samples could limit the number of samples available to be selected for ISR and therefore might not be representative of the entire data set. During the ISR assessment, the same dilution factor that produced the original result should be employed for the reanalysis.

• The complexity, scope, and duration of investigations initiated to address a failed ISR assessment. Good scientific practices should result in an investigation that is appropriate for purpose and documented at a level that will clearly describe the actions taken during the inquiry, a summary of the factual findings, and a conclusion about the applicability of the assay for the purpose of the study.

• Multi-analyte methods where it is likely that the t_max of the parent and another analyzed entity may be different. Reproducibility of all analytes must be demonstrated which may lead to selection of more samples per animal or subject for reanalysis than would be required for a single-analyte assay. The sample selection strategy should be such that the reproducibility of all analytes measured is demonstrated. The acquisition method used to obtain the reanalysis data should be the same as was used for the original result.

• Special matrix types (e.g., tissue or tissue fluids) or special populations (e.g., pediatric). The majority of studies requiring ISR assessments will utilize plasma or serum samples for PK assessments in nonclinical species and adult humans. However, it was also noted that the bioanalyst must consider the ultimate use of the data from any study and provide assurance to a reviewer that the reported results, including those for any metabolites assayed, are fit for the purpose intended. Therefore, the need for ISR assessments in studies of special populations or with special matrix types should be considered case-by-case based on the intended use of the data. Other approaches to demonstrating reproducibility in these unusual situations may be appropriate.

CONCLUSION

Strategies for conducting incurred sample reanalysis were addressed during the Workshop. Recommendations concerning the basis for ISR, general operational principles, assessment timing and scope, sample selection, and acceptance criteria were offered. Adherence to these recommendations should assist the bioanalytical scientist in establishing a robust ISR program. Specific implementation procedures are best left to the scientific and technical judgment and practices of the bioanalytical scientist. It is the responsibility of the bioanalytical scientist to justify and document all aspects of the ISR assessment. A well-constructed ISR program should lead to continuous review and improvement practices for the laboratory conducting ISR experiments and failed ISR assessments must lead to a careful and thorough investigation of the assay.