Method Transfer, Partial Validation, and Cross Validation: Recommendations for Best Practices and Harmonization from the Global Bioanalysis Consortium Harmonization Team

Abstract

This paper presents the recommendations of the Global Bioanalytical Consortium Harmonization Team on method transfer, partial validation, and cross validation. These aspects of bioanalytical method validation, while important, have received little detailed attention in recent years. The team has attempted to define, separate, and describe these related activities, and present practical guidance in how to apply these techniques.

The Global Bioanalytical Consortium (GBC) was established in 2011 as a forum for discussion of all aspects of bioanalytical method validation by the global bioanalytical community, and to allow consensus feedback on those discussions to be presented back to the bioanalytical community and to the regulators (1–3). This paper is a summary of the deliberations of the GBC Harmonization Team A3 on method transfer, partial validation, and cross validation. In order to allow the range of the discussion to be manageable, the scope was limited to work conducted in support of pharmacokinetic (PK) and bioequivalence studies, although it is hoped that the concepts laid out below will prove useful when applied to a wider scope.

The team recognized that validation is a continuous process and method transfer, partial validation, and cross validation forms part of a life cycle of continuous development and improvement of analytical methods.

Methods used to support PK and bioequivalence in drug development need to be developed in a variety of species. Very often, the methods applied have the same basic technologies and may be very similar to each other. There will, however, be divergence between methods as they are applied to different species, and methods within species may also alter over time as issues are identified, analytical ranges change, or populations alter. This may set up a complex interrelationship between methods which is compounded when these methods need to be transferred from one laboratory to another. It is therefore vitally important to manage the life cycle of any group of methods and to have the inter-relationships between the various methods to be well understood and clearly documented.

An attempt to simplify and illustrate the relationships between methods and how they are impacted by validation is given in Fig. 1 below.

Fig. 1.

Schematic of method relationships

Requirements for full validation are well defined in various guidelines (4–7). A variety of white papers also focus on this topic for both chromatographic and ligand binding methods (8–11). Partial validation and method transfer are as yet not so clearly described and this may, in part, be due to the perceived overlaps between method transfer and cross validation. Such overlaps may exist because the definitions and scopes for these activities are not well defined. This paper will attempt to define and clarify these relationships.

There has been discussion in the analytical community, most recently at the Crystal City meeting held in December 2013, about the relative merits of the use of freshly prepared matrix calibration standards. Although some debate continues regarding the definition of “fresh” (i.e., used without prior freezing vs. frozen for a short period prior to use), the team recognizes the need to use freshly prepared matrix calibration standards with appropriately stored quality control (QC) samples in validation batches used to assess the precision and accuracy of the method and in continuation of this to investigate stability. Once stability has been established, both matrix calibration standards and QCs can be prepared in bulk and stored under conditions deemed suitable for storage of study samples for subsequent studies.

METHOD TRANSFER FOR CHROMATOGRAPHIC ASSAYS AND LIGAND BINDING ASSAYS

Method transfer is defined by the team as a specific activity which allows the implementation of an existing analytical method in another laboratory. A method transfer may be conducted at any stage in the life cycle of a method and requires transfer from the initializing site (originator) to either another internal site(s) or to an external receiving laboratory (recipient).

The principle goal of method transfer is to demonstrate that the method is appropriately transferred and validated at the receiving laboratory. A number of aspects need to be considered.

First is the transfer an internal or external method transfer? Testing is required to demonstrate the equivalency of a transferred assay. The degree of testing may vary depending on the operating similarities between the originating laboratory and the recipient laboratory. For the purposes of this discussion, to be considered as an internal transfer, both laboratories must share common operating philosophies, infrastructure, and general management systems (e.g., standard operating procedures, operating process/practices, quality system, training, IT including Laboratory Information Management Systems) to provide consistent daily laboratory oversight. The concept of an internal transfer is generally considered a transfer between two laboratories within the same organization and does not apply to qualification of a new analyst within a validated laboratory.

Criteria for internal transfer must be rigorously applied to ensure that methods are not inappropriately or loosely transferred. In reality, it may be difficult to meet the requirements to be considered as an internal transfer and a transfer within an organization may be equivalent to an external method transfer if the laboratories do not truly share operating systems and philosophies.

Consideration should be given to the type of method being transferred. Ligand binding assays are generally more complex to transfer than chromatographic methods because of critical reagents, the relative importance of reagent lots and consumables lots. Recommended transfer criteria are provided later in this paper for both platform types.

To establish traceability, the working method at the originating laboratory should, if possible, be set up by the receiving laboratory as it is; in other words, without changes to the assay. Subsequent modification of the method may then be determined necessary prior to validation at the recipient laboratory but the changes should be recorded and traceable. Such changes should prompt consideration of partial validation as described later in this paper.

Evaluation of long-term stability in frozen matrix may be required if not adequately demonstrated by the initiating laboratory for the intended study conditions. However, this may require a lengthy storage period and, for the purposes of this discussion, is not considered part of the assay transfer process.

The team recommends the following activities for the various levels of transfer.

• For an internal transfer, the team recommends the following criteria: for chromatographic assays

  • A validation including a minimum of two sets of accuracy and precision data using freshly prepared calibration standards over a 2-day period to demonstrate the method is performing similarly.
  • Quality controls at the lower limit of quantification (LLOQ) should be assessed; however, upper limit of quantification (ULOQ) quality controls (QCs) are not required. No ancillary experiments such as dilution, stability, recovery, etc., are required. If environmental conditions (e.g., temperature, light, humidity) have been demonstrated to be an issue, appropriate evaluations of their impact should be conducted during method transfer.
• For ligand binding assays

  • If two internal labs share the same critical reagents, the validation should include a minimum of four sets of inter-assay accuracy and precision runs on four different days (to prove robustness). QC’s at the LLOQ and ULOQ must be included.
  • Dilution QC’s should be evaluated during accuracy and precision runs.
  • If two internal labs are not using the same critical reagents, a full validation will be required with the exception of the long term stability assessment.
  • Parallelism needs to be tested in incurred samples

  In both cases above, although full validation may not be absolutely necessary from a scientific perspective, it may be desirable for ease of traceability and audit review.

• For an external transfer the team recommends the following criteria:

  • Full validation including all accuracy, precision and bench top stability, freeze thaw stability, and where appropriate extract stability is required.
  • Long-term stability is not required if sufficient stability has been established in the same matrix and storage environment to cover the expected storage period of samples to be assayed at the receiving laboratory.

PARTIAL VALIDATION

The team has defined partial validation as the demonstration of assay reliability following a modification of an existing bioanalytical method that has previously been fully validated. The nature of the modification will determine the extent of validation required.

Partial validations are conducted to validate modifications of previously fully validated bioanalytical methods. Partial validations can range from an intra or inter-assay precision and accuracy experiment to nearly a full validation. Partial validations are performed to assess the validity of a change in method but not a change to the type of method used, (e.g., liquid chromatography–mass spectrometry (LCMS) to ligand binding) where a full validation would be required. In certain circumstances, a cross validation of two methods will be required and this is discussed below.

Parameters to be evaluated in a partial validation should be selected using a risk-based approach considering the potential impacts of any modifications in the method. The decision process of whether to conduct particular partial validation experiments will materially depend on the nature of changes to the method and their degree of significance. In the opinion of the team, the following changes should be considered significant. A change in matrix will normally be considered as a different method and will be validated as such. It may be that changes in the type of sample received, for example, different species is specific populations of subjects may impact the performance of a method (9,11). Issues highlighted by these experiments may lead to further investigation, and subsequent method modification requiring further partial or full validation. A change to the anti-coagulant counter-ion should not be considered a change in matrix and does not require a partial validation (12). In the case where an assay is validated in a specific gender, changing to the other gender, or to a pool, will not require further partial validation experiments unless the therapeutic agent in question is an endogenous molecule where sex-related difference might reasonably be expected. Including additional co-administered medicine seldom affects validated methods or stability and therefore will not require additional validation experiments with the possible exception of assay specificity testing especially for LCMS methods (13).

In the case of introduction of a rare matrix (e.g., CSF, lacrimal fluids, etc.), tissue samples, or feces, the partial validation can be limited to a practical extent given the difficulty in obtaining control materials. A set of “surrogate” matrix QCs might be applied and compared to real matrices.

A significant change to the mobile phase for chromatographic assays would be a change in the organic modifier (e.g., acetonitrile to methanol) or a major change in pH (e.g., ammonium acetate buffer to trifluoroacetic acid). Minor changes in the proportions of mobile phase constituents for the purpose of adjusting retention times in chromatographic methods would not normally prompt a partial validation but it is expected that performance evaluation will be very critically assessed by the analyst to determine if partial validation is required. For all other changes, a scientific judgment must be made to determine the extent of partial validation.

A significant change to the sample preparation procedure would be defined as a complete change in paradigm, such as from protein precipitation to liquid/liquid extract or solid phase extraction. For minor changes, (e.g., a change in elution volume or reconstitution volume), the analyst’s discretion will determine the extent, if any, of further partial validation.

Any modification that may lead to a change in the nature and level of the assay response should be evaluated. Such changes might result from a modification to the assay range, the internal standard, sample extraction or modification of the way the sample is introduced into the assay system. If partial validation is not indicated, a careful review of LLOQ and specificity in subsequent assay batches should be conducted to ensure that any issues are identified at an early stage, and investigated if necessary. It should be kept in mind that increases to ULOQ or injection volume should be accompanied by carry over assessments.

For ligand binding assays any change to critical reagents would prompt at least a partial validation. The analyst should evaluate the types of modification that would necessitate inclusion of a particular evaluation into the partial validation protocol. Precision and accuracy will be required if there is any significant change to the method as noted above. Specificity and matrix effects, including cross reactivity, dilution linearity, and hook effects will need to be performed where a change to the method chemistry has been made. This would include changes in critical reagents for ligand binding method or changes to mobile phase, internal standard, or column chemistry for chromatographic methods.

The primary consideration that should be made is whether the modification of the method is likely to have an impact on the level of the absolute response of the analyte or analytes of interest. The nature of the validation experiments selected should be based on the likely nature of the change to the response. As this is often difficult to assess, caution often dictates that partial validation will include almost all of the principle validation experiments.

Recovery assessment will only be required if there is a significant change to sample preparation procedures or if there was a change of analogue internal standard.

For changes to the instrument conditions, the analyst’s discretion will determine the extent, if any, of further partial validation. A change between very similar instruments (e.g., changes between mass spectrometers that have the same ionization sources) will not require a partial validation despite the expectation that minor changes to the instrument conditions will occur to optimize performance.

The following is a listing of the parameters that might be assessed. Except where indicated, acceptance criteria will mirror those expected in a full validation:

• Inter-assay precision and accuracy:

• Acceptance criteria: the same procedures and acceptance criteria as the full validation

• Selectivity and matrix effect: evaluation of six lots (ten for LBA) of matrix for interference. In the case of rare matrices, or where matrices are ethically difficult to obtain, a justification may be made for not using the authentic matrix.

• Acceptance criteria: the same procedures and acceptance criteria as the initial validation.

• Recovery: evaluated at or near at least three QC levels used during validation. For some sample preparation procedures (e.g., direct precipitation or on-line automation techniques), recovery determination is not applicable and may not be assessed.

• Acceptance criteria: there will be no acceptance criteria for recovery but it should be consistent across the range.

• Study matrix stability: bench top, freeze/thaw, freezer and whole blood.

• Processed sample stability and autosampler stability if there are changes in extract conditions.

• Stock and working standard stability: if concentrations or storage conditions are changed, and also if a lot of change is deemed to be significant for a large molecule.

The following additional parameters should be considered for ligand binding assays

• Intra-assay precision and accuracy

• Dilution linearity: where method modification has affected the analyte response.

• Specificity (cross reactivity): if the binding agent is changed (e.g., new lot) or the drug naïve matrix differs from the original matrix, in the case of concomitantly administered drugs if they share epitopes with the target protein.

• Well to well contamination: if lot number of plates is changed or the ULOQ is increased

• Parallelism: if the matrix or matrix pool has been modified.

• Hook effect: if the matrix is altered or the species is changed or the ULOQ is raised

• Minimum required dilution: if the matrix is altered or the species is changed

• Changes to the sample preparation procedure if their influence on the response and robustness can considered to be minor (as justified during method development): change in reagent, cycle time, washing steps, incubation temperature, pH value

CROSS VALIDATION

The term cross validation has been applied to different situations industry wide. While there is a definition for the term cross validation in the FDA guidance document (5), the Crystal City Conference report (8) acknowledges that the term cross validation was used “liberally” during the conference. In particular, the conference report referred to circumstances where cross validation might be required but without consensus on the issue. These included a need to define a validation process when matrix from different species is used, for matrices with different anticoagulants, transferring methods to other laboratories and transferring methods to other analysts. In reviewing the guidance from different regions, it should be noted that there were no specific references to cross validation in the Brazilian guidance (6). The FDA guidance indicates when a cross validation is required but gives no detail on how to conduct it (5). The EMA and Japanese guidelines provide some limited guidance on the acceptance criteria that might be applied but give no detail beyond that (4,7).

The harmonization team has the following definition of cross validation: Cross validation is a comparison of data from at least two different analytical methods (reference method and test method) or from the same method used by at least two different laboratories (reference site and test site) within the same study. The goal is to determine whether the obtained data are comparable. Cross validation consists of analysis of quality control samples (either spiked, incurred samples, or both), assayed under the different experimental conditions or different sites with validated methods, as appropriate.

The same set of biological samples should be measured by both analytical sites or using the two different analytical methods. It is desirable that cross validation should be performed in advance of study samples being analyzed. It is recommended that the following rationale should be used in deciding how best to perform the cross validation:

• Where the same analytical methodology is being used in at least two different laboratories to compare data from within the same study, it is acceptable to use spiked QC samples to make the comparison.

• Where different analytical methodologies are being used, both spiked QC samples and incurred samples should be used to make the comparison

Using Spiked Quality Control Samples

Quality control samples are prepared at a minimum of three concentrations spanning the calibration range, aliquoted and distributed to the test site. The preparation can be performed by the initial laboratory or by a third party. The advantage of the former is that the methods of preparation will be in line with the successfully validated method and inconsistencies in preparation that may lead to method variance may be avoided; however, it is extremely important that neither method be considered correct relative to the other as there can be no basis for such a conclusion. .

The team recommends a minimum of 15 quality control samples be tested (e.g., 5 replicates at 3 QC levels) at each site.

Acceptance Criteria

Mean accuracy at each concentration level should be between 85.0% and 115.0% of nominal concentration for chromatographic assays and between 80.0% and 120.0% of nominal concentration for biological assays.

The precision (%CV) should be within 15.0% for chromatographic assays or within 20.0% for ligand binding assays.

All results, except those rejected for clearly identified and documented analytical reasons, must be used for the calculation of accuracy and precision.

Using Incurred Study Samples

Assays are preferably conducted on individual samples and not on pools. The reference site should select between 20 and 40 study samples distributed across the study. Select samples randomly but with some discretion to allow selection of representative concentrations across the range of obtained results. The samples are aliquoted and distributed to the test site. Try to avoid selecting samples <3 × LLOQ. If the two methods have different LLOQs, avoid selecting samples below the 3 × LLOQ of the analytical method with the higher LLOQ. For multi-analyte assays it is recommended that the selection should be based on the concentrations of the primary active analyte.

The following points should be considered:

• If the first analysis is conducted as a single replicate, the re-analysis must be conducted as a single replicate.

• If the first analysis is conducted as a multiple replicate, the reanalysis should be conducted in an equal number of replicates

• Study samples may be diluted if the concentration is expected to be above the upper limit of quantitation.

Acceptance Criteria

At least two thirds of the concentrations measured under different experimental conditions must have a deviation (percentage difference) below or equal to 20% of the mean result for chromatographic methods or below or equal to 30% of the mean result for ligand binding methods.

$$\mathrm{Percentage}\mathrm{difference}=((\mathrm{measured}\mathrm{concentration}\mathrm{condition}1-\mathrm{measured}\mathrm{concentration}\mathrm{condition}2)/\mathrm{mean}\mathrm{concentration})\times 100$$

In cases where the acceptance criteria are not met, an investigation should be performed to find the source of method differences. Further samples may need to be assayed to help determine a potential source of laboratory error either in the original or the repeat analyses. Ultimately, if the source of differences is not identified, it would be necessary to evaluate the potential impact the differences may have on the intended study and this may necessitate further development.

Criteria for further evaluation should be clearly defined in the cross validation protocol along with permitted actions for evaluation. It is important to ensure that the protocol controls these processes to ensure that evaluation and investigation are not used as ways to test until successful. It must be clearly stated that these data should be used for method evaluation only and not to be reported as study sample concentrations.

A number of areas of confusion exist with cross validation. The first of these are the apparent divergence between the use of cross validation for a comparison of methods under the guidance and for comparison of methods on method transfer. The requirements that trigger a cross validation under the existing validation guidance are very clear although none of the current guidance’s describe how to conduct such a cross validation (4–7). Very often, organizations conducting a method transfer will ask for a cross validation exercise in a format similar to that which we have outlined above. In effect, the regulatory and transfer cross validations are being used for a very similar purpose, namely to establish the equivalence of two methods.

The other area of confusion arises when the data is evaluated. It is often the case that methods show significant bias. In these cases, there may be a tacit assumption that the original method is correct and the new method divergent from it. In reality, no such differentiation can be made. It is therefore desirable for both participating laboratories to dispassionately assess the data and investigate the problems from the point of view that either or both methods may be introducing bias, in order that the root cause of the discrepancy can be understood. Obviously, this has more potential impact if conducted later in drug development as any bias leading to methods being called into question may raise concerns about data generated to date. For this reason, the group considers that establishment of methods at two sites or changes in the basis of measurement should be introduced as early in development as possible or avoided.

CONCLUSIONS

It is clear that language used to define method transfer, partial validation, and cross validation has been used interchangeably in the past. This harmonization team has tried to isolate and define these related activities and present guidance on how these activities should be applied both individually and together. The team believes that the definitions of method transfer and cross validation it has proposed will add clarity to the way these activities are described and conducted during drug development. It was not possible for the team to be so definitive about partial validation, given the need for scientific evaluation to be conducted in each individual case, but we hope that the discussion of the considerations that should influence decisions on the need for and extent of partial validation will prove of value to analysts and regulators alike.