Fit for purpose? A case study: validation of immunological endpoint assays for the detection of cellular and humoral responses to anti-tumour DNA fusion vaccines

Abstract

Clinical trials are governed by an increasingly stringent regulatory framework, which applies to all levels of trial conduct. Study critical immunological endpoints, which define success or failure in early phase clinical immunological trials, require formal pre-trial validation. In this case study, we describe the assay validation process, during which the sensitivity, and precision of immunological endpoint assays were defined. The purpose was the evaluation of two multicentre phase I/II clinical trials from our unit in Southampton, UK, which assess the effects of DNA fusion vaccines on immune responses in HLA-A2+ patients with carcinoembryonic antigen (CEA)-expressing malignancies and prostate cancer. Validated immunomonitoring is being performed using ELISA and IFNγ ELISPOTs to assess humoral and cellular responses to the vaccines over time. The validated primary endpoint assay, a peptide-specific CD8+ IFNγ ELISPOT, was tested in a pre-trial study and found to be suitable for the detection of low frequency naturally occurring CEA- and prostate-derived tumour-antigen-specific T cells in patients with CEA-expressing malignancies and prostate cancer.

Introduction

Measurements of immune responses provide important endpoints for monitoring the efficacy of anti-cancer vaccines in early phase immunotherapeutic trials [1]. The formal validation of these endpoint assays, which define the success or failure of a study, has now become a necessary pre-trial requirement [2–5]. The validation process itself provides an opportunity to characterize and improve the performance quality and consistency of the endpoint assay. As a result validation permits the generation of reliable and interpretable data sets. However, the task of optimizing and characterizing the performance of immunological endpoint assays for the specific conditions of each trial is by no means insignificant or trivial. The substantial impact on resources in terms of time spent, consumables and appropriately trained personnel is particularly difficult for small academic laboratories. As funding for this type of activity is not a priority for granting bodies it must be accommodated within the clinical trial budget. In recent years there has been a welcome growth of interest in the issues surrounding assay validation in the immunotherapy field [6–13]. In particular, the need for comparability between results published by different laboratories worldwide has led to the formation of international working groups and the introduction of large scale initiatives of inter-laboratory testing programmes and proficiency panels, such as the Cancer Vaccine Consortium of the Cancer Research Institute [8] and the Cancer IMmunoTherapy (C-IMT) Monitoring Panel [6]. For individual laboratories and for the immunotherapy community as a whole there are obvious benefits to be gained from these initiatives, ranging from the small-scale optimization of individual protocols to a wider increase in the understanding of and in the credibility of immunomonitoring results.

We are currently undertaking two multicentre phase I/II clinical trials to assess the effect of DNA fusion vaccines on immune responses in HLA-A2+ patients with carcinoembryonic antigen (CEA)-expressing malignancies and prostate cancer. The studies are designed to test anti-tumour DNA fusion vaccines, encoding either the CEA-derived CAP1 peptide or the Prostate-Specific Membrane Antigen peptide, PSMA₂₇, fused to Domain1 (Dom) of Fragment C (FrC) of tetanus toxoid [1, 14]. We are using ELISA to measure vaccine-induced changes in anti-FrC antibody, and ELISPOT for the detection of IFNγ-secreting tumour antigen-specific CD8+ T-cells and FrC-specific CD4+ T-cells. Here we describe our approach to the pre-trial optimization and validation of the ELISA and ELISPOT assays. Our objectives were to show that the assay performance characteristics would meet the requirements of the intended analytical application and to generate reliable data sets with which to assess the immunological outcome of the vaccines. Specifically we intended to assess changes in responses compared to a baseline value. We tested the performance of the CD8+ T-cell IFNγ ELISPOT for a sample of the patient population in a pre-trial study. Additionally, our participation in a series of inter-lab testing panels with C-IMT [6] has provided a hitherto unavailable means of external validation.

Materials and methods

DNA vaccine clinical trials

The anti-CEA DNA Vaccine (ACVA) study, GTAC 076 (www.advisorybodies.doh.gov.uk/genetics/gtac/index.htm), and the anti-PSMA₂₇ DNA Vaccine (PSMA₂₇) study, GTAC 089, are multicentre phase I/II trials of antigen/pDom fusion genes [1]. Peripheral blood mononuclear cells (PBMCs) and serum samples are being collected for immune monitoring over a series of 16 (ACVA) or 19 (PSMA₂₇) timepoints for the study duration of 64 or 72 weeks, respectively.

Sample collection and storage

All human samples were obtained following ethical approval by the national (NREC) and local research ethics committees (LREC), and were taken and used with informed consent of the donors. Time between venesection and processing of fresh blood samples was pre-defined to be shorter than 4 h. Procedures were performed by trained personnel according to formal standard operating procedures (SOPs). The ELISA was validated using human serum samples, donated by healthy volunteers and stored frozen at −80°C. The IFNγ ELISPOT assays were validated using cryopreserved PBMCs from healthy donors. PBMCs were isolated from heparinized blood samples using Lymphoprep (Axis Shield), recovery and viability assessed using a manual haemocytometer and Trypan Blue exclusion. Aliquots of 5 × 10⁶ PBMCs were cryopreserved in RPMI 1640 with 50% human AB serum (Sigma) and 10% dimethylsulphoxide using a Nalgene controlled rate freezing container overnight at −80°C before transfer to LN₂ vapour phase for storage until required. For the CD8+ ELISPOT, volunteers were genotyped for the presence of HLA-A0201 by PCR in the transplant immunology laboratory of our centre.

Anti-FrC ELISA and anti DOM ELISA

Our vaccine design incorporates DNA encoding a minimized section (Dom) of FrC, which is a ~50 kDa c-terminal portion of tetanus toxoid [1, 14]. For the anti-FrC ELISA, recombinant FrC protein antigen was generated in house by expression in Escherichia coli. Briefly, 96-well plates were coated with 1.25 μg/ml FrC overnight at 4°C. Plates were blocked at 37°C for 1 h with 1% Bovine serum albumin (Sigma). Triplicates of the reference standard tetanus antitoxin human immunoglobulin, (National Institute of Biological Standards and Control, UK) were double diluted to give seven values, ranging from 0.01 IU/ml—0.000156 IU/ml. Quality control (QC) samples, previously bulk-prepared from pools of human serum and stored frozen as single use aliquots, were used as internal standards to cover high, mid and low acceptance ranges of the standard curve. Test sera were plated in triplicate at appropriate dilutions (initial titrations had been performed to establish the dilution that would fall on the linear part of the curve) and plates incubated at 37°C for 1 h. Detection was by addition of goat anti-human IgG (Fc) antibody conjugated with horseradish peroxidase for 1 h, followed by o-phenylenediamine substrate (both Sigma) Absorbance was read at 490 nm using a BIORad Ultramark 680 ELISA Microplate reader. FrC-specific antibody was quantified in relative antibody units (RAU) against the tetanus antitoxin reference standard. The ELISA was designed to detect relative changes in anti-FrC antibody over time. A positive response was set at ≥twofold compared to baseline. The statistical significance of differences between different timepoints was determined where P < 0.05 using a Student’s t-test.

A post-validation modification of the ELISA was developed using Dom protein as antigen. Recombinant Dom protein was generated in a mammalian expression system as described below.

IFNγ ELISPOT

CD8+ memory responses were measured against 10 μg/ml of a viral peptide pool [15] comprising equal amounts of cytomegalovirus (CMV), CMV pp65 493–499, (NLVPMVAVT); influenza A, Matrix 1 58–66, (GILCFVFTL); Epstein Barr virus (EBV), BMLFI 259–217 (GLCTLVAML), and measles, non-structural C protein 84–92 (KLWESPQEI) [16]. An HIV peptide, IV9 RT 476–484 (ILKEPVHGV) [17] served as negative control. All trial patients were consented to and offered counselling with respect to HIV testing; HIV positivity is a key exclusion criterion to our studies, as we anticipated that HIV infection would compromize the patients’ immunocompetence. Tumour antigen derived peptides for the pre-trial study were: CAP1, CEA 605–613 (YLSGANLNL) [18] two heteroclitic derivatives, CAP1-6D, (YLSGADLNL) [19] and CAP1-8D (own unpublished sequence, YLSGANLDL), IMI, CEA 691–699 (IMIGVLVGV) [20], PSMA₂₇, 27–35, (VLAGGFFLL) [21]. HLA A0201-binding peptides were obtained, with certificates of ≥95% purity, from Peptide Protein Research Ltd (Fareham, UK). CD4+ responses were measured against 20 μg/ml recombinant FrC protein. Recombinant cκappa-tagged protein was generated in house, using a mammalian 293-F expression system (Freestyle, Invitrogen), and purified on a column of polyclonal sheep anti-human free kappa linked to Sepharose 4B beads. Protein quality was confirmed on SDS gels and anti-kappa blots and endotoxin levels were tested before use in ELISPOT. In both assays 5 μg/ml phytohemagglutinin (PHA) was used to confirm that cells were able to produce detectable IFNγ.

In brief, clear Multiscreen 96-well ELISPOT plates (Millipore, MAIPS4510) were pre-coated with 15 μg/ml anti-human IFNγ antibody (mAb 1 D1 K, Mabtech), and left overnight at 4°C. Thawed PBMCs (4 × 10⁵/well) were incubated in triplicate for 20 h (CD8+) or 40 h (CD4+), at 37°C, 5% CO₂ with medium only, test antigen (10 μg/ml peptide or 20 μg/ml FrC protein) or 5 μg/ml PHA in RPMI 1640 with 10% human AB serum. IFNγ secreting memory T-cells specific for the antigen were detected as spots using 1 μg/ml biotinylated IFNγ antibody (mAb 7B61 biotin, Mabtech) followed by 1 μg/ml streptavidin alkaline phosphatise (Mabtech) and a BCIP/NBT detection kit (Zymed). Spots were counted using an AID automated ELISPOT plate reader (ELRO2) A positive response to an antigen was defined where mean spot forming cells (SFC)—baseline is greater than 10 SFC/well and also >2σ above medium only wells. The statistical significance of differences between different timepoints was determined where P < 0.05 using a Student’s t-test.

Stability of materials

All commercially sourced reagents were stored according to manufacturers’ instructions; batch numbers were recorded and expiry dates adhered to. The stability of antigenic proteins, peptides and serum samples were assessed over three freeze thaw cycles and at relevant temperatures (4°C, room temperature) for 4, 24 h or 7 days following thawing (results not shown). We determined that these materials should be stored frozen (recombinant protein −20°C, peptides and serum −80°C) and used immediately after thawing. Serum should not undergo more than two freeze-thaw cycles and proteins and peptides should not be subjected to more than three freeze thaw cycles.

The stability of frozen serum (QC samples) was confirmed in the ELISA over a period of 17 months, and ongoing stability checks are performed. The stability of peptides was tested by thin layer chromatography (check for degradants), and had been confirmed over 16 months.

Results

Assay validation

Our approach to assay validation followed guidelines from several sources [5, 22–24]. The overall validation process is outlined in Fig. 1. Endpoint immunoassays defined in the Clinical Trial Protocol were optimized and validated by trained personnel to demonstrate their reliability and suitability as immunological endpoint assays. Sample handling systems, including sample processing, storage, tracking and handling by external study centres, were also validated. Formal SOPs of validated assays and sample handling systems were incorporated into an Analytical Study Plan prior to commencement of the trial. Additional resources for assay validation were obtained through a pre-trial feasibility study and by participation in the C-IMT inter-laboratory monitoring Panel [6]. Each stage was initiated following discussion with a designated qualified person from Cancer Research UK (CR-UK), as the main funding body. We adopted their recommended quality system of Good Clinical Laboratory Practice defined by the British Association for Research into Quality Assurance (BARQA) [24]. In line with this, formally documented Validation Plans and Reports were generated for review and approval by the funding body prior to commencement of the trial. All three of the immunological assays described in this case study had been developed previously in house and were well established as research assays in our laboratory. They therefore required only a limited amount of optimization to transfer the method into the intended clinical setting. The assay validation parameters were specificity, accuracy, sensitivity and precision.

Fig. 1.

Flow chart of the validation process for endpoint immunomonitoring of anti-tumour DNA fusion vaccines in clinical trials. The main validation process (grey background) was performed in conjunction with the funding body, Cancer Research UK (CR-UK) (see main text)

Validation of the anti-FrC ELISA

We used an anti-FrC ELISA with a tetanus antitoxin reference standard to detect humoral responses to the vaccine. The ELISA had originally been developed as a research assay to test the efficacy of vaccine designs in pre-clinical murine models. Some modifications were necessary for use in the human setting. Mice generate high anti-FrC antibody levels in response to vaccination and, unlike many individuals from the human population, do not possess any pre-existing anti-tetanus immunity. As the increase post DNA vaccination was unknown, the ELISA needed to be sensitive enough to measure small changes for clinical trial evaluation.

Reference standard

Quantification of anti-FrC IgG was relative to serial dilutions of the reference standard of tetanus antitoxin human immunoglobulin (see "Materials and methods”). We introduced a series of QC samples to monitor the standard curve, and maintain accuracy. These were made up from pools of human serum with known, independently measured, anti-tetanus titres (high, intermediate and low). The QC samples were also used to test dilutional linearity. This was demonstrated (r ² > 0.99) within the range of 1:200 and 1:1,600, but not for 1:3,200 or 1:6,400 (not shown). It was decided that patient test samples should undergo an initial titration to establish the dilution that would fall within the linear range of the standard curve.

Assay specificity

We confirmed that the tetanus antitoxin reference standard could be used to derive accurate measures of serum antibody titres to the FrC component of tetanus toxin. The relationship between measurements of anti-FrC and anti-tetanus antibody titres were demonstrated in sera from healthy volunteers. In an independent research study healthy volunteers were vaccinated with tetanus toxoid and anti-tetanus IgG responses were assessed pre- and up to 20 weeks post- vaccine by ELISA [25]. The same sera were available for making comparative measurements of anti-FrC IgG. Measurements of FrC-specific antibody were qualitatively comparable, both in amplitude and timing, to responses against the full tetanus toxoid (Fig. 2a, b). Direct comparison of all measurements demonstrated that there was a strong positive correlation between the two independent data sets (r ² = 0.94). We could therefore justify the quantification of anti-FrC antibody in RAU using the tetanus antitoxin standard.

Fig. 2.

Anti-FrC ELISA specificity: the tetanus antitoxin reference standard was demonstrated to be valid for quantifying anti-FrC antibody; a strong correlation was observed (r ² = 0.94) between independent ELISA measurements of a anti-tetanus (IU/ml) and b anti-FrC (relative antibody units) antibody in four healthy volunteers following vaccination with tetanus toxoid adapted from Ref. [25]. c Assay selectivity was demonstrated by comparison of FrC-specific antibody in serum (n = 5) with remaining non-specific background after pre-incubation of serum with FrC protein (mean background18 ± 3%)

As our volunteer population had received a tetanus vaccination at some time in the past, it was not possible to obtain a negative control serum sample. The level of non-specific interference from the biological matrix in the absence of anti-FrC antibody was therefore assessed experimentally. Sera (n = 5) were pre-incubated with FrC to deplete anti-FrC antibodies, and the remaining absorbance provided a measure of non-specific background. FrC-specific antibody titre decreased by 82 ± 3% (Fig. 2c), and confirmed that the assay had a good level of selectivity.

Inter- and intra-assay variability

The within-lab precision (inter-assay variability) and repeatability (intra-assay variability) of the ELISA were assessed simultaneously. Two operators performed four separate ELISA assays, 2 on 1 day and two on another. Overall assay variability was quantified as the mean co-efficient of variance (%CV) between plates, assays and operators (determined as percentage variations of triplicates where %CV = [(σ/mean) × 100%]. Intra-assay variability was determined as the overall mean %CV from series’ of four (high responder) or eight (low responder) sets of triplicates. Inter- and intra-assay variability was acceptable, and did not exceed 25% for high or low responses (Table 1). The mean inter-operator CV was greater, with a maximum of 28% observed for the low responder. This confirmed the need for batch testing to minimize assay variability, and we therefore determined that each series of samples (all timepoints from one patient) should be assayed together on the same day by a single operator.

Table 1.

Anti-FrC ELISA: intra- and inter-assay variability

Parameter	Mean (%CV) High responder (FrC-specific ab 2.3 RAU/ml)	Mean (%CV) Low responder (FrC-specific ab 0.9 RAU/ml)
Inter-assay variation (same operator)	12.0 ± 6.8	13.1 ± 4.0
Inter-operator variation (different operators)	11.9 ± 6.2	15.6 ± 11.3
Intra-assay variation (well-well)	6.5 ± 6.4	3.7 ± 2.5

Overall assay variability was quantified as the mean co-efficient of variance (%CV) between plates, assays and operators (determined as percentage variations of triplicates where %CV = [(σ/mean) × 100%]. Intra-assay variability was determined as the overall mean %CV from series of four (high responder) or eight (low responder) sets of triplicates

Our acceptance criteria for the precision and accuracy of this assay had been set at a value of 25%, and 30% at the lowest QC value [23]. As a secondary endpoint assay, the validation results demonstrated acceptable levels of precision for detecting relative changes in anti-FrC antibody over time.

Post-validation modifications

The ELISA had originally been developed for testing DNA fusion vaccines with FrC specificity. Results from an earlier trial of individual DNA idiotypic scFv-FrC fusion vaccines in patients with follicular lymphoma revealed that positive detection of vaccine-related increases in anti-FrC antibody was dependent on the level of pre-existing immunity (manuscript in preparation [26]). Positive responses (>twofold) were less easily detected in patients with higher initial anti-tetanus immunity. A further parameter of “weak positive” response was therefore introduced where the antibody response was <twofold, with a clear statistically significant increase above baseline over multiple timepoints.

In our current trials, the vaccine designs incorporate a sequence for the minimized Domain 1 portion of FrC [14]. Following successful generation of recombinant Dom protein, we have now developed an anti-Dom ELISA to complement results from the anti-FrC ELISA. Only minor modification was required, within-lab precision has been demonstrated and the assay exhibits comparable performance characteristics (results not shown). Pre-existing titres of anti-Dom antibody appear to be uniformly low in the sera so far tested, compared to the varied pre-existing anti-FrC titres (data not shown, own unpublished observations). Results from the anti-Dom ELISA are therefore able to provide additional confirmation of the “weak positive” anti-FrC responses.

Validation of the anti-CAP1 and anti-PSMA₂₇ IFN γ ELISPOTs

ELISPOT was selected as the most suitable methodology for quantifying both tumour antigen-specific and FrC-specific IFNγ-secreting T-cells in our clinical setting. The assay appeared to fulfill the desired criteria for monitoring T-cell responses to the anti-CEA vaccine [11]. It had a reported sensitivity of 1:10,000–1:50,000 antigen-specific T-cells [11, 27, 28]; it was suitable for batch testing of cryopreserved PBMCs over a series of timepoints [27, 28]; it was cost effective and relatively simple to perform for trained personnel. A further advantage was that both CD8+ and CD4+ assays could be set up simultaneously. This made efficient use of the limited cell numbers, reagent costs and time. Our intention was to use the identification of tumour antigen-specific T cells in the trial population as an indication of where further characterization (e.g. by multimer staining, cytotoxicity assays) should be made.

Our ELISPOT protocol was closely based on that published by Smith et al. [28]. We performed pre-validation optimization of incubation times and reagent concentrations to generate the best signal/noise ratio. Human AB serum was batch tested to generate the lowest background. Both the CD8+ and CD4+ ELISPOTS were validated to the same standard and both assays demonstrated very similar performance characteristics. Here we present the main results from validation of the primary endpoint assay, anti-CAP1/PSMA₂₇ CD8+ T-cell IFNγ ELISPOT. As reference samples with defined numbers of antigen-specific T cells were not available to us at the time of assay validation, we used a pool of viral peptides as positive control (see "Materials and methods”). For our purposes this did not pose a problem, as for the clinical trials our interpretations of the ELISPOT were based on changes over time rather than absolute numbers.

PBMC quality: cryopreservation and sample handling

As cryopreservation has been shown to affect some cellular cytokine response profiles [12], we investigated the validity of using frozen, rather than fresh PBMCs in our IFNγ ELISPOT protocol. Responses to the viral peptide pool were measured in freshly extracted PBMCs from three healthy volunteers. The remaining cells were cryopreserved in liquid N₂. Repeat assay were performed on the thawed cells 1 and 2 weeks later. The cryopreservation process did not lead to loss of response (P > 0.05, Student’s t-test, data not shown), which confirmed the findings of other workers [8, 27–29]. Furthermore, T-cell responses were stable following cryopreservation over the course of 17 months (Fig. 3a). This also agreed with other studies [28], and supported our requirement of batch testing each series of frozen patient samples from the trial. This was especially important for this multicentre trial, where PBMCs would be prepared and cryopreserved at external sites, then shipped to our laboratory in batches for later analysis.

Fig. 3.

CD8+ IFNγ ELISPOT: a PBMC stability: aliquots of 5 × 10⁶ cryopreserved PBMCs from the same donor were thawed at different times over a period of 17 months. Background IFNγ release from unstimulated cells (white columns) was consistently low over time. IFNγ release in response to the viral peptide pool (cells only background subtracted, dark grey columns) was not compromised. b PBMC titration: cell inputs of 2 × 10⁵ and 4 × 10⁵/well gave similar responses to the viral peptide pool when normalized to spots/10⁶ PBMCs. The sum of the individual viral peptides equaled the sum of the pool. This was not observed for an input of 1 × 10⁵, due to a high variability for triplicates (up to 83%) where spot counts <10. c Determination of lowest cut-off value for spot forming cells (SFC)/well was by comparison of mean SFC/well and %CV = [(mean/σ) × 100%] for triplicates. %CV was unacceptably high where SFC <10

It has become increasingly evident that results of cellular assays can be compromised by poor sample quality, through inappropriate handling, inconsistent processing methods, freezing techniques [8, 12, 13]. In a multicentre trial it is absolutely critical to maintain consistency in sample quality. We therefore generated an SOP to standardize the procedures for sample handling at external study centres. The aims were:

1. To promote consistency in sample handling procedures between participating centres.

2. To work within GCP guidelines.

3. To maintain a coherent, auditable sample tracking system.

4. To obtain consistent interpretable results in endpoint assays for monitoring immune responses.

For geographically close external centres (up to 1 h driving time) blood samples were shipped by courier to arrive in Southampton for processing within 4 h of being drawn. For more distant centres training in sample handling, processing and freezing procedures was undertaken according to our SOPs, and the resultant sample quality tested following shipping to Southampton. Testing included monitoring cell yield and viability and comparing functional responses in IFNγ ELISPOT.

Assay sensitivity

Selections of optimal cell numbers per well were made as part of the validation process. The optimal PBMC input needed to be a balance between the requirements of sensitive detection of low frequency tumour associated antigen-specific cells above background, and the limitations of sample availability from trial patients. Following normalization, PBMC inputs of 2 × 10⁵ and 4 × 10⁵/well generated similar results in response to the viral pool (Fig. 3b). Responses to the viral pool also equaled the sum of responses to the individual component peptides. There was some discrepancy in these values for lower cell input of 1 × 10⁵. The lack of correlation here between responses to the viral pool and the individual peptides was found to be caused by a large variability in the triplicates where spot counts/well were <10. This allowed us to identify the cut-off below which the values became unreliable. By taking all measurements (single peptides, viral pool and PHA) and plotting the %CV against the mean SFC/well of triplicates, it became apparent that there was a clear inverse relationship between the two (Fig. 3c). Similar observations have been made for the enumeration of human papilloma virus by Lathey [30]. We set our cut-off at 10 SFC/well, below which the inter-well variability was unacceptably high (>25%). Cell input number was set at 4 × 10⁵/well to maximize the chance of detecting low frequency antigen specific cells.

Inter-assay and inter-operator variability

Inter-assay and inter-operator variability were assessed simultaneously; two trained operators performed identical ELISPOT assays on samples from donors with low (>10 < 50 SFC/well), mid (>50 < 100 SFC/well) or high (>100 SFC/well) responses [8] to the viral pool, individual viral peptides and PHA. Identical assays were also performed on different days to provide an assessment of within-lab inter-assay variability. Results are shown in Table 2. Inter-assay CVs, at 24% or below, were within the acceptable range (<25%), apart from a low response at 51%. The mean SFC/well for this response was 15 ± 10. Spot counts were above the >10 cut-off, but this could not be called a positive response because the mean SFC <2σ above the medium-only value. This response was therefore excluded from the assessment. Inter-operator variability for low and mid value responses (<100 SFC/well) was considerably higher than observed for intra-or inter-assay variability, and reached or exceeded 25% (Table 2). Reasons for this variability were not identified, as cross-counting and cross-analysis of samples processed by the other operator revealed that there were no differences between the two (data not shown).

Table 2.

CD8+ T-cell IFNγ ELISPOT: intra- and inter-assay variability

Donor	A	B
Antigen	VPP	VPP
Mean response ± σ SFC/well	100 ± 25	47 ± 7
Intra-assay mean %CV (triplicates)	7	15
Donor	C	D	E	C	D	E
Antigen	VPP	VPP	VPP	PHA	PHA	PHA
Mean response ± σ SFC/well	15 ± 10	97 ± 23	35 ± 10	372 ± 62	389 ± 56	462 ± 43
Inter-assay mean %CV (same operator)	51	7	nd	13	17	nd
Inter-operator mean %CV (different operators)	62	24	29	16	9	9
Donor	F	F	F	F	F
Antigen	measles	CMV	flu	EBV	VPP
Mean response ± σ SFC/well	0 ± 2	11 ± 2	25 ± 4	41 ± 10	63 ± 9
Inter-assay mean %CV (same operator)	388	22	17	24	14

Variability in responses (donors A–F) is shown as mean %CV = [(σ/mean) × 100%] for intra-assay (well–well, triplicates), inter-plate (plate–plate, same operator) and inter-operator (plate–plate, different operators) validation. Grey shaded boxes show corresponding values for IFNγ release in response to viral peptide pool (VPP), PHA or individual viral antigen peptides

Although there were differences in the absolute spot counts from each operator, a strong positive correlation was observed between the two sets of results (r ² = 0.97). This showed that either operator would generate an identical assessment of positivity from responses to the viral pool or PHA. Similar observations have previously been made by other workers [9, 10]. Our results confirmed that each series of timepoints from a patient should be batch-tested by a single operator to minimize assay variability.

Overall, we concluded that a positive response to an antigen must be >10 spots/well (background removed); for the detection of antigen-specific responses over time in vaccinated patients mean SFC—baseline must be >10/well and also >2σ above baseline. Additionally we mandate a statistically significant increase over two or more timepoints for a result to be interpretable as a response.

Application of the CD8+ IFNγ ELISPOT to the patient population in a pre-trial feasibility study

We were able to examine the performance of PBMCs from the patient population in a pre-trial feasibility study with separate LREC approval. The study was designed to assess whether patients with solid tumours are immunocompetent and able to mount T-cell responses, and whether they possess pre-existing cellular immunity to CEA-derived antigens or PSMA₂₇. Consenting HLA-A2+ patients with CEA-expressing cancer (CEA group, n = 17, serum CEA levels ranging from 1.5 to 956 μg/ml, mean age 68 ± 10 year) or prostate cancer (PrCa group n = 13, serum PSA levels ranging from 0.06 to 180 μg/ml, mean age 75 ± 7 year) were recruited, together with a group of healthy volunteers (n = 6, mean age 34 ± 4 year). We used the validated CD8+ T-cell IFNγ ELISPOT, to measure responses to the CEA derived peptide, CAP1, and peptides from the transmembrane domain of CEA (IMI) and PSMA₂₇.(see Sect. ”Materials and methods” for details of peptides). CAP1 and two heteroclitic peptide analogues of CAP1, CAP1-6D and CAP1-8D (containing substitutions of aspartic acid at positions 6 or 8, respectively) were also evaluated individually, in order to observe whether the epitopes gave differential responses [19].

We had previously calculated that a minimum PBMC yield of 6 × 10⁶/10 ml blood would be required at each timepoint of the DNA vaccine studies to provide sufficient for the endpoint analyses. We were able to confirm that this would be feasible, with respective yields for patients in CEA and PrCa groups of 7.9 ± 3.6 (range 1.5–14.3) and 6.4 ± 3.8 (range 1.3–18.9) × 10⁶/10 ml blood. Mean recovery after cryopreservation was >80% and viability >90%, similar to values obtained for healthy control samples. To ensure sufficient yields for the DNA vaccine trials we also introduced an initial screen to test PBMC yields for candidate patients with borderline lymphopenia (1.0–1.5 × 10⁹/l).

The extent of disease in the study population was varied (Table 3). Widespread disease was present in 16/17 of patients with CEA-expressing malignancies and 5/13 of the PrCa group. ELISPOT results are given in Table 4. All patients exhibited a response to PHA. Positive responses (34/36) to the control viral peptide pool confirmed that T-cell recall responses were intact even in patients with metastatic malignancies. No responses were seen against the HIV negative control peptide. For both patient groups cellular responses to the viral pool were greater than for healthy controls, irrespective of disease extent. Age-associated clonal expansion of CMV-specific CD8+ T-cells has been observed previously in an elderly population (mean age 92) [31], and was associated with phenotypic changes and decreased proportions of CMV specific-IFNγ releasing cells. However, the absolute numbers of IFNγ-secreting PBMCs were similar to those observed in our own study. We are currently generating data for an age-matched control population.

Table 3.

Patient characteristics for pre-trial study to assess pre-existing immunity to tumour-derived antigenic peptides using CD8+ T-cell IFNγ ELISPOT

Patient group	Disease	Most recent treatment	Response to treatment
CEA group (n = 17) patients with CEA expressing malignancies	11 Colorectal carcinomas	Chemotherapy 17/17	Progressed 4/17
	2 Non-small cell lung cancers		Stable 4/17
	2 Oesophageal carcinomas		Partial response 9/17
	1 Hepatocellular carcinoma
	1 Duodenal adenocarcinoma
PrCa group (n = 13) patients with prostate cancer	8 With localized disease	Radical radiotherapy 4/8	Progressed 3/8
		Radical prostatectomy 1/8
		Hormonal therapy 2/8	Partial response 5/8
		No therapy 1/8
	5 With widespread disease	Hormonal therapy 5/5	Progressed 4/5
			Stable 1/5

Table 4.

CD8+ T-cell IFNγ ELISPOT: table of results of pre-trial pre-existing immunity study

Study group	PHA			Viral pool			IMI			CAP-1/deriv			PSMA 27			HIV
	+	Mean SFC/106	σ	+	Mean SFC/106	σ	+	Mean SFC/106	σ	+	Mean SFC/106	σ	+	Mean SFC/106	σ	+
CEA	17	925	889	15	675	503	1	110		0	0	0	1	33		0
PrCa	13	1085	896	13	637	670	5	69	61	5	42	12	2	41	5	0
Control	6	1,625	1,553	6	102	75	4	89	50	0	0	0	0	0	0	0

Responses to PHA and antigenic peptides are given for each study group showing number of responders (+), mean spot forming cells (SFC)/10⁶ PBMCs and standard deviation (σ). Responses to, CAP1, CAP1-6D and CAP1-8D were evaluated for each individual peptide but are displayed summarized

Cellular responses against the self-antigen, CEA trans-membrane peptide (IMI), were detected in 1/17 of the group of patients with CEA expressing cancers (CEA group), 5/13 of the PrCa group and 4/6 of the healthy controls (Table 4). Of these, only the patient in the CEA group had metastatic disease. Low level cellular immune responses to CAP-1 peptides (representative of CAP1, CAP1-6D and CAP1-8D) and PSMA₂₇ were observed mainly in the PrCa group. However, although cellular responses were generally absent in patients with high serum levels of the tumour antigens CEA or PSA, there was no simple correlation between serum levels of CEA and the CAP1 peptide responses, or serum levels of PSA and the PSMA₂₇ peptide responses.

Discussion

ELISPOT and ELISA are widely used tools for monitoring immune responses to anti-cancer vaccines. The assays are relatively simple to perform and facilitate the batch testing of sequential samples. However, current legislation surrounding the use of human material in clinical trials of new investigational agents [2–5] dictates that these endpoint assays should be subject to validation to demonstrate that they are “fit for purpose”. Thus for each new trial the assay validation parameters are tailored to the particular requirements as endpoints. This may range from a definitive quantification of a biomarker to a qualitative indication of a positive or negative response [32]. For immunomonitoring of our DNA vaccine studies, assays were designed to measure relative vaccine-induced changes of FrC-specific antibody, FrC-specific T-cells and tumour-specific CD8+ T-cells over time. Our choice of IFNγ ELISPOT as the primary immunological endpoint was influenced by several factors: the assay had a long history of use with wide acceptance in the community [8, 33–39]; we had used ELISPOT in our pre-clinical models, and possessed the experience and technical capabilities to fulfil the regulatory requirements, and those of the sponsor. For immunomonitoring of our clinical trials, the CD8+ IFNγ ELISPOT is intended to be used in conjunction with other assays, such as tetramer, intracellular staining and cytotoxicity assays, together with phenotypic evaluation by FACS. These assays however presented more technical difficulties with regard to the extent of validation that was possible, and are being undertaken as research assays to consolidate the immunological information generated. The relative merits of ELISPOT and other tests in this context have been subject to a careful review by Coulie and van der Bruggen [40]. More recently, important developments in FACS based multi-parametric analyses have been described [41]; these enable the simultaneous characterization of multiple T-cell functions and hold great promise for the elucidation of vaccine-induced T-cell responses.

For the purpose of this case study, we have focused on the ELISA and ELISPOT assays as conceptual examples of validating immunological endpoints for a specific purpose. The assay validations were performed by trained operators according to a formal validation plan, and SOPs of the validated methods were generated. However, the overall validation process was not confined to the assays themselves, and considerable effort was applied to the implementation of quality systems surrounding sample collection, processing and tracking to ensure consistency between participating centres (Fig. 1).

In addition to the assessments of precision and sensitivity, which had been performed using PBMCs from healthy donors, we were also able to confirm that the primary endpoint assay, the CD8+ IFNγ ELISPOT, was applicable to the relevant patient population. We used the validated ELISPOT to measure pre-existing CD8+ T-cell responses to the viral peptide pool and a panel of tumour antigens in patients with CEA-expressing cancer or prostate cancer, compared to a healthy control group. The yield and quality of PBMCs was not compromised in the patient population, and T-cell responses to recall viral antigens were intact. We concluded that even in patients with advanced disease, T-cell responses to recall antigens are present and can be readily enumerated using our validated CD8+ IFNγ ELISPOT protocol, and that memory T-cell responses to viral peptides do not appear to be negatively affected by disease status.

Tumour antigen-specific T-cell responses were detected in all groups at frequencies of up to 110/10⁶, which confirmed that the ELISPOT was an appropriate tool to use for this purpose. Numerous reports have been made of naturally occurring T-cell immunity to a range of tumour associated antigens (reviewed in [42]), including CAP1 [43]. It is difficult to interpret the biological significance of these observations, as they appear to be dependent on multiple factors; the type of tumour, the extent of disease, the nature of the antigen and whether patients had undergone chemotherapy have all been thought to influence the existence of tumour antigen-specific T-cells in peripheral blood [42]. In our own study, responses to one or more of the test antigens were observed in 1/17 (8%) of the CEA group, 10/13 (77%) of the PrCa group and in 4/6 (67%) of the control group (all IMI-specific). To our knowledge the IMI-specific T cell responses in a healthy population is a new finding, which, although beyond the scope of this paper, warrants further investigation in follow up studies using larger numbers of volunteers. With regard to the patients, our observations suggested that naturally occurring cellular immunity to these tumour antigens may be compromised in the presence of high antigen levels or advanced disease. However, the presence of a measurable T-cell repertoire against tumour antigens such as CEA and PSMA in patients with low disease burden suggests that this clinical setting can be exploited for vaccination against tumour antigens, although the testing of anti-tumour vaccination may best be performed in the adjuvant setting.

During 2005 and 2006 our laboratory participated in an international inter-laboratory testing project to compare protocols for enumeration of antigen-specific T-cell responses, including tetramer staining and IFNγ ELISPOT [6]. In Phase I the major protocol variables influencing sensitivity were identified from the results of 12 European centres. Incorporation of new guidelines into the Phase II panel led to successful increases in sensitivity and reductions in inter-centre variability. From our own perspective we were able to benefit from the combined professional expertise of the other participating centres; we received performance feedback showing that our cell recoveries, low background and detection of positive responses at a frequency of 12 SFC/well compared favourably with other experienced laboratories, and providing confirmation of the validity of our own protocol choices. This form of external validation proved to be invaluable for testing the validity of our method within a wider sphere. Consequently the concept of “fit for purpose” can now be extended from intra-laboratory validation to the purpose of generating comparable and acceptable results across the wider scientific community.