Development of a robust cell-based potency assay for a coxsackievirus A21 oncolytic virotherapy

Abstract

Oncolytic viruses (OV) are part of a burgeoning field of investigational oncolytic therapy (OT), in which lytic viruses dissolve advanced tumors productively and specifically. One such OT is a Coxsackievirus A21 (CVA21) based OV that is currently under clinical evaluation. A tissue culture infectious dose (TCID50) assay was used for CVA21 potency release and stability testing in early clinical development. The titer measured in this method was an extrapolated value from cytopathic effect (CPE) observed during the serial dilution but doesn't represent direct viral killing of cells. Moreover, the assay was not deemed to be optimal to carry into late phase clinical development due to limitations in assay precision, turn-around time, and sample throughput. To address these points, we developed a plaque assay to measure viral plaque forming units to measure the potency value for drug substance (DS), drug product (DP) and virus seed (master and working) stocks. In this manuscript, we describe the steps taken to develop this plaque assay for the late-stage clinical development, which include the assay qualification, validation, and robustness protocols, and describe statistical methods for data analysis. Moreover, the method was validated for linearity, accuracy, precision, and specificity. Furthermore, the plaque assay quantifies OV infectivity with better precision (32% vs 58%), with higher sample throughput (22 samples/week vs 3 samples/week) and shorter assay turnaround time (4 days vs 7 days) than the TCID50 method. This assay development strategy can provide guidance for the development of robust cell-based potency methods for OVs and other infectious viral products.

Highlights

• •Developed a CVA21 oncolytic virotherapy potency method with improved precision, time, and throughput.
• •Qualified and validated the plaque potency method using experimental design and statistical analysis.
• •Specific plaque forming units calculation better represents CVA21 infectious dose and mechanism.
• •Provides a comprehensive guide to potency assay development in viral therapeutics.

1. Introduction

The development of oncolytic viruses for cancer treatment has expanded greatly over the last 10–20 years, employing a wide variety of virus types and multiple transgenes encoding immune modulating proteins. In 2016, reflecting this high level of interest, approximately 40 clinical trials were conducted against multiple cancer types [1]. Between 2000 and 2020, some 119 manuscripts were published describing clinical trials treating 3233 patients [2]. Cancers being investigated include melanoma, pancreatic, hepatic, bladder, ovarian, and glioma, including late-stage cancers. Talimogene laherparepvec (T-VEC), an attenuated herpes simplex virus, type 1 (HSV-1), encoding granulocyte-macrophage colony-stimulating factor (GM-CSF), is licensed for the local treatment of unresectable cutaneous, subcutaneous and nodal lesions in patients with recurrent melanoma after initial surgery [2]. Oncolytic virus types under investigation include both wild-type viruses and genetically engineered virus containing transgenes encoding immunostimulatory genes. One non-genetically modified prototype virus under investigation is Coxsackievirus A21 (CVA21). CVA21 is a non-enveloped picornavirus with single stranded positive sense RNA genome [3]. Some coxsackievirus strains are clinically important and can cause severe pathology such as hand, foot, and mouth disease [[4], [5], [6], [7]]. However, CVA21 infection is only associated with mild cold like symptoms [3]. Several clinical trials have demonstrated the safety and efficacy of Cavatak™ an OV developed from CVA21 [[8], [9], [10], [11], [12]]. Data from the CALM clinical trial demonstrated an overall response rate in 28% of late-stage melanoma patients. Durable responses, persisting for 6 months or more, were seen in 21% of patients [12]. Importantly, anti-cancer activity was demonstrated in satellite non-injected metastatic tumors and also CVA21 appears to induce immunostimulatory molecules in the tumor environment [13]. More importantly, clinical data supports the continuing development of this OV as a stand-alone therapy or in combination with other cancer treatments [12].

Regulatory guidance for oncolytic viruses typically falls under the scope of gene therapy products (e.g. ref: [[14], [15], [16], [17]]). Recommendations and guidance specifically for OV development are becoming available (see e.g. ICH Considerations Oncolytic Viruses, 2008 [18]). In general, the oncolytic virus product will be characterized for safety, specificity, potency, identity, and purity. One standard assessment for virus quality is the ratio of physical, or total particles to infectious particles (the P:I ratio). No specific guidance exists on the acceptable P:I ratio for replicating vectors [19]. Infectious titer may be measured either by determining plaque forming units per volume (PFU/mL) or 50% Tissue Culture Infectious Dose per volume (TCID50/mL). In these assays, serial dilutions of virus are used to infect permissive cells. Cytopathic effect (CPE) can be measured by observing the formation of lytic plaques on a lawn of infected cells (plaque assay) or by observation of infected cells in culture wells in a limiting-dilution format (TCID50) [[19], [20], [21], [22]]. The P:I ratio will vary greatly depending on which method of measurement for infectious dose is used, as the TCID50 assay generally yields a higher value than the plaque assay [21].

The OV potency measurement is critically important to clinical development and dose selection, as well as regulatory approval and commercial success. Therefore, careful consideration must be given to the method used to determine infectious units, and the validation of that method. While measuring CPE in the TCID50 method is commonly used to measure live virus potency, the assay does not directly measure the number of productive infectious particles. Additionally, the method can have more variability than the plaque method due to the imprecise nature of the end point in this assay [21]. For late-stage development of CVA21, a plaque method for measuring potency was chosen as the more robust method. The plaque assay is relatively straightforward to perform, requires no specialized instrumentation or reagents, can be completed in a shorter time frame and is readily transferrable to other labs. Moreover, the titer determined by calculation of the specific plaque forming units by a single potent virus particle better represents the oncolytic viral infectious dose and mechanism of action (MOA). This work provides an overview of the CVA21 plaque assay development from the design, optimization, and validation. This step-by-step description can be utilized as a framework and guidance for the development of cell-based potency methods in the OV field and other infectious viral products.

2. Materials and methods

Cells and virus stocks. Master and working stocks of SK-MEL-28 cells, a melanoma cell line established from patient-derived tumor samples (ATCC HTB-72, SK1980-524) [23], were prepared in-house, and cultured in Eagles Minimum Essential Media (EMEM) + 10% Fetal Bovine Serum (FBS) and 1% Penicillin-Streptomycin (Pen/Strep). Test articles (TA) of CVA21 virus included in-house generated Drug Substances (DS), Drug Product (DP), Working and Master Virus Seeds used at various stages of method development. A single lot of virus prepared from representative bulk DS in a stabilizing matrix buffer, was thawed and refrozen in a large number of single use 0.1 mL aliquots and used throughout the assay development and execution. This was referred to as the System Suitability Sample (SSS) and was tracked for assay acceptance criteria and assay drift.

Test article assay dilution medium. EMEM with l-glutamine containing 2% FBS and 1% Pen/Strep.

Sample dilutions. Appropriate sample (test articles, TA) dilutions were critical to generate assay results which met the assay acceptance criteria (at least two wells with 5–55 plaques per well) and thus determine a valid end point plaque titer. In a 96 well plate, 10-fold serial dilutions were performed on all TA to obtain a concentration approximately 1E+03. These were referred to as the pre-dilutions. For example, if a sample was expected to be at ∼1E+09 PFU/mL, then this sample was pre-diluted by 6 × 10-fold dilution steps. The final pre-dilution of each TA was then further serially diluted using 3-fold dilutions, for a total of 5 additional dilutions. The final [6] TA dilutions were added to the plate of SK-MEL-28 cells.

Plaque Assay Method. In the final version of the plaque assay method (Fig. 1), SK-MEL-28 cells were seeded in 12-well cell culture plates at 5.0E+05 cells/well and incubated at 37 °C, 5% CO₂ for 24 ± 4 h to allow the cells to adhere. The cells were then infected with diluted CVA21 TA and incubated at 37 °C, 5% CO₂ for 90–105 min to allow for virus adsorption. Following the adsorption period, an overlay (1% Methyl cellulose at 1.5mL/well) was added, and the infected cell plates then returned to the incubator for 72 ± 2 h. After this incubation, the overlay was removed by pipetting, cell plates were washed twice with Dulbecco's phosphate buffered saline (DPBS) and fixed with 80% methanol. The plates were then stained with Coomassie brilliant blue staining solution, which binds to all cells that are uninfected and adherent to the plate. Infected cells will not adhere to the plate surface, thus leaving a clear open spot which is referred to as a plaque (Fig. 4).

Fig. 1.

Flow chart of method procedure.

Fig. 4.

Coomassie stain of CVA21 plaques in SK- MEL-28 cells. Two identical test articles (CVA21) were serially diluted and applied to the SK-MEL-28 cells, starting in the first well in the top row, or the third well in the top row, and diluting down the column. The number of plaques is indicated for each well in red numbers. Wells with 5–55 plaques could be counted to yield the final PFU/mL titer (see Methods – Titer calculation for PFU/mL calculation)

Operationally one analyst could run six plates at one time, consisting of nine x TAs and three x SSS. One analyst could perform two runs per week, equal to eighteen TAs, with data turnaround time of 9–10 days.

Titer calculation. The virus titer was calculated for each dilution in Plaque forming units per milliliter (PFU/mL) according to the formula:

(Plaque count of each well) x (the sample dilution) x (the reciprocal of the volume inoculated).

Plaques on the plates were visually counted manually with a light box (Fig. 4). In the example in Fig. 2., the plaque titer calculation was as follows:

Fig. 2.

Scatterplots of the variance (n = 24; 3 analysts and 8 runs each) of the high concentration TA (Fig. 2A), and the low concentration TA (Fig. 2B). Each point in the scatterplots represents an assay variance estimate based on the titer results from 6 runs, performed by three analysts on two separate days in the legend, −1 represents Low, 0 represents middle and 1 represents the high range of each parameter tested.

For well B1: 49 × 4.50E+06 x (1000/150) = 1.47E+09.

For well C1: 33 × 1.35E+07 x (1000/150) = 2.97E+09.

For well A2: 11 × 4.05E+07 x (1000/150) = 2.97E+09.

Therefore, the final titer for the TA sample = Geomean of all countable wells (B1+C1+A2), which is 2.35E+09 PFU/mL. Titer calculations were based on all dilutions in which the observed plaques ranged from 5 to 55. The final titer was then determined by calculating the geometric mean of all titers from all dilutions that were within the acceptable range on one plate.

Assay acceptance criteria were defined as follows.

• a.For any well that has plaques that are either Too Numerous to Count (TNTC) (>55 plaques per well) or Too Few to Count (TFTC) (less than or equal to 4), that dilution was not to be used in the titer calculation.
• b.If at any time, a well was identified as not being acceptable for use, a N/A was documented for the specific well and the well eliminated from titer calculation. Possible examples of when a N/A would be applicable could be if an analyst recognized that they inoculated the same well twice or if they suspect that they failed to inoculate a well.
• c.A single titer per TA was calculated by determining the geometric mean of each final plate titer across all assays.
• d.Negative controls must remain negative for the test.
• e.The System Suitability Sample must fulfill the above acceptance criteria points a-c and must give a countable geometric titer to be able to considered the run as valid.
• f.A minimum of two wells must be counted within the range of 5–55 plaques/well.

TCID50 Assay Method. The TCID50 method was developed and validated for Phase I/Phase II clinical trial drug product release and stability potency measurements [8,23,24]. Briefly, 96 well plates were seeded with SK-MEL-28 cells at 1.0E+05 cells/well for approximately 24 h. The following day, after cells reached 70–90% confluency, they were infected with 100 μL of the serially diluted Test Articles (TAs) and appropriate positive (SSS) and negative (media only) controls. After 5 days, plates were evaluated for cytopathic effect (CPE) under a microscope. Wells were scored as positive if any part of the well showed CPE. The TCID50/mL value was calculated using the Karber equation.

Plaque Assay optimization protocol. The primary goal of this study was to investigate how assay variability was impacted by certain assay parameters of interest, and to identify the optimal setting where assay variability is minimized. Four parameters were chosen as being those which were most likely to contribute to the variability of the assay method. These included the cell seeding density, adsorption time, overlay volume and cell passage numbers (the specific levels studied for the parameter are shown in Table 1). An I-optimal response surface design was constructed to study the main effect, quadratic effect, as well as two-way interaction effects of the four parameters on assay variability. The entire set of the design conditions were tested with one low (1.0E+07 PFU/mL) and one high (1.0E+09 pfu/mL) concentration of the SSS. See details of the experimental design as well as more description of the statistical analysis in Supplemental Section A.1.

Table 1.

Method parameters tested in optimization study.

Parameters	Levels
	Low	Center	High
Cell seeding density (cells/well)	4.50E+05	5.00E+05	5.50E+05
Adsorption time (minutes)	90	105	120
Overlay volume (mL/well)	1.0	1.5	2.0
cell passage (passage number)	5	NA	33

Qualification protocol. Qualification testing was performed according to the plaque method, as described above (Methods - Plaque Assay, and Fig. 1.), after performing the method optimization protocol and setting the final assay method parameters. Representative CVA21 DS diluted in stabilization matrix buffer was used as the TA. The TA, diluted to provide five estimated target concentrations, plus one undiluted positive control (SSS) and one negative control (medium only) were evaluated (Table S1). The estimated target concentrations were calculated based on the dilution factor of DS into the formulation buffer. For the qualification design, two analysts each performed six runs, and each run consisted of 6 plates (Table S2). Five of the six plates were infected with CVA21 TA, and each plate was infected with two TA concentrations. The sixth plate was infected with SSS and negative control. The resulting data were used for assessment of linearity, accuracy, specificity, and intermediate precision. Linearity, accuracy, and precision were evaluated using the standard methods as recommended in USP chapter 1024 [24] (see Supplemental section A.2.). The SSS estimated value was used to set up the qualification protocol, and the actual concentration and acceptance criteria were established based on the data from this qualification.

Validation protocol. Like the method qualification protocol, the method validation protocol was performed using representative CVA21 DS in stabilization matrix buffer. The concentration range of DS dilutions in the validation protocol (5 concentrations) was wider than that used for the method qualification and encompassed the projected DP concentration. Thus, the validation plan performed with the range of DS concentrations represented DP as well. Due to the required (high) sample dilution of TA in the assay, slight variations of the sample matrix between the DS and DP were not deemed to be important for matrix interference considerations or concerns, as those components were sufficiently diluted out in the final assay. Thus, DS was representative of DP for plaque assay considerations. All targeted test article concentrations (Table S3) were prepared in the stabilization matrix buffer prior to testing and stored at −60 °C to −80 °C. As in the qualification protocol, a total of twelve runs were performed by two analysts. The study design was essentially the same as that for the qualification protocol, with the same purpose of evaluating linearity, accuracy, and precision of the assay. Validation testing was performed according to the established assay procedure which was set after method qualification (Methods - Plaque Assay, and Fig. 1.). All assay acceptance criteria needed to be met for data acceptance, including the SSS to be within the pre-determined range of 8.11E+09 PFU/mL to 2.6E+09 PFU/mL (note, the accurate SSS concentration value was determined during method qualification). Assay variability obtained from the qualification data was used to pre-define the validation acceptance criteria for accuracy, precision, and linearity.

Robustness Protocol. The robustness protocol was performed after the method was validated. Four critical method parameters were chosen to evaluate at a high, middle, and low level. These four parameters were cell passage number, methyl cellulose overlay volume, virus adsorption time and cell seeding density/well (Table 2.). A D-optimal design for the four parameters with ten random blocks was used, where the four parameters were varied from the middle point to assess the assay performance during routine testing. Three dilutions of one representative CVA21 DS in stabilizing matrix buffer served as the TA (Table S4). The design required two analysts to each perform five runs and each run consisted of six plates (Table S5). All assay acceptance criteria needed to be met for data acceptance, including the SSS to be within the pre-determined range of 8.11E+09 PFU/mL to 2.6E+09 PFU/mL. A %difference criterion that quantified the average titer changes across the low, middle and high levels of any of the critical method parameters was prespecified in the protocol. To demonstrate assay robustness, the calculated %difference result for each parameter needed to meet the corresponding acceptance criterion. Also, a mixed-effects response surface analysis was performed to evaluate the main and interaction effect of robustness parameters on titer response (see also Supplemental Section A.2.).

Table 2.

Robustness protocol parameters and levels tested.

Parameters	Levels
	Low	Center	High
Cell seeding density (cells/well)	4.0E+05	5.0E+05	6.0E+05
Adsorption time (minutes)	75	90	105
Overlay volume (mL/well)	1.25	1.50	1.75
cell passage (passage number)	6 $\pm$ 2	$25\pm$ 2	35 $\pm$ 2

3. Results

Initial method design and preliminary optimization. There are multiple parameters optimized to achieve the final robust and reproducible CVA21 plaque method that is described below (Fig. 1).

During development of the method, many parameters, reagents, and materials were compared and evaluated. To determine the most suitable host cells, SK-MEL-28, MRC5, ARPE-19, Vero, and HeLa cells were compared. SK-MEL-28 cells express the major CVA21 virus receptor Intercellular Cell Adhesion Molecule Type 1 (ICAM1) and the minor receptor Decay Accelerating Factor (DAF), and are thus very permissive for CVA21 infection [23]. SK-MEL-28 cells were superior for plaque formation compared to the other cell lines tested (unpublished data). To achieve robust assay performance, the plate format (6, 12 and 24 wells), choice of overlay (agarose, methyl cellulose), and overlay concentration and volume were evaluated. Furthermore, cell seeding density, virus volume and concentration, cell and virus adsorption times, and peak time for plaque formation were all optimized. The full set of parameters evaluated, and the preliminary values selected are listed in Table 3.

Table 3.

Conditions and reagents tested during assay method development.

No.	Parameter	All parameter values tested	Value selected to carry forward
1	Host cell	SK-MEL-28; MRC5; ARPE19; Vero and Hela	SK-MEL-28
2	Plate format	6 or 12 or 24 well plates	12 well plates
3	Overlay	Agarose; methyl cellulose; avicell	Methyl cellulose
4	Methyl cellulose concentration	0.7%, 1%, 1.25%, 1.5%	1%
5	Volume of 1% methyl cellulose overlay	1.0, 1.5, 2.0 mL per well	1.5 mL/well
6	Cell seeding density	1.0E+05, 2.0E+05, 2.5E+05, 3.0E+05, 3.5E+05, 4.0E+05, 4.5E+05, 5.0E+05, 5.5E+05, 6.0E+05 cells/well	5.0E+05 cells/well
7	Volume of virus at a fixed concentration	100, 150, 250, 500uL per well	150 μL per well
8	Cell + virus adsorption time	45, 60, 75, 90, 105, 120min	90–105 min
9	Infected cell incubation, post overlay addition	48, 60, 68, 72, 76, 96, 120hr at 37 °C, 5% CO2	70–74 h

Method optimization. During initial assay establishment, the CVA21 plaque assay was optimized by varying a single parameter while holding constant all other parameters (i.e., One Factor At a Time (OFAT)). As each parameter was selected, using experimental outcome data from multiple runs, that parameter was incorporated into the basic protocol. Thus, the selection of each parameter depended on the selection of previous optimized parameters. In this way, building the best set of assay conditions, the plaque assay was developed. After a working protocol was established, a set of four critical parameters was selected to evaluate in a more robust way. Specifically, an assay optimization study (Design Of Experiment (DOE)) was conducted with the purpose to gauge the effects of four critical parameters on assay variability and to establish the optimal setting at which assay variability was minimized. For this purpose, an I-optimal response surface design was constructed with the grouping factors being the four critical parameters. For each condition of this design, an assay variance estimate is obtained based on the titer results from 8 runs, performed by three analysts each performing the assay on two separate days, so that impact of the four parameters on assay variability could be evaluated. Scatterplots of assay variance results against the factors are shown in Fig. 2. The response surface model was fit to the assay variance results (log scale) and no interaction effects were significant, and no main effects were significant at the 0.05 significance level. Further, an optimal setting of the four parameters which leads to the minimal predicted variance based on the fitted response surface model was described. Specifically, for test article (TA) with high concentration (1.0E+09 pfu/mL), the assay adsorption time and cell passage needed to be set at the low levels (90 min, passage number 5, respectively) and the cell concentration and overlay set to the high levels (5.5E+05 cells/well, and 2.0 mL/well, respectively), and for TA with low concentrations (1.0E07 pfu/mL), the adsorption time, cell concentration and cell passage needed to be adjusted to the high level (120 min, 5.5E+05 cells/well, and passage 33, respectively) and overlay to the middle level (1.5mL/well).

Method qualification and validation. Following the DOE optimization study, the plaque assay method procedure was finalized as depicted in Fig. 1. We then qualified this method and the data generated during the method qualification (Fig. 3) was used to assess the method linearity, accuracy, repeatability, intermediate precision, and the system suitability value/criteria (Fig. 3 & Table 4).

Fig. 3.

Scatterplots of virus titer values (log10) against parameter levels for (A) 8.11E+09 PFU/mL TA concentration (high), (B) 5.17E+07 PFU/mL TA concentration (middle) and (C) 3.35E+05 PFU/mL TA concentration (low).

Table 4.

Qualification parameters and results.

Parameter	Results
Linearity	The slope and coefficient of determination ($R^2$) of the regression analysis based on the data from all 12 runs are both estimated to be 0.98. The % dilution bias based on the overall regression analysis (using all 12 runs) is the calculated to be −1.05%.
Accuracy	The average relative bias (%) for the test samples range from −0.43% to 23.79%.
Repeatability	The calculated overall %RSD for repeatability is 21.6%.
Intermediate Precision	The calculated overall %RSD for intermediate precision is 26.4%.
System suitability	Accepted plaque titer range of System Suitability Sample is determined to be (2.6E+09, 8.2E+09 PFU/mL). %RSD of three valid runs from this assay for any release and stability sample should not exceed 62%.

Based upon the qualification outcome, the following criteria for assay acceptance were derived.

• 1.The System Suitability Sample (SSS) titer must be within the range of 2.6E+09 to 8.2E+09 PFU/mL.
• 2.The SSS must be repeated on three plates within each test run. The SSS should have a % RSD no larger than 62%.
• 3.Within each test run, the individual titers (reportable results) for each TA must be within 0.3 log of their geomean. For Example: If a viral titer is reported as 1.0E+09 PFU/mL, the individual titers must be 1.0E+09 ± 0.3 log or within the range of 5.0E+08 to 2.0E+09 PFU/mL.

After successful qualification of the plaque method (Fig. 3), it was then validated. Similar to the qualification protocol, the validation protocol assessed the quality parameters including linearity, accuracy, repeatability, intermediate precision, specificity, and range. Unlike the qualification, with respect to the validation, parameter acceptance criteria were pre-specified. The pre-specified acceptance criteria and justification for each parameter were defined (Table 5).

Table 5.

Validation protocol pre-specified acceptance criteria and justifications.

Assay Performance Parameter	Definition	Assessment	Criterion	Justification
Specificity	Specificity refers to the capability of accurately identifying and measuring the target analyte in a sample, even in the presence of other components that are likely to be present	Negative for Negative Control (NC)	Lack of plaques in the negative control wells	Lack of plaques in the negative control indicates that no agent in the matrix is contributing to the formation of plaques in the test sample.
Accuracy	The accuracy of an analytical procedure expresses the closeness of agreement between the value which is accepted either as a conventional true value or an accepted reference value and the value found.	%Recovery	For each validation sample, the calculated overall % recovery based on the overall geometric mean of all the reportable titers for that sample should fall within 55%–175%.	This criterion is recommended based on the results from the CVA21 qualification study and aligns with the acceptance criterion for linearity.
Intermediate Precision	The precision of an analytical procedure expresses the closeness of agreement between a series of measurements obtained from multiple sampling of the same homogeneous sample under the prescribed conditions.	Intermediate Precision %RSD	Intermediate precision %RSD should be no larger than 40%.	This criterion is recommended based on the results from the CVA21 qualification study and along with the acceptance criterion for accuracy ensures the assay can routinely pass acceptable lots even if it is performing at the indicated levels of bias and variability.
Repeatability		Repeatability %RSD	Repeatability %RSD is no larger than 30%.	This criterion is recommended based on the results from the CVA21 qualification study.
Linearity	Linearity is the ability of the assay to return values that are directly proportional to the concentration of the target pathogen or analyte in the sample.	% Dilution bias	The absolute value of % dilution bias based on the overall regression analysis (using all reportable results) is no larger than 5% per two-fold dilution.	This criterion is recommended based on the results from the CVA21 qualification study.
Range	The range of an analytical procedure is the interval between the upper and lower concentration (amounts) of analyte in the sample (including these concentrations) for which it has been demonstrated that the analytical procedure has a suitable level of precision, accuracy and linearity.	Plaque titer range with acceptable relative accuracy, precision, and linearity	N/A	Based on the results for linearity, accuracy, and intermediate precision, the range of the assay will be reported.

The acceptance criteria were based largely on what was learned about the assay from the assay qualification study. Results of the validation protocol are shown in Table 6. Each pre-specified acceptance criterium was met for the protocol, thus the validation was successful.

Table 6.

Validation protocol results.

Assay Performance Parameter	Criterion	Results
Linearity	The absolute value of % dilution bias based on the overall regression analysis (using all reportable results) is no larger than 5% per two-fold dilution.	The % dilution bias based on overall regression analysis (using all reportable results) was 0.768% per two-fold dilution.
Accuracy	For each validation sample, the calculated overall % recovery based on the overall geometric mean of all the reportable titers for that sample should fall within 55%–175%.	The calculated overall % recovery for validation sample ranged from 87.6% to 99.8%.
Intermediate Precision	Intermediate precision %RSD should be no larger than 40%.	Intermediate precision %RSD (for all validation samples) was estimated to be 32%.
Repeatability	Repeatability %RSD is no larger than 30%.	Repeatability %RSD (for all validation samples) was estimated to be 22%.
Specificity	Lack of plaques in the negative control wells.	Lack of plaques in the negative control samples tested in this validation study indicated that no agent in the matrix contributes to the formation of plaques in the test sample. The assay was therefore demonstrated to be specific for the virus in the test sample, CVA21.
Range	N/A	Based on the results for linearity, accuracy, and intermediate precision, the range of this assay was determined to be (3.31E+06, 8.11E+09 PFU/mL).

Analysis for Robustness Study. The final step to the assay method was robustness testing. The goal of robustness testing is to demonstrate that the assay method is resistant to small but deliberate perturbations of the assay method parameters, which might be experienced during routine performance of the assay method. A total of 100 data points, 30 from each of the three test samples and 10 from the SSS, were obtained by 2 analysts over 5 days (Fig. 3). A mixed-effects response surface model was fit to the data which allowed the evaluation of all the main effects and all two-way interaction effects of the studied parameters. For the test sample at target titer level 8.11E+09 PFU/mL, the only significant main effect was that of cell passage. This result corroborated the data in the scatterplot in Fig. 3, where it can be seen that higher titer values corresponded to higher cell passage number (Fig. 3A). For the test sample at target titer level 5.17E+07 PFU/mL, no main effect was significant (Fig. 3B). At the target titer level 3.30E+05 PFU/mL, both cell passage and overlay had significant main effects (Fig. 3C). The interaction between cell passage and cell density and the interaction between overlay and adsorption time were significant. Even though two of the parameters had a significant impact on the titer response, for each sample, the calculated % difference results for all samples were below the threshold of the acceptance criteria. Therefore, the robustness criteria were met.

As an additional evaluation, the assay performance including linearity, accuracy and precision, were also evaluated with the goal to demonstrate that even with the presence of the deliberate perturbations of the assay method parameters, acceptable assay performance was still achieved. In fact, comparable assay performance was demonstrated in the robustness study as in the qualification and validation studies assays.

Description for the statistical methods used to derive the results for the above four studies are included in the supplemental material.

Summary of parameter values and variability across protocols. Similar results were obtained between the qualification and validation protocols performed during CVA21 plaque method development. Slope and R² for plots of expected versus observed mean titers were similarly high for the two protocols, demonstrating very good assay function (Table 4, Table 5, Table 6). In each of the studies specificity was demonstrated (no plaques in media alone). The testing concentration range was increased with each protocol. This served to keep pushing the assay range limits, to return potency determinations for both high concentration TA and low concentration TA expected for CVA21 virus seeds, and DS and DP concentrations. The method range determined during qualification was 1.58E+07 to 1.58E+09 PFU/mL, in the validation protocol the range validated was 3.31E+06 to 8.11E+09 PFU/mL and in the robustness protocol the range assessed was 3.30E+05 to 8.11E+09 PFU/mL.

Comparison of the TCID50 and Plaque potency methods. It was of interest to compare the performance of the plaque method to the existing CVA21 TCID50 potency method. Head-to-head comparison between the two methods was not performed, however various attributes of the method performance between these two methods were compared to determine if there were significant operational differences (Fig. 5). Historical data from the TCID50 method was compared to the plaque data generated during method validation. The TCID50 qualification data demonstrated the following assay characteristics; linearity (slope = 1.003, and R² values = 0.973) Intermediate Precision (58% RSD), Repeatability (%RSD) 52%, accuracy (% Recovery) 57.2–167% and specificity (no cytopathic effect (CPE) in negative control wells). While linearity and accuracy comparable between two assay the precision is less for the plaque assay (repeatability = 22% and intermediate precision = 32% from the validation protocol) than TCID50 assay (56.6%), demonstrating greater variability in the TCID50 assay. The TCID50 was linear across 4 logs (2.19E+03 to 2.19E+07 TCID50/mL) (Fig. 5A), while the plaque assay was also linear across 4 logs (3.30E+05 to 8.11E+09 PFU/mL, assessed in the Robustness protocol) (Fig. 5B). With respect to assay operations, the turnaround times for the TCID50 and plaque were seven and five days respectively, whereas a weekly throughput by a single analyst was 3 and 18 TA by TCID50 and plaque methods respectively. Overall, the accuracy, precision and linearity for both methods are within expectations for cell-based assays the plaque assay demonstrated similar or better performance compared to TCID50 (Fig. 5). In addition, for assay throughput and duration, the plaque assay was also superior (Table 7).

Fig. 5.

Validation curves of both assays. (A) TCID50 Assay, (B) Plaque Assay data from Validation.

Table 7.

Comparing the performance of two assays.

Assay Parameter	Results
	TCID50	Plaque
Linearity	R2 = 0.973; Slope = 1.003	R2 = 0.995; Slope = 1.011
Accuracy (% recovery)	57.2%–167.6%	87.6%–99.8%
Intermediate Precision (%RSD)	58%	32%
Repeatability (%RSD)	51%	22%
Specificity	No cytopathic effect (CPE) in Negative Controls	No plaques in Negative Controls
Range	2.19E+03 to 2.19E+07 TCID50/mL	3.31E+06 to 8.11E+09 pfu/mL
Assay duration	7 days	4 days
Throughput by analyst	3 samples/week	22 samples/week

4. Discussion

Research and development of oncolytic viruses is a nascent but promising field with potential for providing a new approach to treat certain types of cancer. Health authorities around the world have begun to consider regulatory guidance for these products, but the number of guidance documents specific to OV is currently limited. Licensure depends on guidance developed for gene therapy products, which are not typically vectored by oncolytic viruses. In addition to purity and identity, a key measure of the virus is its titer, or potency. The drug potency value is necessary for determining dose, product release and stability, and lot to lot consistency [22]. Potency measured in PFU/mL is also a component of the (total particles versus the infectious particles) P:I ratio, a value which reflects the specific activity of the drug product. The P:I ratio indicates the fraction of virus particles which are delivered but do not contribute to the MOA. The noninfectious particles may contribute to an undesired inflammatory response, thus there is the goal to reduce this ratio in the final DP. The plaque potency assay serves as one component of the “matrix” of assays requested by the FDA, for complete characterization of the potency of a gene therapy product. In this manuscript, we have described the establishment of a plaque assay to support late phase clinical development of the CVA21 OV. The development steps, from initial optimization to final validation and robustness testing, can serve as a guide to assay development methodology. Importantly, during performance of multiple method evaluation steps, preliminary data can be generated to obtain an understanding of assay performance, thorough SSS potency tracking, and data generation for inclusion in regulatory filings.

The plaque assay is a classic method in virology and has multiple reagents and steps which should be optimized for individual species of virus. However, it brings lot of challenges (low throughput, reproducibility, range etc.) to make it as a regulatory compliance method for phase-III product release. After implementation of each of the steps described herein, the expectation is that the assay method will be long-lived, highly reliable and appropriate for commercial purposes. The thorough optimization process generates a method with a high degree of reproducibility and robustness. Importantly, a stable, reliable assay method is necessary for managing assay transfer to other countries for import testing purposes. There are several advantages of the plaque assay over other cell-based potency assays, and other relative potency assays (e.g., those measuring virus infectivity instead of infectious particles directly). The plaque assay is well understood by regulatory agencies around the world; it is straightforward to perform; there are no sophisticated high-tech instruments needed; and throughput can be easily increased by training additional analysts. The assay output is the direct measurement of infectious virus particles; therefore, no reference standard is necessary. The plaque assay is also a model for the MOA of the OV, an FDA requirement for potency assays. The MOA steps include that CVA21 virus bind to specific cell surface receptors (ICAM-1), invade the cells, and replicate intracellularly, to the point of cell lysis.

In the initial design and preliminary optimization of the method, each assay variable or parameter was generally adjusted individually, to obtain favorable data (i.e., optimal data generated from a representative sample). A limited number of analysts performed the method during that development stage. In a methodical way, values for the best preliminary set of assay parameters were narrowed using OFAT approaches. This process allowed analysts practical experience, and trouble-shooting practice with the assay components. It was unknown whether varying one parameter had either a direct or indirect impact upon other parameters concurrently and, as such, it was not known if the assay was actually optimal after OFAT. Therefore, the next development step explored assay parameters concurrently, using the DOE approach. In the DOE, or optimization study, multiple parameters were varied simultaneously [25,26]. Parameters selected for investigation were those considered to have a high impact on assay performance, as determined during the initial OFAT. The ultimate goal for this assay method was assay validation for late phase release and stability protocols. Virus and cell substrate compatibility with buffers, plate format and overlay material were optimized. Special attention was paid to understand and select the best cell substrate. Selection of receptor positive, permissive SK-MEL-28 cells allowed virus cytopathic effect to be easily observed, counted, and interpreted. Operator subjectivity for plaque enumeration can be of concern. Use of multiple operators and rules for positive plaque identification serve to reduce subjectivity of the data analysis. The DOE optimization study was designed to investigate potential interactions between several key assay parameters. These were chosen as factors which might have the most impact (e.g., adsorption time, overlay volume, and cell density), or might be of question to regulators (e.g., cell passage number). A mixed-effects surface design analysis was used to determine parameter interactions, parameter effect on assay output (i.e., titer), and the best parameter values to reduce assay variability. While there was no significant effect of parameter interaction on assay variability, several of the parameters did significantly affect the assay data (i.e., titer), and statistical analysis projected the best set of values necessary to reduce titer variability.

Analysis of the DOE data yielded optimal parameter values which, taken together, maximally reduced assay variability, and thus the method parameters were considered optimized. This is the point at which a more thorough examination of assay performance was designed and executed, that is, assay qualification. Assay qualification involved examination of assay accuracy, precision (repeatability), intermediate precision, linearity, and range, and involved multiple analysts across multiple days. Acceptance criteria were not defined prior to execution of the qualification protocol. Resulting data were used to define the assay variability and the assay system suitability sample. The assay precision, accuracy, and linearity statistics obtained from the qualification protocol facilitated the determination of acceptance criteria for a validation protocol. The validation protocol specified and justified pre-defined assay acceptance criteria for accuracy, precision, and linearity. If the pre-defined criteria were met after the validation protocol was completed, the assay was valid for measuring the potency of TA including DS, DP and virus seeds. After assay qualification was achieved, the assay could then be used for release of clinical lots for early phase clinical development. Qualification and validation resulted in similar data which indicated the assay had excellent linearity, range, specificity, with good precision and accuracy. When compared to the CVA21 TCID50 historical data, the plaque assay had better precision. The plaque assay was also operationally better than the TCID50 due to the possibility of higher assay throughput and decreased turn-around time. The successful robustness protocol served to further confirm assay readiness for Phase III testing. The robustness data was evaluated using an approach similar to the DOE analysis, that is, the surface design analysis. Data analysis indicated that small but deliberate perturbations to four critical attributes of the assay, did not negatively impact the method. This indicates that the assay, when performed under routine GMP testing, could absorb minor assay variation due to e.g., pipetting variability.

In summary, a potency assay was developed for CVA21. The steps used to optimize and validate the assay method were described, as well as the statistical methods for data analysis. This step-by-step guide can serve as a strategy for others considering assay method development for infectious viral products.

CRediT authorship contribution statement

Venkateswarlu Chamcha: Writing – review & editing, Writing – original draft, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. Li He: Investigation, Data curation. Jenny Xu: Investigation, Data curation. Andrew Swartz: Writing – review & editing, Investigation, Conceptualization. Erin Green-Trexler: Investigation. Kevin Gurney: Writing – review & editing, Supervision. Tessie McNeely: Writing – review & editing, Supervision, Investigation.

Declaration of generative AI and AI-assisted technologies in the writing process

The authors have not used generative AI or AI-assisted technologies in the writing process.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

The following is the Supplementary data to this article.