The Prioritization of Cancer Antigens: A National Cancer Institute Pilot Project for the Acceleration of Translational Research

Abstract

The purpose of the National Cancer Institute (NCI) pilot project to prioritize cancer antigens was to develop a well-vetted ranked prioritized list of cancer vaccine target antigens based on pre-defined and pre-weighted objective criteria. An additional aim was for the NCI to test a new approach for prioritizing translational research opportunities based on an Analytic Hierarchy Process for dealing with complex decisions. Antigen prioritization involved developing a list of “ideal” cancer antigen criteria/characteristics, assigning relative weights to those criteria using pair-wise comparisons, selecting 75 representative antigens for comparison and ranking, assembling information on the pre-defined criteria for the selected antigens, and ranking the antigens based on the pre-defined, pre-weighted criteria. Using the pair-wise approach, the result of criteria weighting, in descending order was: (1) Therapeutic function, (2) Immunogenicity, (3) Role of the antigen in oncogenicity, (4) Specificity, (5) Expression level and percent of antigen positive cells, (6) Stem cell expression, (7) Number of patients with antigen positive cancers, (8) Number of antigenic epitopes and (9) Cellular location of antigen expression. None of the 75 antigens had all of the characteristics of the “ideal” cancer antigen. However, 46 were immunogenic in clinical trials and 20 of them had suggestive clinical efficacy in the “Therapeutic function” category. These findings reflect the current status of the cancer vaccine field, highlight the possibility that additional organized efforts and funding would accelerate the development of therapeutically effective cancer vaccines, and accentuate the need for prioritization.

Introduction

Virtually any mutant, over- or abnormally-expressed protein in cancer cells can serve as a target for cancer vaccines and/or T cell therapy [1–75]. Scores of cancer vaccines have been shown to be immunogenic in clinical trials, and many of them have demonstrated efficacy in at least small numbers of patients. No cancer vaccine has yet been approved by the FDA, despite extensive developmental efforts by academia and industry. Nevertheless, there is consensus that optimally designed cancer vaccine trials combining the best antigens with the most effective immunotherapy agents might yield positive clinical results.

Cancer vaccine development is limited by several factors, including funding constraints. Limited resources mandate transparent methods to prioritize developmental opportunities with the least possible bias. An NCI immunotherapy agent workshop was held in July 2007 to rank agents with high potential to serve as immunotherapeutic drugs¹. The ranking was based on the likelihood for efficacy in cancer therapy and was exceedingly well-vetted, with broad and substantial input from academia, industry and the government. Many of the ranked immunotherapeutic agents have been shown to be effective as components of cancer vaccine regimens in pre-clinical models, but this abundance of promising opportunities raises immediate questions as to which antigen or sets of antigens is/are most appropriate for co-development. Our current effort to prioritize cancer antigens represents the logical next step in attempting to focus translational efforts on cancer vaccine regimens with the highest potential for success.

The task of ranking cancer antigens is immense, and the number of potential cancer antigens almost limitless. At present, investigator-initiated funding of science dictates innovation, i.e., that each investigator discovers and develops his/her own antigens. This leads to an ever increasing number of potential vaccine targets as well as validation of those targets through pre-clinical and early clinical cancer vaccine development. Few investigators have both the financial and organizational resources to advance their vaccines past early developmental stages.

The NCI, recognizing the untapped potential of therapeutic cancer vaccines as well as many other novel therapies, embarked on a new approach to the identification, prioritization, and funding of translational cancer research based on recommendations of the Translational Research Working Group (TRWG) ². The primary objective is to identify specific translational cancer research projects that warrant a dedicated effort to accelerate progress through focused collaborations. This process requires a mechanism for identifying high priority translational research projects based on scientific validity, clinical need, and technical feasibility. The NCI’s initial endeavor to implement the TRWG’s recommendation for prioritization of translational opportunities has focused on evaluation of a method to select cancer antigens for subsequent development through the Immune Response Modifier (IRM) Pathway, one of the six TRWG Pathways leading from fundamental laboratory discoveries to definitive testing in clinical trials [76, 77].

The IRM Pathway was selected as the pilot effort for several reasons. It is the most complex of the TRWG Pathways and successful application of a prioritization process in this context is expected to be generalizable to other TRWG pathways. In addition, the immunology community had already prioritized immunotherapy agents at the NCI Immunotherapy Agent Workshop³, an experience that greatly facilitated implementation of this pilot project.

The methodology for prioritization of cancer antigens was based on the Analytic Hierarchy Process (AHP), a structured technique and mathematical model for dealing with complex decisions. AHP has been refined since its initial description by Thomas L. Saaty in the 1970’s [78], and has been used throughout the world in a wide variety of decision settings spanning government, business, industry, healthcare, and education. AHP is considered most useful to teams contending with complex problems that involve human perception and judgment [79]. The process breaks down a complex problem into a hierarchy of sub-problems that can be compared to each other on a pair-wise basis. It has unique advantages where major decision elements are difficult to quantify or compare, or where communication among team members is impeded by their different specializations, terminologies, or perspectives. For the current project, criteria for cancer vaccines were determined. The criteria were then broken down into sub-criteria for greater granularity within each higher level criterion. A panel of cancer vaccine experts used pair-wise comparisons to weight first the criteria, then the sub-criteria within the criteria. The AHP converted the weighted criteria into numerical values that could be analyzed and compared for the ranking of antigens and to permit the comparison of rankings based on hypothetical alternative weightings.

The AHP generated primary and alternative priority rankings of 75 cancer antigens on the basis of criteria pre-identified and weighted by a broadly constituted panel of cancer vaccine experts. These rankings are dynamic, given that priorities change as knowledge accrues from new studies. The associated lists of weighted criteria inform investigators as to what experimental evidence is required to advance antigens to higher priority levels. Above all, the rankings provide a basis for deciding which antigens are most likely to pay off on investments to generate cancer vaccines for testing in later-stage clinical trials.

Materials and Methods

Decision Lens, Inc. provided the AHP methodology as a web-based tool with four modules⁴. The first phase of the process focused on identifying the participants, criteria, and alternatives to be prioritized. In the second phase, criteria essential to the decision were identified, grouped, compared, and weighted using the Build Model and Compare Criteria modules. The third phase focused on the Evaluate Alternatives module, wherein alternatives (antigens) were compared to each of the weighted criteria to determine their benefit or value using customized rating scales. The Reporting module provided a flexible tool for the analysis of information to facilitate informed decision-making.

Phase I: Decision Preparation

The key objective of the decision preparation phase was to gather the critical data needed to make the decision and to define expectations for key participants regarding the decision process. There were three distinct steps to the process.

The first step was to determine who would be participating in the prioritization process. The NCI selected investigators who participated in the Immunotherapy Agent Workshop. The Workshop participants had been selected on the basis of recommendations from the American Association for Cancer Research (AACR), American Association of Immunologists (AAI), American Society of Clinical Oncology (ASCO), American Society of Hematology (ASH), Cancer Vaccine Consortium (CVC), International Society for Biological Therapy of Cancer (iSBTc), and NCI intramural and extramural program staff. Experts from this group were used to contribute to the criteria determination, weighting, and evaluation steps of the process (List of participants available as Supplements A, B and C).

The 19 investigators listed in Supplement A provided the criteria used to evaluate cancer antigens. Top-down and bottom-up approaches were used. For the top-down approach, approximately half of the experts were asked to submit via e-mail what they regarded to be characteristics of an “ideal” cancer antigen. For the bottom-up approach, the remaining experts were asked which characteristics made the following antigens good or poor candidates for therapeutic development: (1) mutated segment of p53, (2) MUC1, (3) MAGE-A3, (4) HER-2/neu, (5) gp100, and (6) mutated proteins unique to each patient. The two lists were vetted, combined and structured into a list of criteria and sub-criteria. Using the same information source, definitions for each criterion and sub-criterion were developed. The final criteria and definitions are shown in Table 1, columns 1 and 2.

TABLE 1.

Cancer antigen pilot prioritization: criteria & sub-criteria definitions & weightings

THERAPEUTIC FUNCTION (Weight of Criteria - 0.32)
Sub-criteria	Definition	Weight of Sub-criteria
Controlled vaccine trial suggestive Data ranked as being superb, very strong, adequate, or fair.	Clinical trial data showing that a vaccine induced clinical responses in at least a small number of patients, or provided suggestive evidence of benefit vs. controls
Superb Data Controlled vaccine trial suggestive		100.0% (1.0)
Very Strong Data Controlled vaccine trial suggestive		93.0% (0.93)
Adequate Data Controlled vaccine trial suggestive		85.0% (0.85)
Fair Data Controlled vaccine trial suggestive		75.0% (0.75)
Responses in T cell therapy		65.0% (0.65)
Pre existent immunity/survival correlation		15.0% (0.15)
Positive appropriate animal models		10.0% (0.1)
Not applicable (N/A)		0.0% (0.0)
IMMUNOGENICITY (Weight of Criteria - 0.17)
Immunogenic in clinical trials	T cell and/or Ab responses elicited in clinical trials	100.0% (1.0)
T cell immunity observed	Spontaneous T cell responses observed in some patients	39.0% (0.39)
Immunogenic in appropriate animal models	Immunogenic in animal models with natural levels of antigen expression similar to humans	11.0% (0.11)
Ab immunity observed	Spontaneous Ab observed in some patients	10.0% (0.1)
Not applicable (N/A)		0.0% (0.0)
ONCOGENICITY (Weight of Criteria - 0.15)
Oncogenic “self” protein	Associated with oncogenic process, i.e., oncogenic “self” protein	100.0% (1.0)
Persistent viral Ag	Persistently expressed viral antigen	34.0% (0.34)
Function uncertain, correlated to decreased survival	Uncertain function, but increased expression correlated with decreased survival and/or more aggressive or advanced disease	25.0% (0.25)
Tissue differentiation, not oncogenic	Associated with tissue differentiation, but not “oncogenic”	12.0% (0.12)
Tumor-related stroma	Expression on tumor-related stroma, but not on malignant cells	12.0% (0.12)
Not applicable (N/A)		0.0% (0.0)
SPECIFICITY (Weight of Criteria - 0.15)
Absolute specificity	Absolutely specific, e.g., mutated oncogene, idiotype protein or viral protein	100.0% (1.0)
Onco-fetal antigen	Antigens expressed in fetus with no or little expression in adult tissue. Includes cancer testis antigens	54.0% (0.54)
Over expressed in cancer	Over expressed in cancer, but expressed in some normal adult tissues	35.0% (0.35)
Abnormal post-translational modification	Core protein expressed in normal tissue, but expressed in cancer with unique post-translational changes, e.g. glycosylation or phosphorylation	23.0% (0.23)
Tissue specific (expendable tissue)	Tissue specific expression in normal adult tissue relatively expendable for survival, e.g., prostate and melanocytes	21.0% (0.21)
Unique random mutations	Unique random mutations specific to each patient	10.0% (0.1)
Tumor stroma antigen	Normal antigen expressed on tumor stroma	10.0% (0.1)
Not applicable (N/A)		0.0% (0.0)
EXPRESSION LEVEL & % POSITIVE CELLS (Weight of Criteria - 0.07)
High level, All cancer cells	Highly expressed on all cancer cells in patients designated for treatment	100.0% (1.0)
High level, Most cancer cells	Highly expressed on most cancer cells in patients designated for treatment	37.0% (0.37)
Lower level, All cancer cells	Lower level of expression on all cancer cells in patients designated for treatment	23.0% (0.23)
Lower level, Most cancer cells	Lower level of expression on most cancer cells in patients designated for treatment	8.0% (0.08)
Not applicable (N/A)		0.0% (0.0)
STEM CELL EXPRESSION (Weight of Criteria - 0.05)
Stem cell expression, presumptive	Evidence for expression on putative cancer stem cells	100.0% (1.0)
No info concerning stem cells - but on all stages from premalignant to metastatic	Present at all stages of tumor development, from premalignant to metastatic cancer cells, but without information about putative stem cells	66.0% (0.66)
No info concerning stem cells, but on most cancer cells	Expression on the all or most cancer cells, but without information about putative stem cells	20.0% (0.2)
Not applicable (N/A)		0.0% (0.0)
NUMBER OF PATIENTS WITH ANTIGEN POSITIVE CANCERS (Weight of Criteria - 0.04)
Many patients, high level	High level of expression in many patients with a particular tumor type	100.0% (1.0)
Many patients, lower level	Low level of expression in many patients with a particular tumor type	16.0% (0.16)
All patients/ unique antigens	Unique antigens from random mutations presumed to be present in all patients	14.0% (0.14)
Few patients, high level	High level of expression in a small subset of patients with a particular tumor type	11.0% (0.11)
Not applicable (N/A)		0.0% (0.0)
NUMBER OF EPITOPES (Weight of Criteria - 0.04)
Longer antigen	Longer antigen with multiple epitopes and the potential to bind to most MHC molecules	100.0% (1.0)
Short antigenic segment	Short antigenic segment with one or few epitopes and the potential to bind to only selected MHC molecules	13.0% (0.13)
CELLULAR LOCATION OF EXPRESSION (Weight of Criteria - 0.02)
Cell surface expression; no or little circulating Ag	Normally expressed on the cell surface with no or little circulating antigen	100.0% (1.0)
Internal with MHC presentation	Internal only with MHC presentation	95.0% (0.95)
Cell surface expression; and circulating Ag	Normally expressed on the cell surface with substantial circulating antigen	25.0% (0.25)
Not applicable (N/A)		0.0% (0.0)

The cancer antigens to be prioritized were determined through a search of the PubMed database over the last 5 years using the terms “cancer vaccine target”. One hundred of the most frequently mentioned antigens were selected and submitted to the participating experts for categorization according to the pre-defined criteria and sub-criteria. Eighty investigators (listed in Supplement B) with expertise in one or several of the cancer antigens were asked to categorize the one or several antigens according to the criteria and sub-criteria. These experts were typically corresponding authors on published articles concerning the specific antigens. In certain cases, when necessary and where appropriate, experts not directly involved with the particular antigen were asked to categorize select antigens based on the pre-defined criteria. For some antigens, several experts were asked to categorize the antigen. A few experts did not respond and certain antigens were no longer under development. In the final analysis, 75 antigens were scored according to the pre-defined criteria.

Differences in scoring were debated and voted on at the face-to-face “assessment of alternatives” meeting described below. An example of the Antigen Information Form sent to the antigen experts is provided in Supplement D.

Phase II: Criteria refinement and weighting

The criteria and sub-criteria were used as the basis for discussion during a web-facilitated remote meeting using the Decision Lens model. They were discussed and definitions refined based on the combined expertise of the 19 expert participants (Supplement A). The criteria were then compared in a pair-wise fashion to determine the experts’ cumulative judgment’ of their relative importance to each other. The relative importance of each criterion to each of the other criterion was voted on by each expert and the relative importance of each was given a numerical rating on a scale from minus 9 to plus 9. The sub-criteria within each criterion were then compared in a similar pair-wise fashion by the same process.

Each expert participant had a single vote of equal weight. Participants who were unable to complete their pair-wise comparisons during the facilitated meeting were able to complete the process on line at a later date. Thirty-six pair-wise comparisons were employed to assess the relative priority of the 9 criteria. Similar pair-wise comparisons of sub-criterion within each criterion were determined to generate the relative weight of each sub-criterion to other sub-criterion. Sub-criteria were compared only to sub-criteria within their parent criteria. The cumulative results of the ratings of all of the experts were converted to a set of priority ratios for the criteria and sub-criteria. The results were nonlinear in their value differences.

Phase III: Assessment of alternatives

The weighted criteria and sub-criteria, which were used as rating scales, were used to assess the relative priority of each of the 75 cancer antigens at a face-to-face meeting of 16 participants that was hosted by the NCI (Supplement C). The information provided by up to 3 experts (Supplement B) per antigen on the Antigen Information Sheet (Supplement D) was entered in the Decision Lens software tool. The sub-criteria/rating scales were ordered from highest to lowest weight, but information on the relative weights of each criterion and sub-criterion was shared with participants only after the evaluation was completed. Each antigen was assigned to a meeting participant who acted as a reviewer and led the discussion of that antigen.

Each antigen was categorized according to the criteria and sub-criteria. If an antigen fulfilled more than one sub-criterion within a criteria, the sub-criteria with the highest value was selected. If a difference of opinion among participants was noted, it was discussed and then voted upon. Often consensus was not reached. When consensus was not reached the votes ended up with a value between the two sub-criteria. The value scores were calculated by taking the average of the ratings, then multiplying it by the weight of the criterion to cumulate to an aggregate score. The participants voted using a radio-frequency keypad and each vote had equal weight. Participants who were unable to complete antigen prioritization during the facilitated meeting were able to complete the process on line at a later date.

Results

Weighting of Criteria

The AHP pair-wise comparison process resulted in a weighted model where the criteria relative weights reflect the derived priorities of the group of participants (Figure 1 and Table 1). The numerical values reflect the relative priorities of each criterion. As an example, pair-wise comparisons of criteria determined that “Therapeutic Function” represented 32% of the weight and “Immunogenicity” represented 17% of the weight, whereas “Cellular Location of Expression” represented only 2% of the weight. Thus, “Therapeutic Function” was deemed to be approximately twice as important as “Immunogenicity” and approximately 16-times more important than the “Cellular Location of Expression”.

Figure 1.

Criteria for an idea cancer antigen were weighted by pair-wise comparison and the resulting relative weights are indicated. Therapeutic function was considered the most important criteria, and was more than twice (0.32/0.15) as important as specificity or oncogenicity.

In some cases, there was considerable variation in response during the pair-wise comparison process. The participants were asked to explain their positions so that their implicit knowledge could become explicit and possibly result in readjustment of votes. However, the final weighting did not require and often did not achieve consensus.

Weighting of Sub-Criteria/rating scales

The sub-criteria were similarly weighted by pair-wise comparisons. Weighting is presented in Table 1. The sub-criteria, which served as the rating scales for each criterion, are also nonlinear. The top sub-criterion for each antigen received full value for the criterion. Other sub-criteria received less value for the criteria with the level dependent upon the pre-determined weighting. For example, for the criterion “Specificity”, an antigen deemed to have “Absolutely Specificity” received 100% of the value for that criterion, whereas an antigen that was “Over-expressed in Cancer” as the highest ranking within this category only received 35% of that value. The experts agreed that top sub-criterion for each criterion approximately portrayed an “Ideal Cancer Antigen” (Table 2).

TABLE 2.

Characteristics of an “ideal” cancer antigen

CRITERIA	TOP SUB-CRITERIA
THERAPEUTIC FUNCTION	Superb Data Controlled vaccine trial suggestive
IMMUNOGENICITY	T cell and/or Ab responses elicited in clinical trials
ONCOGENICITY	Associated with oncogenic process, i.e., oncogenic “self” protein
SPECIFICITY	Absolutely specific, e.g., mutated oncogene, idiotype protein or viral protein
EXPRESSION LEVEL & % POSITIVE CELLS	Highly expressed on all cancer cells in patients designated for treatment
STEM CELL EXPRESSION	Evidence for expression on putative cancer stem cells
NUMBER OF PATIENTS WITH ANTIGEN POSITIVE CANCERS	High level of expression in many patients with a particular tumor type
NUMBER OF EPITOPES	Longer antigen with multiple epitopes and the potential to bind to most MHC molecules
CELLULAR LOCATION OF EXPRESSION	Normally expressed on the cell surface with no or little circulating antigen

The criterion “Therapeutic Function” carried the most weight in the prioritization process. This category also generated substantial debate regarding the assessment of available information. The basis of the criterion was defined as “clinical trial data showing that a vaccine induced clinical responses in at least a small number of patients, or provided suggestive evidence of benefit vs controls”. The quality of published or publicly reported data was often disputed by the panel members. In anticipation of this discussion, the sub-category “Controlled vaccine trials suggestive” was subdivided before the meeting into the following sub-criteria:

1. Superb data suggesting therapeutic benefit in a controlled vaccine trial,

2. Very strong data suggesting therapeutic benefit in a controlled vaccine trial,

3. Adequate data suggesting therapeutic benefit in a controlled vaccine trial,

4. Fair data suggesting therapeutic benefit in a controlled vaccine trial.

   Although subjective, these four sub-criteria parallel the evaluation process commonly used to assess NIH grant applications and emphasized the need for expert evaluation at all stages of the process. The other sub-criteria within the criterion of “Therapeutic Function” were

5. Responses in T cell therapy trial,

6. Pre existent immunity/survival correlation, and

7. Positive data in appropriate animal models.

The results of the evaluation and weighting of the 75 cancer antigens are presented in Table 3. The results presented in Figure 2 show the cumulative score for each antigen. The color-coded bars indicate the relative contribution of each criterion.

TABLE 3.

Cancer antigen pilot prioritization: ranking based on predefined & preweighted criteria

CRITERIA►	CUMMU- LATIVE SCORE	THERAPEUTIC FUNCTION (0.32)	IMMUNO- GENICITY (0.17)	ONCO- GENICITY (0.15)	SPECIFICITY (0.15)	EXPRES- SION LEVEL & % POSITIVE CELLS (0.07)	STEM CELL EXPRES- SION (0.05)	NUMBER OF PATIENTS WITH ANTIGEN POSITIVE CANCERS (0.04)	NUMBER OF EPITOPES (0.04)	CELLULAR LOCATION OF EXPRES- SION (0.02)
ANTIGENS ▼ (Rank/ Reference number & Name)
1. WT1	0.81	0.75 Fair data	1.0 Imm in trials	1.0 Oncogenic	0.54 Oncofetal	0.37 High most	1.0 Stem cells	1.0 Many patients high level	1.0 Multiple	0.95 Internal MHC
2. MUC1	0.79	0.75 Fair data	1.0 Imm in trials	1.0 Oncogenic	0.23 Post- translational changes	1.0 High all	1.0 Stem cells	1.0 Many patients high level	1.0 Multiple	0.25 Surface Circulating
3. LMP2	0.78	0.75 Fair data	1.0 Imm in trials	0.34 Persistent viral	1.0 Absolute	0.37 High most	1.0 Stem cells	1.0 Many patients high level	1.0 Multiple	0.95 Internal MHC
4. HPV E6 E7	0.77	0.89 Very Strong-8 Adequate-6 Fair-1	1.0 Imm in trials	0.34 Persistent viral	1.0 Absolute	0.23 Low all	0.73 All stages-11 Stem cells-3	0.16 Many patients lower level	1.0 Multiple	0.95 Internal MHC
5. EGFRvIII	0.76	0.76 Fair-12 Adequate-2	1.0 Imm in trials	0.62 Oncogenic-8 Uncertain but decreased survival-7	1.0 Absolute	0.37 High most	1.0 Stem cells	0.11 High level - Small subset	0.13 Single	1 Surface Little Circulation MHC
6. HER-2/neu	0.75	0.85 Adequate	1.0 Imm in trials	1.0 Oncogenic	0.35 Over expressed	0.37 High most	0.66 All stages	0.11 High level - Small subset	1.0 Multiple	0.25 Surface Circulating
7. Idiotype	0.75	0.76 Fair-13 Adequate-2	1.0 Imm in trials	0.12 Differentiation	1.0 Absolute	1.0 High all	0.66 All stages	0.14 Unique	1.0 Multiple	1.0 Surface Little Circulation MHC
8. MAGE A3	0.71	0.79 Fair-9 Adequate-6	1.0 Imm in trials	0.25 Uncertain but decreased survival	0.54 Oncofetal	0.37 High most	1.0 Stem cells	1.0 Many patients high level	1.0 Multiple	0.95 Internal MHC
9. p53 non- mutant	0.67	0.42 Fair-9 N/A-7	1.0 Imm in trials	1.0 Oncogenic	0.35 Over expressed	0.37 High most	1.0 Stem cells	1.0 Many patients high level	1.0 Multiple	0.95 Internal MHC
10. NY-ESO-1	0.66	0.75 Fair data	1.0 Imm in trials	0.25 Uncertain but decreased survival	0.54 Oncofetal	0.37 High most	1.0 Stem cells	0.11.0 High level - Small subset	1.0 Multiple	0.95 Internal MHC
11. PSMA	0.65	0.75 Fair data	1.0 Imm in trials	0.25 Uncertain but decreased survival	0.21 Tissue specific	1.0 High all	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	1.0 Surface Little Circulation MHC
12. GD2	0.65	0.75 Fair data	1.0 Imm in trials	0.12 Differentiation	0.35 Over expressed	1.0 High all	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	0.62 Surface with indeterminate circulating
13. CEA	0.62	0.75 Fair data	1.0 Imm in trials	0.12 Differentiation	0.35 Over expressed	0.37 High most	0.66 All stages	1.0 Many patients high level	1.0 Multiple	0.25 Surface Circulating
14. MelanA/MART1	0.60	0.77 Fair-11 Superb-1.0 Adequate-1 T cell Tx-1	1.0 Imm in trials	0.12 Differentiation	0.21 Tissue specific	0.37 High most	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	0.95 Internal MHC
15. Ras-mutant	0.60	0.1 Animal	1.0 Imm in trials	1.0 Oncogenic	1.0 Absolute	0.23 Low all	1.0 Stem cells	0.16 Many patients lower level	0.13 Single	0.95 Internal MHC
16. gp100	0.59	0.75 Fair data	1.0 Imm in trials	0.12 Differentiation	0.21 Tissue specific	0.37 High most	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	0.95 Internal MHC
17. p53 mutant	0.58	0.35 N/A-8 Fair-7	1.0 Imm in trials	1.0 Oncogenic	0.1 Unique random mutations	1.0 High all	0.77 All stages-10 Stem cells-5	0.14 Unique	0.13 Single	0.95 Internal MHC
18. Proteinase3 (PR1)	0.57	0.7 Fair-15 N/A-1	1.0 Imm in trials	0.12 Differentiation	0.35 Over expressed	0.37 High most	0.2 Most cancer cells	1.0 Many patients high level	0.13 Single	0.95 Internal MHC
19. Bcr-abl	0.56	0.00	1.0 Imm in trials	1.0 Oncogenic	1.0 Absolute	0.23 Low all	1.0 Stem cells	0.16 Many patients lower level	0.13 Single	0.95 Internal MHC
20. Tyrosinase	0.56	0.65 T cell Tx	1.0 Imm in trials	0.12 Differentiation	0.21 Tissue specific	0.37 High most	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	0.95 Internal MHC
21. Survivin	0.55	0.1 Animal	1.0 Imm in trials	1.0 Oncogenic	0.35 Over expressed	0.37 High most	0.66 All stages	1.0 Many patients high level	1.0 Multiple	0.95 Internal MHC
22. PSA	0.55	0.75 Fair data	1.0 Imm in trials	0.12 Differentiation	0.21 Tissue specific	0.08 Low most	0.66 All stages	0.16 Many patients lower level	1.0 Multiple	0.25 Surface Circulating
23. hTERT	0.54	0.15 Preexistent	1.0 Imm in trials	1.0 Oncogenic	0.35 Over expressed	0.23 Low all	1.0 Stem cells	0.16 Many patients lower level	1.0 Multiple	0.95 Internal MHC
24. Sarcoma translocation breakpoints	0.54	0.00	0.39 T cell observed	1.0 Oncogenic	1.0 Absolute	1.0 High all	1.0 Stem cells	1.0 Many patients high level	0.13 Single	0.95 Internal MHC
25. EphA2	0.53	0.1 Animal	1.0 Imm in trials	1.0 Oncogenic	0.35 Over expressed	0.37 High most	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	1.0 Surface Little Circulation MHC
26. PAP	0.52	0.69 Fair-9 Adequate-4 Animal-2	1.0 Imm in trials	0.12 Differentiation	0.21 Tissue specific	0.23 Low all	0.2 Most cancer cells	0.16 Many patients lower level	1.0 Multiple	0.25 Surface Circulating
27. ML-IAP	0.50	0.00	1.0 Imm in trials	1.0 Oncogenic	0.35 Over expressed	0.37 High most	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	0.95 Internal MHC
28. AFP	0.49	0.15 Preexistent	1.0 Imm in trials	0.24 Uncertain but decreased survival-11 Differenetiation-1	0.54 Oncofetal	0.37 High most	1.0 Stem cells	1.0 Many patients high level	1.0 Multiple	0.25 Surface Circulating
29. EpCAM	0.48	0.1 Animal	1.0 Imm in trials	0.12 Differentiation	0.35 Over expressed	1.0 High all	1.0 Stem cells	1.0 Many patients high level	1.0 Multiple	1.0 Surface Little Circulation MHC
30. ERG (TMPRSS2 ETS fusion gene)	0.48	0.00	0.39 T cell observed	1.0 Oncogenic	1.0 Absolute	0.37 High most	0.66 All stages	1.0 Many patients high level	0.13 Single	0.95 Internal MHC
31. NA17	0.48	0.59 Fair-11 N/A-3	1.0 Imm in trials	0.00	0.35 Over expressed	0.00	0.00	1.0 Many patients high level	0.13 Single	0.95 Internal MHC
32. PAX3	0.47	0.00	1.0 Imm in trials	1.0 Oncogenic	0.54 Oncofetal	0.08 Low most	0.2 Most cancer cells	0.00	1.0 Multiple	0.95 Internal MHC
33. ALK	0.46	0.00	0.39 T cell observed	1.0 Oncogenic	0.42 Overexressed -6 Absolute-3 N/A-3	1.0 High all	1.0 Stem cells	1.0 Many patients high level	0.27 Single-8 Multiple-2	0.95 Internal MHC
34. Androgen receptor	0.45	0.1 Animal	0.39 T cell observed	1.0 Oncogenic	0.35 Over expressed	0.37 High most	0.66 All stages	1.0 Many patients high level	1.0 Multiple	0.95 Internal MHC
35. Cyclin B1	0.44	0.1 Animal	0.39 T cell observed	1.0 Oncogenic	0.35 Over expressed	0.32 High most-7 Low all-4	0.66 All stages	1.0 Many patients high level	1.0 Multiple	0.95 Internal MHC
36. Polysialic Acid	0.44	0.00	1.0 Imm in trials	0.12 Differentiation	0.54 Oncofetal	1.0 High all	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	1.0 Surface Little Circulation MHC
37. MYCN	0.42	0.00	0.39 T cell observed	1.0 Oncogenic	0.54 Oncofetal	0.37 High most	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	0.95 Internal MHC
38. RhoC	0.42	0.00	0.39 T cell observed	1.0 Oncogenic	0.35 Over expressed	0.37 High most	0.66 All stages	1.0 Many patients high level	1.0 Multiple	0.95 Internal MHC
39. TRP-2	0.42	0.1 Animal	1.0 Imm in trials	0.12 Differentiation	0.21 Tissue specific	0.37 High most	1.0 Stem cells	1.0 Many patients high level	1.0 Multiple	0.95 Internal MHC
40. GD3	0.41	0.00	1.0 Imm in trials	0.12 Differentiation	0.35 Over expressed	1.0 High all	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	1.0 Surface Little Circulation MHC
41. Fucosyl GM1	0.41	0.00	1.0 Imm in trials	0.12 Differentiation	0.35 Over expressed	1.0 High all	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	1.0 Surface Little Circulation MHC
42. Mesothelin	0.41	0.00	1.0 Imm in trials	0.25 Uncertain but decreased survival	0.35 Over expressed	0.37 High most	0.66 All stages	1.0 Many patients high level	1.0 Multiple	1.0 Surface Little Circulation MHC
43. PSCA	0.41	0.75 Fair data	0.11 Animal	0.12 Differentiation	0.21 Tissue specific	0.37 High most	0.2 Most cancer cells	0.16 Many patients lower level	1.0 Multiple	1.0 Surface Little Circulation MHC
44. MAGE A1	0.40	0.00	1.0 Imm in trials	0.25 Uncertain but decreased survival	0.54 Oncofetal	0.00	1.0 Stem cells	0.1.06 Many patients lower level	1.0 Multiple	0.95 Internal MHC
45. sLe(a)	0.40	0.00	1.0 Imm in trials	0.12 Differentiation	0.35 Over expressed	1.0 High all	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	0.25 Surface Circulating
46. CYP1B1	0.40	0.00	1.0 Imm in trials	0.00	0.35 Over expressed	1.0 High all	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	0.95 Internal MHC
47. PLAC1	0.39	0.00	0.39 T cell observed	1.0 Oncogenic	0.54 Oncofetal	0.37 High most	0.2 Most cancer cells	0.11 High level - Small subset	1.0 Multiple	1.0 Surface Little Circulation MHC
48. GM3	0.38	0.1 Animal	1.0 Imm in trials	0.12 Stroma	0.35 Over expressed	0.37 High most	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	0.25 Surface Circulating
49. BORIS	0.38	0.1 Animal	0.11 Animal	0.1 Oncogenic	0.54 Oncofetal	0.08 Low most	0.66 All stages	0.16 Many patients lower level	1.0 Multiple	0.95 Internal MHC
50. Tn	0.37	0.00	1.0 Imm in trials	0.25 Uncertain but decreased survival	0.23 Post- translational changes	0.37 High most	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	1.0 Surface Little Circulation MHC
51. GloboH	0.37	0.00	1.0 Imm in trials	0.12 Differentiation	0.35 Over expressed	0.37 High most	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	1.0 Surface Little Circulation MHC
52. ETV6-AML	0.37	0.00	0.00	1.0 Oncogenic	1.0 Absolute	0.23 Low all	0.66 All stages	0.11 High level - Small subset	0.13 Single	0.95 Internal MHC
53. NY-BR-1	0.36	0.00	0.39 T cell observed	0.12 Differentiation	1.0 Absolute	0.37 High most	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	1.0 Surface Little Circulation MHC
54. RGS5	0.35	0.00	1.0 Imm in trials	0.00	0.35 Over expressed	0.37 High most	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	0.95 Internal MHC
55. SART3	0.35	0.00	1.0 Imm in trials	0.00	0.35 Over expressed	0.37 High most	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	0.95 Internal MHC
56. STn	0.34	0.00	1.0 Imm in trials	0.25 Uncertain but decreased survival	0.23 Post- translational changes	0.37 High most	0.2 Most cancer cells	0.16 Many patients lower level	1.0 Multiple	1.0 Surface Little Circulation MHC
57. Carbonic anhydrase IX	0.34	0.00	1.0 Imm in trials	0.00	0.35 Over expressed	0.37 High most	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	0.25 Surface Circulating
58. PAX5	0.33	0.00	0.39 T cell observed	1.0 Oncogenic	0.21 Tissue specific	0.23 Low all	0.2 Most cancer cells	0.16 Many patients lower level	1.0 Multiple	0.95 Internal MHC
59. OY-TES1	0.32	0.00	0.1 Ab observed	1.0 Oncogenic	0.54 Oncofetal	0.08 Low most	0.2 Most cancer cells	0.16 Many patients lower level	1.0 Multiple	0.95 Internal MHC
60. Sperm protein 17	0.30	0.1 Animal	0.11 Animal	0.25 Uncertain but decreased survival	0.54 Oncofetal	0.37 High most	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	1.0 Surface Little Circulation MHC
61. LCK	0.28	0.00	1.0 Imm in trials	0.00	0.35 Over expressed	0.00	0.00	0.00	1.0 Multiple	0.95 Internal MHC
62. HMWMAA	0.27	0.1 Animal	0.11 Animal	0.00	0.35 Over expressed	1.0 High all	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	1.0 Surface Little Circulation MHC
63. AKAP-4	0.26	0.1 Animal	0.11 Animal	0.12 Differentiation	0.54 Oncofetal	0.37 High most	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	0.25 Surface Circulating
64. SSX2	0.26	0.00	0.39 T cell observed	0.25 Uncertain but decreased survival	0.54 Oncofetal	0.08 Low most	0.2 Most cancer cells	0.11 High level - Small subset	1.0 Multiple	0.95 Internal MHC
65. XAGE 1	0.23	0.00	0.1 Ab observed	0.00	0.54 Oncofetal	0.37 High most	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	0.95 Internal MHC
66. B7H3	0.22	0.00	0.00	0.25 Uncertain but decreased survival	0.35 Over expressed	0.37 High most	0.2 Most cancer cells	1.0 Many patients high level	1.0 Multiple	1.0 Surface Little Circulation MHC
67. Legumain	0.19	0.1 Animal	0.11 Animal	0.00	0.35 Over expressed	0.37 High most	0.2 Most cancer cells	0.00	1.0 Multiple	1.0 Surface Little Circulation MHC
68. Tie 2	0.18	0.1 Animal	0.11 Animal	0.00	0.23 Post- translational changes	0.00	0.00	1.0 Many patients high level	1.0 Multiple	1.0 Surface Little Circulation MHC
69. Page4	0.17	0.00	0.00	0.12 Differentiation	0.21 Tissue specific	0.37 High most	0.00	1.0 Many patients high level	1.0 Multiple	0.95 Internal MHC
70. VEGFR2	0.16	0.1 Animal	0.11 Animal	0.12 Stroma	0.1 Stromal	0.00	0.00	1.0 Many patients high level	1.0 Multiple	0.25 Surface Circulating
71. MAD-CT-1	0.15	0.00	0.1 Ab observed	0.00	0.54 Oncofetal	0.00	0.00	0.00	1.0 Multiple	0.95 Internal MHC
72. FAP	0.14	0.1 Animal	0.00	0.00	0.1 Stromal	0.00	0.00	1.0 Many patients high level	1.0 Multiple	1.0 Surface Little Circulation MHC
73. PDGFR- beta	0.14	0.00	0.11 Animal	0.12Stroma	0.1 Stromal	0.00	0.66 All stages	0.00	1.0 Multiple	1.0 Surface Little Circulation MHC
74. MAD-CT-2	0.14	0.00	0.1 Ab observed	0.00	0.54 Oncofetal	0.00	0.00	0.16 Many patients lower level	1.0 Multiple	0.00
75. Fos-related antigen 1	0.13	0.1 Animal	0.11 Animal	0.00	0.1 Stromal	0.00	0.00	0.11 High level - Small subset	1.0 Multiple	1.0 Surface Little Circulation MHC

Row 1: Criteria are listed in descending order of weighting. The (numbers) refer to relative weighting of each criterion. The cumulative (numbers) total 1.

Column 1: Antigens are listed in descending order of ranking. The numbers refer to the ranking as well as to a literature reference for each antigen.

Column 2: Cumulative scores are listed in descending order of ranking. The cumulative score for each antigen is the sum product of pre-determined weights for the 9 criteria. Cumulative score for each antigen is the sum product of the weight of each Criteria multiplied by the score of the Sub-criteria. Total Score = (Weight of Criteria 1)×(Score of Sub-Criteria for Criteria 1) + (Weight of Criteria 2)×(Score of Sub-Criteria for Criteria 2) + (Weight of Criteria 3)×(Score of Sub-Criteria for Criteria 3) + etc.

Columns 3–11 Numbers represent the weight of the top Sub-criteria appropriate for that antigen within the Criteria denoted in the column. The terms are abbreviations for the names for the Sub-criteria. The full names and weighted scores for each Sub-criterion are presented in column 3 of Table 1. If there are no numbers following the term, the panel members all agreed on the Sub-criteria. If there are numbers following the terms, the panel members disagreed on what should be the top Sub-category. The numbers designate the number of votes for each Sub-category.

Figure 2.

Cancer antigen pilot prioritization: representation of ranking based on predefined and preweighted criteria and sub-criteria. Inset indicates the color used to designate each criterion and its relative weight. The number at the end of each bar indicates the relative rank of that antigen.

No antigen exhibited all of the top sub-criteria (Table 2). By this assessment, no antigen, among those selected, satisfied the criteria for an “Ideal” cancer antigen. The dominant criterion was “Therapeutic Function” and the top 14 antigens all have significant contributions from that criterion, namely “Fair” to “Very Strong Data Controlled vaccine trial”. Altogether, 20 antigens were deemed to have at least “Fair Data Controlled vaccine trial suggestive”. None were deemed to have “Superb Data” by any of the experts.

The second dominant criterion was “Immunogenicity”. In all, 46 of the 75 antigens – including the top 14 – had documented immunogenicity in human clinical trials. The total weight of “Therapeutic Function” plus “Immunogenicity” was 0.49. The dominance of “Therapeutic Function” and “Immunogenicity” biased the ratings towards antigens already in analyzable clinical trials, i.e., antigens further along in the developmental process.

To assess priorities without bias toward already having been in clinical trials, the antigens were re-ranked, excluding “Therapeutic Function” and “Immunogenicity” (Figure 3). After excluding these top 2 criteria, the antigen ranking was dominated by the criteria of “Oncogenicity”, “Specificity” and “Stem Cell Expression”. In this alternative model the break point region of translocated fusion genes (Ewing’s sarcoma and alveolar rhabdomyosarcoma; ALK, bcr-abl and ETV6-AML) and mutant oncogenes (ras) rose to the top. The method of reporting data in Table 3 allows re-prioritization of the antigens and development of alternative rankings based on alternative assessment or weighting of criteria and sub-criteria of interest.

Figure 3.

Representation of ranking following exclusion of “therapeutic efficacy” and “immunogenicity”. Inset indicates the color used to designate each criterion and its relative weight. The number at the end of each bar indicates the relative rank of that antigen.

Discussion

This study developed a well vetted, priority-ranked list of cancer vaccine target antigens based on pre-defined and pre-weighted objective criteria developed by a panel of content experts. The AHP method and Decision Lens platform provided the framework to catalogue and weight vaccine development decision criteria, and to rank 75 selected antigens. This process was done in three stages by three panels of cancer vaccine experts with overlapping members. The first panel defined the criteria to be ranked for priority. The second panel weighted the criteria. The third panel ranked the 75 antigens according to the pre-defined pre-weighted criteria based on information on the antigens provided by researchers familiar with the individual antigens. The broad nature of the input underpinning the list of criteria should facilitate subsequent NCI or other funding agency discussions as to which antigens to test in subsequent focused translational/clinical studies.

This study is termed a “Pilot Prioritization Project” with the emphasis on “Pilot”. One of the goals was to determine whether the methods employed could be used to rank priorities for subsequent efforts to accelerate translational research. The finding that 20 of the 75 evaluated antigens had some clinical efficacy and that 46 of them had validated immunogenicity in human clinical trials documents the extent and vigor of the cancer vaccine field, but also accentuates the need for prioritization. Notably, there are cancer antigens under development that were not included in this prioritization effort, again, accentuating the breadth of the opportunities in the field.

Of the 46 antigens with validated immunogenicity and 20 with suggestive clinical data, none are FDA-approved for general use. It is generally assumed that development of any of the top antigens will require concerted collaboration on the part of experts in cancer vaccine development. We anticipate that the prioritization of Immunotherapy Agents with High Potential for Cancer Therapy ⁵ and the current ranking of cancer antigens will jointly lay the foundation for such focused collaborations.

The final scoring and ranking was necessarily done with incomplete knowledge. Much more was known about some antigens than others. Antigens that had undergone the most prior research had a marked advantage in the ranking. The experts ranked “Therapeutic Efficacy” and “Immunogenicity” as the top criteria. Within these categories there were many types of trials with different endpoints and different patient selection criteria. Thus, it was necessary to further divide the criteria according to the level of the data. The sub-categorization was not precise, and was subjective along the scale of Fair to Superb Data. The experts had varying opinions about the quality of the data and the panel had no opportunity to examine raw data from any trials. An in-depth analysis of primary clinical data for the antigens would be required to substantiate the results before any definitive action could be taken. Furthermore, the ranking at best represents the current state of our knowledge and will change as new information becomes available.

The order changed appreciably when re-analyzed without the top criteria, “Therapeutic Efficacy” and “Immunogenicity”. The leading criteria then became “Oncogenicity”, “Specificity” and “Stem Cell Expression”, and the priorities of the break point region of translocated fusion genes (Ewing’s sarcoma and alveolar rhabdomyosarcoma; ALK, bcr-abl and ETV6-AML) and mutant oncogenes (ras) rose to the top. Arguably, it will be harder for them to achieve therapeutic efficacy as these antigens require selective MHC presentation of a small and single epitope. Thus, there may be some underlying biologic justification for their lower ranking.

Knowledge within other categories was often also incomplete or inadequate. For example:

1. “Stem cell expression” was deemed to be important, but the group recognized that the field of stem cell identification is rapidly evolving. Future thoughts and assessments about cancer stem cells could be markedly different.

2. The criterion of “Oncogenicity” was important. However, many antigens not considered to be “oncogenic” are associated with a poor prognosis and are clearly involved in helping to sustain the malignant phenotype. Thus, the definition of “oncogenic” may be too restrictive. The outcome of immunologic pressure is often the evolution of antigen-negative variants. It would seem beneficial to target antigens, which if lost, resulted in diminished ability of the cancer cells to survive or thrive. Necessity for maintaining a malignant phenotype is a broader definition than “oncogenic” per se and might be more relevant.

3. It was felt by the experts that antigens with “no or little circulating antigen” were substantially preferable to antigens with “circulating antigen”. However, the group did not have access to actual side-by-side data quantifying circulating antigen and did not define a threshold value discriminating between the two. Moreover, in certain cases the amount of circulating antigen was not well characterized in the literature.

No antigen exhibited all of the top sub-criteria. By this assessment, no antigen, among those selected, satisfied the criteria for an “ideal” cancer antigen. Some of the deficiencies such as stem cell expression are biologic and can’t be changed. Others such as immunogenicity and level of therapeutic efficacy can potentially be changed with additional experiments and more data and, most compellingly, by the use of more effective vaccine formulations and schedules of administration. For antigens too early in development to have garnered evidence of clinical efficacy or immunogenicity, the dominance of those criteria in the experts’ ratings provides a road-map for investigators by emphasizing that high quality data concerning these criteria are critical for prioritization of antigens for focused subsequent development.

Another question is whether there is “Ideal” cancer antigens left to be discovered. It can be assumed that the first antigens discovered would be amongst the most abundant and the most immunogenic. Abundance and immunogenicity are both major criteria. By extrapolation, it can be argued that many of the antigens left to be discovered would be less abundant and less immunogenic molecules.

Of the 75 antigens evaluated 46 were immunogenic in clinical trials and 20 of them had suggestive clinical efficacy in the “Therapeutic function” category with documented vaccine induced clinical responses in at least a small number of patients, or suggestive evidence of benefit vs. controls. However, none were deemed to have “superb data” in the category of “Therapeutic function”. The lack of “superb data” could be multifactorial, including inadequate trial design or patient selection, and inadequate vaccine formulation or regimens. These deficiencies can be overcome by more intelligent trial design based on assessment of past “productive failures”.

Two profound biologic issues limiting the efficacy of cancer vaccines are the strength of immunological tolerance and the intrinsic limitations on the ability of T cells to expand in number in response to antigenic stimulation. There are normally exceedingly strict biologic limits imporsed on the immune system to prevent excessive T-cell activation and expansion. The same biological restrictions limit cancar vaccines. Immunotherapeutic agents that can circumvent many of the biological restrictions have been invented and formulated and proven to be biologically active, including dendritic cell activators and growth factors, vaccine adjuvants, T-cell stimulators and growth factors, genetically-modified T cells, immune checkpoint inhibitors, and agents to neutralize or inhibit suppressive cells, cytokines, and enzymes. Unfortunately, few of these agents are broadly available for the development of effective multiple component cancer vaccine regimens. The tools needed to raise T-cell levels to extraordinary levels in vivo and to maintain T cell number for prolonged periods of time are at hand. A major problem facing immunotherapy today is a lack of broad availability of agents already in existence that could be effective in multiple component regimens as well as administrative difficulties funding and carrying out such multiple component regimens. It is highly likely that therapeutic regimens composed of optimal vaccine formulations with combinations of already invented immunotherapy agents in the above categories would lift the level of data into the “superb data” subcategory for many of the 20 antigens as well as others less studied. The current prioritization process, by validating that at least 20 antigens have “suggestive clinical efficacy”, highlights the need for an administrative and funding structure capable of translating these scientific discoveries into effective cancer therapies.

The AHP approach has several advantages over more standard evaluation and prioritization approaches. The AHP framework requires detailed discussion of the specific criteria in advance of the prioritization, permitting a comparison of individual perceptions and forcing the group to reach consensus on interpretations and definitions. This is presumed to improve the consistency of responses and has the effect of generating confidence in the results and “buy-in” among stakeholders. AHP allows the information to be evaluated quantitatively and qualitatively, using both subjective and objective ranking scales. The ability to apply non-linear weights to criteria and ranking scales was viewed as a distinct advantage over a system that simply averages the results. The Decision Lens platform provided an organized and consistent way to organize and view data, thereby facilitating evaluation. The transparency of the process was a benefit in that disagreements were quickly recognized and could be discussed. Finally, the web-based asynchronous approach was viewed as an efficient use of experts’ time.

The flexibility of the AHP/Decision Lens approach in permitting “what if” scenarios was exceptionally valuable in understanding how changing the weight of the criteria and sub-criteria would affect the outcome and helped to provide a comfort level with the generated priority list. The approach accommodates viewing the data with selected criteria given any proportion of the weighing, including zero. The flexibility of the system has the advantage of simplifying re-evaluation of alternatives when additional information becomes available, and allows for modification of criteria as more experience with generating cancer vaccines is gained. As one example, the flexibility will allow for alternative assessments of prioritization for the same antigen in different tumor types in circumstances where the antigen has markedly different expression patterns.

It must be noted that the AHP does not make decisions; rather it provides a way to analyze and prioritize alternatives. One of the limitations of AHP is that it only ranks degrees of positivity. In some cases there can be “deal breaking” negative information that needs to be assessed outside of the AHP. A list of ranked alternatives provides a rational basis for decisions at the executive level. This pilot prioritization study produced a ranked list of cancer antigens that can be used by the broad immunotherapy community when considering further investment in experimental research for individual antigens as they move toward the goal of translating the most promising cancer antigens into vaccines for cancer treatment or prevention.

Statement of Translational Relevance

We report on the development of a prioritized list of cancer vaccine target antigens using well-vetted criteria generated by expert panels. The elucidation and weighting of criteria to assess cancer antigens will assist investigators in the immunotherapy field in determining the characteristics and the experimental data required to select the most promising antigens for further development and testing in clinical trials.