Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2008 May 1.
Published in final edited form as: J Acquir Immune Defic Syndr. 2007 Feb 1;44(2):203–212. doi: 10.1097/01.qai.0000248356.48501.ff

Phase 2 Study of an HIV-1 Canarypox Vaccine (vCP1452) Alone and in Combination With rgp120

Negative Results Fail to Trigger a Phase 3 Correlates Trial

Nina D Russell *,, Barney S Graham , Michael C Keefer §, M Juliana McElrath *,, Steve G Self ∥,, Kent J Weinhold #, David C Montefiori #, Guido Ferrari #, Helen Horton *, Georgia D Tomaras #, Sanjay Gurunathan **, Lynn Baglyos **, Sharon E Frey ††, Mark J Mulligan ‡‡, Clayton D Harro §§, Susan P Buchbinder ∥∥, Lindsey R Baden ¶¶, William A Blattner ##, Beryl A Koblin ***, Lawrence Corey *,; the National Institute of Allergy and Infectious Diseases HIV Vaccine Trials Network
PMCID: PMC2362395  NIHMSID: NIHMS44935  PMID: 17106277

Abstract

Background

A goal of T-cell HIV vaccines is to define the correlation between a vaccine-induced immune response and protection from HIV infection. We conducted a phase 2 trial to determine if a canarypox vaccine candidate (vCP1452) administered with rgp120 subunit protein would “qualify” for a trial to define a correlate of efficacy.

Methods

A total of 330 healthy volunteers were enrolled into 4 groups: 120 received vCP1452 alone (0, 1, 3, and 6 months), 120 received vCP1452 with 2 different regimens of rgp120 coadministration, and 90 received placebo. HIV-specific antibody responses were measured by enzyme-linked immunoassay (ELISA) and neutralizing activity. T-cell responses were measured by chromium release and interferon-γ (IFNγ) enzyme-linked immunospot (ELI-Spot) assay.

Results

Significant neutralizing antibody responses to the HIV MN strain were detected in all vaccine groups, with net responses ranging from 57% (95% confidence interval [CI]: 40% to 71%) to 94% (95% CI: 85% to 99%). Net cumulative HIV-specific CD8+ IFNγ ELISpot assay responses were 13% (95% CI: 21% to 26%) for recipients of vCP1452 alone and 16% (95% CI: 2% to 29%) for recipients of vCP1452 plus rgp120.

Conclusions

Overall, the HIV-specific CD8+ cytotoxic T lymphocyte (CTL) response was not sufficient to qualify the regimen for a subsequent trial designed to detect an immune correlate of protection requiring a minimum CD8+ CTL frequency of 30%.

Keywords: canarypox vector, cytotoxic T lymphocyte, enzyme-linked immunospot assay, HIV vaccine, rgp120 subunit protein, vCP1452


The development of a preventive HIV vaccine is the best long-term solution to controlling HIV and AIDS. Multiple approaches to vaccine development are being pursued and include DNA, viral vector, protein, or peptide-based vaccines, alone or in combination. The attenuated poxvirus vectors, particularly the canarypox (ALVAC) vectors, have been some of the most extensively studied. An ALVAC gp160 product was the first replication-incompetent HIV vaccine to demonstrate consistent CD8+ cytotoxic T lymphocyte (CTL) responses after vaccination of seronegative persons.1 Subsequent canarypox vaccine candidates were developed to increase expression of more complex HIV-1 gene inserts, and these have undergone extensive evaluation.27

The most recent generation of canarypox HIV vaccines, the ALVAC-HIV vCP1452, encodes HIV-1 gp120, the entire Gag protein, a portion of pol, and several human CTL epitopes from nef and pol. It also contains sequences encoding the E3L and K3L vaccinia proteins,8 which are known to inhibit apoptosis and prolong protein expression in human cells.9 In cell culture, vCP1452 demonstrates efficient protein expression, which is enhanced significantly over the earlier generation canarypox vector, vCP205.9 Five canarypox vaccines were previously evaluated in 9 different AIDS Vaccine Evaluation Group (AVEG) trials with or without rgp120 or rgp160 in high- and low-risk volunteers and demonstrated a favorable safety profile.15,7 The 4 to 6 immunizations administered in these studies elicited neutralizing antibodies to homologous HIV-1 strains in 88% of participants and CD8+ CTLs to Env and/or Gag epitopes in up to 59% of participants.4,7,10,11 To examine the safety an immunogenicity of the vCP1452 canarypox vector vaccine further, alone and in combination with a protein subunit boost, we undertook a double-blind, randomized, phase 2 study.

In this trial, HIV Vaccine Trials Network (HVTN) protocol 203, 4 doses of vCP1452 vaccine were tested at 0, 1, 3, and 6 months alone or in combination with 2 different regimens of rgp120 (3 and 6 months vs. 0, 1, and 6 months). The primary objectives of the trial were to expand the safety profile for these products and to ascertain whether an HIV-specific CD8+ CTL response could be detected at a high enough frequency to qualify the product for subsequent large-scale trials. The trial was co-designed with a test-of-concept efficacy trial that had as its primary objective the definition of a correlate of immunity. Additionally, previous HIV ALVAC vaccine trials relied on a traditional chromium release CTL assay to provide a qualitative assessment of T-cell responses;4,5,7,11,12 this was the first trial to use the interferon-γ (IFNg) enzyme-linked immunospot (ELISpot) assay to measure and quantitate CD8+ CTLs as a primary endpoint.13

MATERIALS AND METHODS

Vaccines

The recombinant ALVAC-HIV vCP1452 vaccine (Sanofi Pasteur, Swiftwater, PA) expresses HIV-1 envelope gp120 (MN) linked to the transmembrane portion of HIV-1 gp41 (LAI), the HIV-1LAI gene encoding the entire Gag protein, the portion of pol encoding protease, a polynucleotide encompassing several human nef and pol CTL epitopes, and sequences encoding the E3L and K3L vaccinia virus proteins (Sanofi Pasteur). The approximate titer was 107.26 median tissue culture infective dose (TCID50) for each 1.0-mL dose. The recombinant AIDSVAX™ B/B gp120 subunit is derived from a clade B primary isolate GNE8 sequence and HIV-1MN .14,15 Each dose consisted of 300 μg/mL of MN rgp120 and 300 μg/mL of GNE8 rgp120 in 1.2 mg of aluminum hydroxide adjuvant (VaxGen Inc., Brisbane, CA). PLACEBO-ALVAC (Sanofi Pasteur) contained virus stabilizer and freeze-drying medium, reconstituted with sterile 0.4% sodium chloride. AIDSVAX PLACEBO was a sterile suspension of aluminum hydroxide adjuvant. ALVAC vCP1452 or placebo was administered in the left arm and AIDSVAX B/B subunit or placebo was administered in the right arm as a 1.0-mL intramuscular injection.

Subjects

The study enrolled healthy HIV-1–uninfected adults aged 18 to 60 years. Risk for HIV infection was assessed at study entry based on a standardized interview of past and current sexual and drug use behaviors. All subjects were in good general health, HIV-seronegative, and were counseled to use birth control and avoid pregnancy throughout the course of the study. All participants provided written informed consent, and each of the 10 trial sites obtained approval for the study through their local institutional review boards.

Study Design

Eligible participants received injections at 4 time points (Table 1) and were monitored for 18 months for safety and HIV-specific humoral and cellular immune responses. Local and systemic reactogenicity symptoms were graded according to the following criteria: (1) mild: transient or minimal symptoms, (2) moderate: symptoms requiring modification of activity, and (3) severe: incapacitating symptoms resulting in bed rest and/or loss of work or social activities. Serious adverse experiences (SAEs) were reported according to the Division of Acquired Immune Deficiency Syndrome (DAIDS) SAE reporting manual. Grade 3 or 4 AEs assessed as definitely, probably, or possibly related were also reported as SAEs.

TABLE 1.

Vaccine Trial Schema

Immunization Schedule in Months (d)
Group Number Injection Arm 0 (0) 1 (28) 3 (84) 6 (168)
A ALVAC alone 120 Left A A A A
Right P P P P
B ALVAC + AIDSVAX combined 60 Left A A A A
Right P P GP GP
C 60 Left A A A A
Right GP GP P GP
D Placebo 90 Left AP AP AP AP
Right P P P P

A indicates ALVAC-HIV recombinant canarypox virus vaccine expressing HIV antigens (vCP1452), approximate dose of 107.26 TCID50; AP, placebo for ALVAC-HIV recombinant canarypox virus vaccine, approximate dose of 107.26 TCID50; GP, AIDSVAX B/B recombinant gp120 vaccine, dose of 300 μg/mL MN rgp120 and 300 μg/mL GNE8 rgp120 antigen in 1.2 mg of aluminum hydroxide adjuvant; P, placebo for AIDSVAX B/B vaccine, aluminum hydroxide adjuvant.

Study Procedures

Whole blood was collected on days 0, 98, 182, 273, 364, and 546 and was shipped overnight to the HVTN Central Immunology Laboratories (CILs) at Duke University and the Fred Hutchinson Cancer Research Center. Sera and peripheral blood mononuclear cells (PBMCs) were cryopreserved and thawed using standard procedures for immunogenicity studies.

Enzyme-Linked Immunosorbent Assays

Qualitative enzyme-linked immunosorbent assays (ELISAs) were performed (Duke University CIL) to determine anti-Gag antibody responses. Cryopreserved sera from days 0, 98, 182, 364, and 546 were tested in duplicate with purified p24 Gag (Quality Biological Inc., Gaithersburg, MD). A score (optical density [OD] antigen – OD nonantigen) was considered positive if the OD was ≥0.2.

Neutralizing Antibodies

Sera from days 0, 42, 98, 182, and 364 were tested for neutralization (Duke University CIL) against HIV-1MN and HIV-1IIIB in an MT-2 cell-killing assay as described previously.16 Titers were the reciprocal serum dilution at which 50% of cells were protected from virus-induced killing as measured by neutral red uptake. Positive responses were defined as titers ≥10.

Interferon-γ Enzyme-Linked Immunospot Assays

ELISpot assays were used to detect HIV-1–specific IFNγ-producing T cells from cryopreserved PBMCs as previously described.13 Briefly, 96-well plates (Millipore Corp., Billerica, MA) were coated with anti-IFNγ monoclonal antibody (mAb; Mabtech Inc., Cincinnati, OH) overnight at 4°C. PBMCs were thawed and incubated overnight. T-cell depletions were performed using anti-CD4+ mAb-coated immunomagnetic beads (Miltenyi Biotec, Auburn, CA) before adding 2 × 105 cells per well of CD4+ cells or CD8+ cells. HIV-1–specific Env, Gag, Pol, and Nef 15-mer peptides overlapping by 11aa (MN, HXB2, HXB2, and BRU/LAI strains, respectively) were added at a final concentration of 1 μg/mL. Cells stimulated with phytohemagglutinin and a pool of 27 immunodominant 8- to 10-mer peptides (Anaspec Inc., San Jose, CA) from cytomegalovirus, Epstein-Barr virus, and influenza were used as positive controls.17 Responses were tested in duplicate.

Spot-forming cells (SFCs) were counted using an automated ELISpot reader (Cellular Technology Ltd., Cleveland, OH). Responses were considered positive if the mean SFCs in the experimental wells were ≥55 per 1 × 106 cells and ≥4 times the mean of the negative control wells.18

Chromium Release Assays

Chromium release assays were performed in triplicate at 2 effector/target ratios (50:1 and 25:1) to detect CD8+ CTLs on fresh PBMCs shipped overnight to the Duke University CIL, as previously described.5,11,19,20 Recombinant vaccinia constructs used to infect autologous B-cell line targets included vP1291 (expressing gp120MN, the TM portion of gp41MN, and gag/proteaseIIIB), vP1288 (expressing polIIIB), and vP1218 (expressing nefIIIB).

Statistical Methods

Categoric data were tested using the 2-sided Fisher exact test. Laboratory assay net response rates were considered statistically significant if the exact 95% confidence interval (CI) did not include 0; pairwise differences were tested using the nonparametric Wilcoxon rank sum test. Reactogenicity data were evaluated using the Kruskal-Wallis test. All differences were statistically significant if P ≥ 0.05, and no adjustments were made for multiple comparisons. For CTLs and ELISpot assays, cumulative response rates on day 98 or 182 were computed. Standard optimization techniques were used to obtain nonparametric maximum likelihood estimates (NPMLEs) of cumulative response rates.21

The study design was intended to provide a basis for a go/no-go decision regarding a phase 3 efficacy trial (HVTN 501). The decision criterion was based on the null hypothesis that the true response rate was not >30%, setting the sample size at 120 vaccinees per regimen to provide at least 90% power to distinguish response rates of 30% and 45%. Thus, HVTN 203 was independently designed to qualify vCP1452 alone and vCP1452 plus rgp120, the latter by pooling groups B and C, for potential efficacy evaluation in HVTN 501.

RESULTS

Accrual and Demographic Data

Three hundred 30 participants were enrolled at 10 sites in the United States between December 2000 and August 2001 and were randomized to 1 of 4 groups (see Table 1). Overall, 305 (92%) participants completed all 4 vaccinations, with 279 (85%) completing follow-up. A total of 25 participants discontinued vaccinations for diverse reasons, including 1 pregnancy and 1 HIV infection. No participants discontinued because of concerns regarding vaccine toxicity.

Table 2 shows the group stratification by gender, sexual preference, race, HIV risk group, and age. The median age of enrolled participants was 35.0 years; 82% of enrollees were non-Hispanic whites, and 12% were non-Hispanic African Americans. For all demographic variables, no statistically significant differences were observed across groups (P > 0.05, Fisher exact test).

TABLE 2.

Demographics and Vaccination Frequencies

Group
ALVAC Alone
ALVAC + AIDSVAX Combined
Placebo
A (n = 120) B (n = 60) C (n = 60) D (n = 90) Total (n = 330)
Gender
 Male 83 (69%) 37 (62%) 40 (67%) 66 (73%) 226 (68%)
 Female 37 (31%) 23 (38%) 20 (33%) 24 (27%) 104 (32%)
Sexual preference
 Homosexual 62 (52%) 22 (37%) 26 (43%) 49 (54%) 159 (48%)
 Heterosexual 50 (42%) 33 (55%) 29 (48%) 34 (38%) 146 (44%)
 Bisexual 8 (7%) 5 (8%) 5 (8%) 7 (8%) 25 (8%)
Race
 White, non-Hispanic 101 (84%) 45 (75%) 51 (85%) 75 (83%) 272 (82%)
 African American 14 (12%) 10 (17%) 6 (10%) 10 (11%) 40 (12%)
 Hispanic 4 (3%) 3 (5%) 1 (2%) 2 (2%) 10 (3%)
 Asian/Pacific Islander 0 (0%) 1 (2%) 0 (0%) 1 (1%) 2 (1%)
 Native American/Alaskan 1 (1%) 0 (0%) 1 (2%) 2 (2%) 4 (1%)
 Other 0 (0%) 1 (2%) 1 (2%) 0 (0%) 2 (1%)
Risk group
 Low 68 (57%) 38 (63%) 39 (65%) 49 (54%) 194 (59%)
 High 52 (43%) 22 (37%) 21 (35%) 41 (46%) 136 (41%)
Age (y)
 18−40 79 (67%) 48 (80%) 39 (65%) 63 (70%) 229 (69%)
 41−50 35 (29%) 11 (18%) 18 (30%) 19 (21%) 83 (25%)
 >50 6 (5%) 1 (2%) 3 (5%) 8 (9%) 18 (5%)
 Median 37.0 32.0 34.0 35.0 35.0
 Range 19−55 19−52 20−54 19−58 19−58
Vaccination frequencies
 Day 0 120 (100%) 60 (100%) 60 (100%) 90 (100%) 330 (100%)
 Day 28 119 (99%) 58 (97%) 59 (98%) 87 (97%) 323 (98%)
 Day 84 118 (98%) 56 (93%) 57 (95%) 84 (93%) 315 (95%)
 Day 168 117 (98%) 53 (88%) 55 (92%) 80 (89%) 305 (92%)

For all demographic variables, no statistically significant differences were observed across groups (P > 0.05, Fisher exact test).

Safety and Tolerability

Overall, the vCP1452 and rgp120 injections were well tolerated. Two vaccinees (1 in group A and 1 in group B) experienced severe pain or tenderness that completely resolved within 24 to 72 hours of onset. When compared with placebo, rates for maximum pain or tenderness and for maximum erythema or induration were significantly higher in all vaccine groups for the ALVAC injection (left arm, P > 0.01) but not for the rgp120 injection (right arm, P ≥ 0.13).

Systemic reactogenicity, including malaise, myalgia, headache, nausea, or fever, was generally mild, although 6 vaccinees (1 in group A, 1 in group B, and 4 in group C) experienced severe systemic side effects that resolved within 1 to 5 days after vaccination. Of the participants experiencing a temperature ≥38.3°C, 6 of 120, 2 of 60, 0 of 60, and 1 of 90 were observed in groups A, B, C, and D, respectively. There were no significant trends or abnormalities in hematologic, renal, or hepatic profiles or in CD4+ T-cell counts seen in any of the 4 groups. Comparing rates of participants experiencing 1 or more AEs, there were no significant differences between treatment groups overall (P = 0.82, Fisher exact test).

Three grade 3 AEs for which relatedness to the study product was definite (n = 1), probable (n = 1), or “cannot rule out” (n = 1) were reported: 2 from group A and 1 from group B. These included 2 severe reactogenicity experiences (“probably related” chills in group A and “definitely related” bilateral arm pain or tenderness in group B). The third grade 3 AE (group A) was a T-cell lymphoma diagnosed 4 months after the final vaccination; the participant was treated with chemotherapy, and disease was still present at the end of the study. In addition, there were 2 participants with miscarriages (groups A and C) determined to be unrelated to the study product (1 group C participant had 2 miscarriages). There were 6 participants (5 vaccinees and 1 placebo recipient) who became HIV infected, 2 before completion of the immunization schedule and 4 between 1 and 7 months after vaccination.

Humoral Immunogenicity

Anti-Gag p24 binding antibodies were detected by ELISA (data not shown) at day 182, 2 weeks after the final vaccination, in all vaccine groups with response rates ranging from 23% (95% CI: 8% to 38%) to 36% (95% CI: 21% to 51%). Responses peaked at day 182 and were detectable by day 364 in only 2 group A participants. There was no significant difference between vaccine groups in the number of recipients mounting p24 antibodies at day 182, suggesting that rgp120 coadministration in groups B and C did not significantly enhance Gag-specific antibody responses.

As depicted in Figure 1, HIV-1MN neutralizing antibody (nAb) titers were significantly higher (P < 0.01) in all vaccine groups for days 98, 182, and 364 when compared with placebo but only in group C for day 42. At day 98, net response rates ranged from 70% (95% CI: 57% to 81%) to 83% (95% CI: 69% to 93%). Notably, nAb titers were low when detectable (geometric mean titer [GMT] range: 18.1−44.9) in those receiving ALVAC alone (group A). Their responses peaked after the third vaccination, decreasing in magnitude by days 182 and 364.

FIGURE 1.

FIGURE 1

nAbs against HIV-1MN in peripheral blood measured by vital dye neutralization assay. Antibody levels are expressed as log titers by day within group. Each box plot displays the 25th percentile, the median, and the 75th percentile. The whiskers on each box plot show the fifth and 95th percentiles, and n equals the number of subjects tested in each group at each time point. Titers <10 were set equal to 5 for computations. Pairwise differences between treatment groups were tested using the 2-sided nonparametric Wilcoxon rank sum test. When compared with placebo, statistically significant differences (*P ≤ 0.01) were detected in all vaccine groups at days 98, 182, and 364 and in group C at day 42.

The magnitude and frequency of nAb titers to HIV-1MN were consistently higher (average GMT >3-fold) in groups receiving rgp120 compared with vCP1452 alone (P < 0.01 at day 182). Interestingly, a single boost with vCP1452 plus rgp120 after 2 doses of vCP1452 alone (group B, day 98) was superior to 2 doses of vCP1452 plus rgp120 boosted with vCP1452 alone (group C; P < 0.01). By month 6, however, group C responses were clearly boosted with 1 additional dose of vCP1452 plus rgp120, whereas group B responses were not (see Fig. 1). The explanation for this is unclear but suggests that 3 injections of subunit protein are better than 2 and/or that the rgp120 schedule with the 5-month boost interval (0, 1, and 6 months) is superior to the 3-month interval (3 and 6 months). By day 364, without any additional boosts, the antibody responses in all groups receiving vaccine were detected at low titers and were not significantly different irrespective of schedule. This suggests that Env-specific memory B cells persist and is consistent with previous observations regarding the longevity of these responses.22

Induction of nAbs to HIV-IIIIB was examined in a subset (n = 62) of samples from day 182 (data not shown). These sera were less able to neutralize HIV-1IIIB than HIV-1MN, and greater response frequencies were observed in the groups receiving rgp120 (14 of 20 subjects [70%] in groups B and C) versus vCP1452 only (0 of 25 subjects [0%] in group A) or placebo (0 of 17 subjects [0%] in group D) (P < 0.01). Similarly, day 182 anti-HIV-1IIIB antibody titers were significantly higher in the rgp120-treated groups (P < 0.01). There was no evidence of nAb activity against primary clade B isolates.

Cellular Immunogenicity

Vaccine-induced HIV-1–specific T-cell responses were measured at 2 weeks after the third and fourth vaccinations (days 98 and 182) in fresh PBMCs using a chromium release CTL assay. Significant net cumulative CD8+ CTLs to any antigen (gag, env, nef,or pol) were detected at day 98 or day 182 in 16% (95% CI: 2% to 35%) to 24% (95% CI: 5% to 43% of vaccinees with no significant differences between those who received ALVAC alone versus ALVAC plus rgp120 (Table 3A). There were no significant CD8+ CTL net point prevalence rates at day 98 or day 182 (see Table 3B). No significant CD4+ T-cell responses were measured via chromium release assay (data not shown).

TABLE 3.

CD8+ CTL Responses to Any HIV Antigen Detected via Chromium Release Assay at Day 98 or Day 182*

A. Cumulative
Group Observed Cumulative Response NPMLE Cumulative Response NPMLE Net Cumulative Response 95% CI
A 34/106 = 32% 38% 18% (3% to 32%)
B 16/47 = 34% 45% 24% (5% to 43%)
C 15/49 = 31% 37% 16% (2% to 35%)
B and C 31/96 = 32% 40% 20% (6% to 35%)
D 13/73 = 18% 21% n/a n/a
B. Point Prevalence
Group Day Response Net Response§ (95% CI)
A 98 23/87 = 26% 14% (−2 to 30%)
182 15/77 = 19% 9% (−7% to 26%)
B 98 11/37 = 30% 17% (−3% to 36%)
182 5/37 = 14% 3% (−16% to 24%)
C 98 8/42 = 19% 6% (−13% to 26%)
182 7/38 = 18% 8% (−11% to 28%)
B and C 98 19/79 = 24% 11% (−5% to 28%)
182 12/75 = 16% 6% (−11 to 23%)
D 98 8/63 = 13% n/a
182 6/58 = 10% n/a
*

Indicates response to “any HIV antigen” is defined as a response to any of the Env-, Gag-, Pol-, or Nef-expressing recombinant vaccinia vectors used in the assay.

Observed cumulative response rates included any positive responses detected on day 98 and/or day 182.

Because traditional probabilities based on observed cumulative rates tend to be biased and underestimate response, standard optimization techniques were used to obtain NPMLEs of cumulative response rates.19

§

For each vaccine group, net point prevalence response rates were computed by subtracting the placebo group response rate and were considered to be statistically significant if the corresponding 95% CI did not include 0.

n/a indicates not applicable.

IFNγ ELISpot assays were conducted in 2 HVTN CILs that divided cryopreserved PBMCs based on study site. Significant net cumulative IFNγ-secreting CD8+ T-cell responses were detected to env, gag, pol,or nef at days 98 or 182 in 23% (95% CI: 6% to 40%) of group B participants (Table 4A). For group A participants, net cumulative CD8+ ELISpot responses were only 13% (95% CI: 21% to 26%). Gag-specific responses contributed most significantly to the overall CD8+ T-cell response rates, and net point prevalence response rates to any of the antigens only reached statistical significance for group B at day 98 (28% response, 95% CI: 8% to 47%; see Table 4B). The day 98 net point prevalence response rates in the other 2 active groups did not reach statistical significance, nor did the CD8+ T-cell responses at day 182 in any of the vaccine groups, suggesting that the higher day 98 group B response rate might be a function of the multiplicity of comparisons rather than a true higher response rate.

TABLE 4.

CD8+ and CD4+ IFNγ ELISpot Assay T-Cell Responses to Any HIV Antigen at Day 98 or Day 182*

A. Cumulative
CD8+
CD4+
Group Observed Cumulative Response NPMLE Cumulative Response NPMLE Net Cumulative Response 95% CI Observed Cumulative Response NPMLE Cumulative Response NPMLE Net Cumulative Response 95% CI
A 25/101 = 25% 28% 13% (−1% to 26%) 7/113 = 6% 7% 1% (−7% to 9%)
B 17/48 = 35% 38% 23% (6% to 40%) 10/53 = 19% 20% 15% (3% to 28%)
C 11/50 = 22% 24% 9% (−7% to 26%) 5/55 = 9% 10% 4% (−5% to 15%)
B and C 28/98 = 29% 31% 16% (2% to 29%) 15/108 = 14% 15% 9% (0% to 18%)
D 8/62 = 13% 15% n/a n/a 4/78 = 5% 6% n/a n/a
B. Point Prevalence
CD8+
CD4+
Group Day Response Net Response§ (95% CI) Response Net Response (95% CI)
A 98 15/90 = 17% 9% (−7 to 26%) 3/103 = 3% N/A
182 13/76 = 17% 9% (−8% to 26%) 5/97 = 5% 2% (−10% to 17%)
B 98 15/42 = 36% 28% (8% to 47%) 6/49 = 12% 9% (−6% to 27%)
182 7/42 = 17% 9% (−11% to 29%) 5/48 = 10% 7% (−8% to 25%)
C 98 6/44 = 14% 6% (−13% to 26%) 3/52 = 6% 3% (−11% to 20%)
182 7/42 = 17% 9% (−11% to 29%) 3/50 = 6% 3% (−11% to 20%)
B and C 98 21/86 = 24% 17% (0% to 33%) 9/101 = 9% 6% (−6% to 20%)
182 14/84 = 17% 9% (−8 to 26%) 8/98 = 8% 5% (−8% to 20%)
D 98 4/53 = 8% n/a 2/72 = 3% n/a
182 4/50 = 8% n/a 2/68 = 3% n/a
*

Response to “any HIV antigen” is defined as a response to any of the Env, Gag, Pol, or Nef peptide pools used in the assay.

Observed cumulative response rates included any positive responses detected on day 98 and/or day 182.

Because traditional probabilities based on observed cumulative rates tend to be biased and underestimate response, standard optimization techniques were used to obtain NPMLEs of cumulative response rates.19

§

For each vaccine group, net point prevalence response rates were computed by subtracting the placebo group response rate and were considered to be statistically significant if the corresponding 95% CI did not include 0.

n/a indicates not applicable.

CD4+ T-cell responses were also examined by IFNγ ELISpot assay. A significant net cumulative CD4+ T-cell response rate of 15% was detected to env, gag, pol,or nef at days 98 or 182 (95% CI: 3% to 28%) for participants in group B only (see Table 4A). No significant net point prevalence CD4+ T-cell response rates were observed. ELISpot assays were also performed on unfractionated PBMCs at the day 364 time point from a subset of 191 subjects, and no appreciable responses were detected (data not shown).

DISCUSSION

After investigation of recombinant vaccinia in the late 1980s,23,24 recombinant canarypox vectors became the approach in the 1990s for achieving vaccine-induced HIV-specific T-cell responses. Several generations of canarypox vaccines were tested in phase 1-II clinical trials, alone and in combination with recombinant envelope protein boosts.17,11,25 These trials established an excellent safety profile for this vaccine regimen at the current dose levels, which is further supported by this study. Of note, a recent trial evaluated the safety and immunogenicity of ALVAC vCP1452 at a higher dose (108 TCID50) and found a significantly higher rate of local and systemic reactogenicity,20 suggesting that the 107.26-TCID50 dose used in this study may be approximating the maximum tolerated dose level.

Prior examinations have shown that primary poxvirus immunizations can result in higher magnitude antibody responses when followed by a protein boost.23,24 This study compared 2 different schedules for delivering vCP1452 in combination with rgp120 protein (see Table 1). Interestingly, Gag-specific binding antibody responses were similar across all groups, indicating that codelivery of rgp120 did not enhance or interfere with the antibody-inducing ability of vCP1452 alone. However, at the peak time point after the fourth vaccination, nAb responses were higher in those who received the earlier 0-, 1-, and 6-month rgp120 vaccinations compared with those who did not receive rgp120 until 3 and 6 months. This small improvement in day 182 antibody levels is in contrast to the diminished T-cell response measured in that group, suggesting that simultaneous injection of a protein subunit may influence the T-cell priming afforded by a vaccine vector. This finding may be important for the design of future combination vaccine regimens.

The major focus of this vaccine concept was to induce HIV-specific CD8+ CTLs because of their role in controlling viral replication and disease progression. This trial was initiated at the time of a paradigm shift in the methods for measuring T-cell responses.2633 The measurement of cytokine production in activated T cells was found to be a correlate of lytic activity as measured by the standard chromium51 release assay, leading to the development of IFNγ ELISpot assays that could be validated using cryopreserved cells.13 This study was the first to use this high-throughput platform in a relatively large-scale multicenter vaccine trial. Disappointingly, the CD8+ T-cell responses in this study were significantly lower than expected based on the responses observed with earlier generation canarypox products such as vCP205. The ELISpot assay responses were highest in the group that received rgp120 at months 3 and 6 in combination with vCP1452, but these responses were at a level substantially lower than what was required to advance to a predesigned test-of-concept immune correlates trial.

In addition to genes encoding HIV antigens, the vCP1452 vector expresses the vaccinia E3L and K3L genes, with properties that may influence antigen expression and presentation efficiencies. E3L and K3L inhibit the IFN-induced, double-stranded, RNA-activated protein kinase, PKR. The expression of these genes can also inhibit apoptosis, which may permit prolonged protein expression in human cells;9 their inclusion was intended to improve the overall level of immunogenicity. It is possible, however, that inhibition of apoptosis and other IFN-induced effects may have altered HIV antigen presentation directly or through reduced cross-priming. This may explain why the frequency of HIV-specific CTL responses was even lower in this study than in studies evaluating the vCP205 construct lacking the E3L and K3L genes.

The design of phase 2 trials is inextricably linked to the goals of subsequent phase 3 efficacy evaluation. The efficacy trial planned for vCP1452 with or without rgp120 was a test-of-concept study designed to determine whether a vaccine-induced immune response was correlated with the prevention or control of early HIV-1 infection. Three major factors were required for the test-of-concept analysis: (1) adequate enrollment of high-risk subjects to ensure accumulation of sufficient infection events, (2) a minimal level of vaccine-induced efficacy, and (3) a sufficiently high frequency of measurable vaccine-induced immune responses. The first factor is a function of trial site capacity and resources, and the second factor is only known by performing the efficacy trial. Providing data for the third factor was the primary purpose of this phase 2 evaluation. The efficacy trial planned for vCP1452 required a vaccine-induced CD8+ CTL frequency of at least 30%. To ensure that the response would be sufficient to define a correlate of immunity, a 42% point estimate was needed with a lower bound of the 95% confidence interval above 30%. Because this level of vaccine-induced CD8+ CTLs was not observed in this trial, plans for the test-of-concept trial were abandoned. Although it is still possible that the canarypox vaccine would demonstrate some level of efficacy not correlated with the IFNγ ELISpot response (as suggested in a recently published ALVAC–simian immunodeficiency virus [SIV] infant macaque study34), the immune responses were not measurable in high enough frequency in phase 2 to establish the correlate of immunity in the designed test-of-concept trial. A recombinant vCP205-like canarypox vector vaccine (vCP1521) that contains subtype E env but lacks the vaccinia E3L and K3L sequences is being given with a protein boost (AIDSVAX B/E) in an efficacy trial currently underway in Thailand, however, which may be able to answer this question.

ACKNOWLEDGMENTS

The authors gratefully acknowledge the participation of all the study volunteers and the support of numerous colleagues and staff at the participating sites and laboratories. Additionally, they sincerely appreciate the efforts of Mark Deers for all his work on data preparation and analysis.

Funding provided through the HIV Vaccine Trials Network, sponsored by the Division of AIDS, National Institute of Allergy and Infectious Diseases, National Institutes of Health, Bethesda, MD (grant UO1 AI 46747).

Appendix

Additional Safety Data

Table A1 provides additional safety data.

TABLE A1A.

Maximum Local Reactogenicity After ALVAC Vaccination in Left Arm*

Group
ALVAC Alone
ALVAC + AIDSVAX Combined
Placebo
Local Reactions A (n = 120) B (n = 60) C (n = 60) D (n = 90)
Maximum pain or tenderness
 None 3 (3%) 0 (0%) 1 (2%) 32 (36%)
 Mild 53 (44%) 32 (53%) 37 (62%) 49 (54%)
 Moderate 63 (53%) 27 (45%) 22 (37%) 9 (10%)
 Severe 1 (1%) 1 (2%) 0 (0%) 0 (0%)
P <0.01 <0.01 <0.01 n/a
Maximum erythema or induration
 None 93 (78%) 45 (75%) 47 (78%) 84 (93%)
 0.01−10.00 cm2 17 (14%) 8 (13%) 9 (15%) 6 (7%)
 10.01−25.00 cm2 6 (5%) 3 (5%) 3 (5%) 0 (0%)
 >25.00 cm2 4 (3%) 4 (7%) 1 (2%) 0 (0%)
P <0.01 <0.01 <0.01 n/a
Lymph nodes
 None 116 (97%) 58 (97%) 57 (95%) 89 (99%)
 0.1−1.4 cm 2 (2%) 1 (2%) 1 (2%) 1 (1%)
 1.5−3.0 cm 2 (2%) 1 (2%) 2 (3%) 0 (0%)
 >3.0 cm 0 (0%) 0 (0%) 0 (0%) 0 (0%)
P 0.30 0.47 0.23 n/a
*

ALVAC injections were given in the left arm only.

Pairwise differences between the vaccine and placebo groups were tested using the 2-sided Kruskal-Wallis test, where differences were considered to be statistically significant if P ≤ 0.05.

n/a indicates not applicable.

TABLE A1B.

Maximum Local Reactogenicity After AIDSVAX Vaccination in Right Arm*

Group
ALVAC Alone
ALVAC + AIDSVAX Combined
Placebo
Local Reactions A (n = 120) B (n = 60) C (n = 60) D (n = 90)
Maximum pain or tenderness
 None 15 (13%) 3 (5%) 3 (5%) 8 (9%)
 Mild 70 (58%) 40 (67%) 42 (70%) 65 (72%)
 Moderate 35 (29%) 16 (27%) 15 (25%) 17 (19%)
 Severe 0 (0%) 1 (2%) 0 (0%) 0 (0%)
P§ 0.35 0.13 0.26 n/a
Maximum erythema or induration
 None 110 (92%) 49 (82%) 51 (85%) 77 (86%)
 0.01−10.00 cm2 8 (7%) 7 (12%) 5 (8%) 13 (14%)
 10.01−25.00 cm2 1 (1%) 2 (3%) 1 (2%) 0 (0%)
 >25.00 cm2 1 (1%) 2 (3%) 3 (5%) 0 (0%)
P§ 0.18 0.39 0.82 n/a
Lymph nodes
 None 115 (96%) 59 (98%) 56 (93%) 89 (99%)
 0.1−1.4 cm 4 (3%) 0 (0%) 1 (2%) 1 (1%)
 1.5−3.0 cm 1 (1%) 1 (2%) 3 (5%) 0 (0%)
 >3.0 cm 0 (0%) 0 (0%) 0 (0%) 0 (0%)
P§ 0.22 0.76 0.04 n/a
*

AIDSVAX injections were given in the right arm only.

Participants in group B received 2 injections of the AIDSVAX placebo (months 0 and 1) and 2 injections of the rgp120 vaccine (months 3 and 6) in the right arm.

Participants in group C received 3 injections of the rgp120 vaccine (months 0, 1, and 6) and 1 injection of the AIDSVAX placebo (month 3) in the right arm.

§

Pairwise differences between the vaccine and placebo groups were tested using the 2-sided Kruskal-Wallis test, where differences were considered to be statistically significant if P ≤ 0.05.

n/a indicates not applicable.

TABLE A1C.

Maximum Systemic Reactogenicity After Vaccination

Group
ALVAC Alone
ALVAC + AIDSVAX Combined
Placebo
Systemic Reactions A (n = 120) B (n = 60) C (n = 60) D (n = 90)
Maximum systemic reaction
 None 22 (18%) 20 (33%) 20 (33%) 31 (34%)
 Mild 56 (47%) 27 (45%) 21 (35%) 38 (42%)
 Moderate 41 (34%) 12 (20%) 15 (25%) 21 (23%)
 Severe 1 (1%) 1 (2%) 4 (7%) 0 (0%)
P* 0.01 1.00 0.39 n/a
Maximum temperature (°C)
 <37.8 101 (84%) 51 (85%) 54 (90%) 85 (94%)
 37.8−38.2 13 (11%) 7 (12%) 6 (10%) 4 (4%)
 38.3−38.8 3 (3%) 2 (3%) 0 (0%) 1 (1%)
 ≥38.9 3 (3%) 0 (0%) 0 (0%) 0 (0%)
P* 0.02 0.07 0.35 n/a
*

Pairwise differences between the vaccine and placebo groups were tested using the 2-sided Kruskal-Wallis test, where differences were considered to be statistically significant if P ≤ 0.05.

n/a indicates not applicable.

Footnotes

Presented in part at the Keystone Symposia, Banff, Alberta, Canada, April 13, 2005 [abstract 408].

REFERENCES

  • 1.Clements-Mann ML, Weinhold K, Matthews TJ, et al. Immune responses to human immunodeficiency virus (HIV) type 1 induced by canarypox expressing HIV-1MN gp120, HIV-1SF2 recombinant gp120, or both vaccines in seronegative adults. NIAID AIDS Vaccine Evaluation Group. J Infect Dis. 1998;177:1230–1246. doi: 10.1086/515288. [DOI] [PubMed] [Google Scholar]
  • 2.Horig H, Lee DS, Conkright W, et al. Phase I clinical trial of a recombinant canarypoxvirus (ALVAC) vaccine expressing human carcinoembryonic antigen and the B7.1 co-stimulatory molecule. Cancer Immunol Immunother. 2000;49:504–514. doi: 10.1007/s002620000146. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Wright PF, Mestecky J, McElrath MJ, et al. Comparison of systemic and mucosal delivery of 2 canarypox virus vaccines expressing either HIV-1 genes or the gene for rabies virus G protein. J Infect Dis. 2004;189:1221–1231. doi: 10.1086/382088. [DOI] [PubMed] [Google Scholar]
  • 4.Gupta K, Hudgens M, Corey L, et al. Safety and immunogenicity of a high-titered canarypox vaccine in combination with rgp120 in a diverse population of HIV-1–uninfected adults: AIDS Vaccine Evaluation Group Protocol 022A. J Acquir Immune Defic Syndr. 2002;29:254–261. doi: 10.1097/00126334-200203010-00005. [DOI] [PubMed] [Google Scholar]
  • 5.Evans TG, Keefer MC, Weinhold KJ, et al. A canarypox vaccine expressing multiple human immunodeficiency virus type 1 genes given alone or with rgp120 elicits broad and durable CD8+ cytotoxic T lymphocyte responses in seronegative volunteers. J Infect Dis. 1999;180:290–298. doi: 10.1086/314895. [DOI] [PubMed] [Google Scholar]
  • 6.Tartaglia J, Excler JL, El Habib R, et al. Canarypox virus-based vaccines: prime-boost strategies to induce cell-mediated and humoral immunity against HIV. AIDS Res Hum Retroviruses. 1998;14(Suppl 3):S291–S298. [PubMed] [Google Scholar]
  • 7.Belshe RB, Stevens C, Gorse GJ, et al. Safety and immunogenicity of a canarypox-vectored human immunodeficiency virus Type 1 vaccine with or without gp120: a phase 2 study in higher- and lower-risk volunteers. J Infect Dis. 2001;183:1343–1352. doi: 10.1086/319863. [DOI] [PubMed] [Google Scholar]
  • 8.Davies MV, Chang HW, Jacobs BL, et al. The E3L and K3L vaccinia virus gene products stimulate translation through inhibition of the double-stranded RNA-dependent protein kinase by different mechanisms. J Virol. 1993;67:1688–1692. doi: 10.1128/jvi.67.3.1688-1692.1993. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Fang ZY, Limbach K, Tartaglia J, et al. Expression of vaccinia E3L and K3L genes by a novel recombinant canarypox HIV vaccine vector enhances HIV-1 pseudovirion production and inhibits apoptosis in human cells. Virology. 2001;291:272–284. doi: 10.1006/viro.2001.1209. [DOI] [PubMed] [Google Scholar]
  • 10.The AIDS Vaccine Evaluation Group 022 Protocol Team Cellular and humoral immune responses to a canarypox vaccine containing human immunodeficiency virus type 1 Env, Gag, and Pro in combination with rgp120. J Infect Dis. 2001;183:563–570. doi: 10.1086/318523. [DOI] [PubMed] [Google Scholar]
  • 11.Belshe RB, Gorse GJ, Mulligan MJ, et al. Induction of immune responses to HIV-1 by canarypox virus (ALVAC) HIV-1 and gp120 SF-2 recombinant vaccines in uninfected volunteers. NIAID AIDS Vaccine Evaluation Group. AIDS. 1998;12:2407–2415. doi: 10.1097/00002030-199818000-00009. [DOI] [PubMed] [Google Scholar]
  • 12.Corey L, McElrath MJ, Weinhold K, et al. Cytotoxic T cell and neutralizing antibody responses to human immunodeficiency virus type 1 envelope with a combination vaccine regimen. AIDS Vaccine Evaluation Group. J Infect Dis. 1998;177:301–309. doi: 10.1086/514202. [DOI] [PubMed] [Google Scholar]
  • 13.Russell ND, Hudgens MG, Ha R, et al. Moving to human immunodeficiency virus type 1 vaccine efficacy trials: defining T cell responses as potential correlates of immunity. J Infect Dis. 2003;187:226–242. doi: 10.1086/367702. [DOI] [PubMed] [Google Scholar]
  • 14.Gilbert PB, Peterson ML, Follmann D, et al. Correlation between immunologic responses to a recombinant glycoprotein 120 vaccine and incidence of HIV-1 infection in a phase 3 HIV-1 preventive vaccine trial. J Infect Dis. 2005;191:666–677. doi: 10.1086/428405. [DOI] [PubMed] [Google Scholar]
  • 15.Flynn NM, Forthal DN, Harro CD, et al. Placebo-controlled phase 3 trial of a recombinant glycoprotein 120 vaccine to prevent HIV-1 infection. J Infect Dis. 2005;191:654–665. doi: 10.1086/428404. [DOI] [PubMed] [Google Scholar]
  • 16.Montefiori DC, Robinson WE, Jr, Schuffman SS, et al. Evaluation of antiviral drugs and neutralizing antibodies to human immunodeficiency virus by a rapid and sensitive microtiter infection assay. J Clin Microbiol. 1988;26:231–235. doi: 10.1128/jcm.26.2.231-235.1988. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Currier JR, Kuta EG, Turk E, et al. A panel of MHC class I restricted viral peptides for use as a quality control for vaccine trial ELISPOT assays. J Immunol Methods. 2002;260:157–172. doi: 10.1016/s0022-1759(01)00535-x. [DOI] [PubMed] [Google Scholar]
  • 18.Coplan PM, Gupta SB, Dubey SA, et al. Cross-reactivity of anti-HIV-1 T cell immune responses among the major HIV-1 clades in HIV-1-positive individuals from 4 continents. J Infect Dis. 2005;191:1427–1434. doi: 10.1086/428450. [DOI] [PubMed] [Google Scholar]
  • 19.Ferrari G, King K, Rathbun K, et al. IL-7 enhancement of antigen-driven activation/expansion of HIV-1-specific cytotoxic T lymphocyte precursors (CTLp). Clin Exp Immunol. 1995;101:239–248. doi: 10.1111/j.1365-2249.1995.tb08345.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Goepfert PA, Horton H, McElrath MJ, et al. Recombinant canarypox vaccine expressing human immunodeficiency virus type 1 protein given at a high dose in seronegative human participants. J Infect Dis. 2005;192:1249–1259. doi: 10.1086/432915. [DOI] [PubMed] [Google Scholar]
  • 21.Hudgens MG. Estimating cumulative probabilities from incomplete longitudinal binary responses with application to HIV vaccine trials. Stat Med. 2003;22:463–479. doi: 10.1002/sim.1334. [DOI] [PubMed] [Google Scholar]
  • 22.Evans TG, Frey S, Israel H, et al. Long-term memory B-cell responses in recipients of candidate human immunodeficiency virus type 1 vaccines. Vaccine. 2004;22:2626–2630. doi: 10.1016/j.vaccine.2003.12.011. [DOI] [PubMed] [Google Scholar]
  • 23.Graham BS, Matthews TJ, Belshe RB, et al. Augmentation of human immunodeficiency virus type 1 neutralizing antibody by priming with gp160 recombinant vaccinia and boosting with rgp160 in vaccinia-naive adults. The NIAID AIDS Vaccine Clinical Trials Network. J Infect Dis. 1993;167:533–537. doi: 10.1093/infdis/167.3.533. [DOI] [PubMed] [Google Scholar]
  • 24.Cooney EL, McElrath MJ, Corey L, et al. Enhanced immunity to human immunodeficiency virus (HIV) envelope elicited by a combined vaccine regimen consisting of priming with a vaccinia recombinant expressing HIV envelope and boosting with gp160 protein. Proc Natl Acad Sci USA. 1993;90:1882–1886. doi: 10.1073/pnas.90.5.1882. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Bures R, Gaitan A, Zhu T, et al. Immunization with recombinant canarypox vectors expressing membrane-anchored glycoprotein 120 followed by glycoprotein 160 boosting fails to generate antibodies that neutralize R5 primary isolates of human immunodeficiency virus type 1. AIDS Res Hum Retroviruses. 2000;16:2019–2035. doi: 10.1089/088922200750054756. [DOI] [PubMed] [Google Scholar]
  • 26.Doherty PC. The new numerology of immunity mediated by virus-specific CD8(+) T cells. Curr Opin Microbiol. 1998;1:419–422. doi: 10.1016/s1369-5274(98)80059-3. [DOI] [PubMed] [Google Scholar]
  • 27.Larsson M, Jin X, Ramratnam B, et al. A recombinant vaccinia virus based ELISPOT assay detects high frequencies of Pol-specific CD8 T cells in HIV-1-positive individuals. AIDS. 1999;13:767–777. doi: 10.1097/00002030-199905070-00005. [DOI] [PubMed] [Google Scholar]
  • 28.Tan LC, Gudgeon N, Annels NE, et al. A re-evaluation of the frequency of CD8+ T cells specific for EBV in healthy virus carriers. J Immunol. 1999;162:1827–1835. [PubMed] [Google Scholar]
  • 29.Lalvani A, Brookes R, Hambleton S, et al. Rapid effector function in CD8+ memory T cells. J Exp Med. 1997;186:859–865. doi: 10.1084/jem.186.6.859. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Smith JG, Liu X, Kaufhold RM, et al. Development and validation of a gamma interferon ELISPOT assay for quantitation of cellular immune responses to varicella-zoster virus. Clin Diagn Lab Immunol. 2001;8:871–879. doi: 10.1128/CDLI.8.5.871-879.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Maecker HT, Dunn HS, Suni MA, et al. Use of overlapping peptide mixtures as antigens for cytokine flow cytometry. J Immunol Methods. 2001;255:27–40. doi: 10.1016/s0022-1759(01)00416-1. [DOI] [PubMed] [Google Scholar]
  • 32.Ogg GS, McMichael AJ. HLA-peptide tetrameric complexes. Curr Opin Immunol. 1998;10:393–396. doi: 10.1016/s0952-7915(98)80110-6. [DOI] [PubMed] [Google Scholar]
  • 33.Altman JD, Moss PA, Goulder PJ, et al. Phenotypic analysis of antigen-specific T lymphocytes. Science. 1996;274:94–96. [PubMed] [Google Scholar]
  • 34.Van Rompay KK, Abel K, Lawson JR, et al. Attenuated poxvirus-based simian immunodeficiency virus (SIV) vaccines given in infancy partially protect infant and juvenile macaques against repeated oral challenge with virulent SIV. J Acquir Immune Defic Syndr. 2005;38:124–134. doi: 10.1097/00126334-200502010-00002. [DOI] [PubMed] [Google Scholar]

RESOURCES