Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Jul 1.
Published in final edited form as: Nanomedicine. 2015 Mar 6;11(5):1097–1107. doi: 10.1016/j.nano.2015.02.008

Protein transfer-mediated surface engineering to adjuvantate virus-like nanoparticles for enhanced anti-viral immune responses

Jaina M Patel a, Min-Chul Kim b,c, Vincent F Vartabedian a, Yu-Na Lee b, Sara He a, Jae-Min Song d, Hyo-Jick Choi e, Satoshi Yamanaka a, Nikhil Amaram a, Anna Lukacher a, Carlo Montemagno e, Richard W Compans f, Sang-Moo Kang b, Periasamy Selvaraj a,*
PMCID: PMC4512837  NIHMSID: NIHMS670595  PMID: 25752855

Abstract

Recombinant virus-like nanoparticles (VLPs) are a promising nanoparticle platform to develop safe vaccines for many viruses. Herein, we describe a novel and rapid protein transfer process to enhance the potency of enveloped VLPs by decorating influenza VLPs with exogenously added glycosylphosphatidylinositol-anchored immunostimulatory molecules (GPI-ISMs). With protein transfer, the level of GPI-ISM incorporation onto VLPs is controllable by varying incubation time and concentration of GPI-ISMs added. ISM incorporation was dependent upon the presence of a GPI-anchor and incorporated proteins were stable and functional for at least 4 weeks when stored at 4°C. Vaccinating mice with GPI-granulocyte macrophage colony-stimulating factor (GM-CSF)-incorporated-VLPs induced stronger antibody responses and better protection against a heterologous influenza virus challenge than unmodified VLPs. Thus, VLPs can be enriched with ISMs by protein transfer to increase the potency and breadth of the immune response, which has implications in developing effective nanoparticle-based vaccines against a broad spectrum of enveloped viruses.

Keywords: Virus-like particles, protein transfer, Influenza, GPI-anchored proteins, GM-CSF

Background

Current influenza vaccines are produced using embryonic hen eggs, cell cultures, or recombinant technologies. However, immunogenicity of the current inactivated split vaccine is subtype-strain specific and does not generate effective cross-protection against heterologous viral strains (1, 2). Therefore, there is an urgent need to develop a quick, safe, and broadly protective influenza vaccine. Recent animal studies and human clinical trials suggest that virus-like nanoparticles (VLPs) can be used as an alternative influenza vaccine (3–5). VLPs are derived from budding membranes of cells transfected to express viral matrix and envelope proteins and, therefore, mimic viral shape and structure through the expression of repetitive viral antigens without viral genomes. Influenza VLPs are typically 80–120 nm in diameter (6), allowing for trafficking to lymph nodes as well as optimal uptake by localized dendritic cells upon vaccination (7, 8). Therefore, VLP vaccination elicits both a strong humoral and cellular adaptive immune response against the virus (3, 9–11). Furthermore, VLP-based vaccines can quickly be produced in large quantities compared to conventional influenza vaccines. However, VLPs, similar to conventional influenza vaccines, induce mainly subtype specific protection with minimum cross protection against heterologous viral strains.

To enhance VLP immunogenicity for stronger cross-protection, VLPs have been genetically modified using the recombinant baculovirus (rBV) system to express immunostimulatory molecules (ISMs), such as membrane-bound cytokines, GM-CSF, CD40L, and flagellin, on the VLP surface (12–14). Another approach to introduce proteins onto VLPs is by pseudotyping, which involves fusing peptides or proteins-of-interest and viral proteins together on the VLPs (15, 16). Genetic modification is time-consuming and other limitations include increased insect-cell cytopathology as a result of increased multiplicity of infection when using multiple rBVs (17). More importantly, the level of ISM and viral protein incorporation cannot be controlled by these genetic approaches and the creation of fusion proteins may affect protein functionality.

Herein we show for the first time that purified GPI anchored-immunostimulatory molecules (GPI-ISMs) can be stably incorporated onto influenza VLPs in a concentration dependent manner within 2 hours (Figure 1) using a protein transfer technology. Vaccination of mice with VLPs modified by protein transfer with GPI-GM-CSF led to enhanced anti-heterologous virus antibody responses and complete protection against a heterologous viral challenge, which was not elicited by unmodified VLPs. The ease of protein transfer makes this method an attractive model system to adjuvantate enveloped VLP-based nanoparticle vaccines and enhance cross-protection against heterologous influenza viruses.

Figure 1. Schematic of protein transfer of GPI-anchored proteins onto influenza VLPs.

Figure 1

Open in a new tab

Transmembrane or secretory ISMs can be converted to GPI-anchored forms by attaching the DNA sequence correlating to the extracellular domain of transmembrane proteins or secretory ISMs to the DNA sequence corresponding to the GPI-anchor signal sequence from naturally occurring GPI-anchored CD59. The DNA constructs were then inserted into mammalian cell expression plasmid vectors and transfected into CHO-K1 cells. Transfected CHO-K1 cells were grown in large quantities using roller bottles, lysed, and then the GPI-ISMs are purified by mAb-affinity chromatography. Protein transfer is performed by incubating enveloped influenza VLPs with purified GPI-ISMs at 37°C for 2–6 h. Unincorporated GPI-ISMs are washed out by centrifugation at 13,200 rpm at 4°C for 30 min. Resulting incorporation is detected by western blot analysis. CM – cell membrane; ECD – extracellular domain; HA - influenza hemagglutinin protein; M1 - influenza matrix protein 1.

Methods

Animals

Female BALB/c mice (6–8 weeks old) were purchased from Jackson Laboratory and experiments were conducted as per approved IACUC guidelines of Emory University and Georgia State University.

DNA Constructs

To construct GPI-GM-CSF, nucleotides corresponding to the GPI-signal sequence from CD59 was attached to the C-terminus of mouse GM-CSF DNA as previously described (18–20). Similarly, GPI-ICAM-1 was constructed by attaching CD59 GPI-signal sequence to the extracellular domain of mouse ICAM-1 using an AflII restriction enzyme site (21). GPI-IL-12 was made by attaching CD59 GPI-signal sequence to mouse IL-12 p35 followed by an IRES sequence and mouse IL-12 p40-CD59 or soluble IL-12 p40. The GPI-IL-12 construct was placed in a pUB6blast vector (Invitrogen), whereas GPI-GM-CSF and GPI-ICAM-1 constructs were placed in a pcDNA3Neo vector (Invitrogen).

GPI-ISM purification

GPI-ISMs were isolated using mAb-affinity chromatography from the detergent lysate of CHO-K1 cell transfectants as described previously (22, 23). Briefly, CHO-K1 cells transfected to express GPI-ISMs were grown in large quantities using roller bottles. Cell pellets were then lysed with 50 mM Tris-HCl pH 8, 2% n-octyl-β-D-glucopyranoside (A.G. Scientific), 1:100 dilution of Protease Inhibitor Cocktail (Sigma), 5 mM EDTA, 20 mM sodium iodoacetate, and 2 mM PMSF overnight at 4°C. The lysate was centrifuged at 14,000 rpm for 1 h at 4°C and the supernatant was passed through a 70 μm cell strainer to remove cell debris. The lysate was first passed through a Sepharose 4B bead (Sigma-Aldrich) pre-column followed by the corresponding affinity chromatography column. The affinity column was then washed with 50 mM Tris-HCl, pH 8.0, 1% Triton X-100, 200 mM NaCl followed by a wash with 20 mM Tris-HCl, pH 7.5, 0.1% n-octyl-β-D-glucopyranoside. GPI-ICAM-1 and GPI-GM-CSF were then eluted with 100mM Triethylamine, 1% n-octyl-β-D-glucopyranoside, pH 11.6, whereas GPI-IL-12 was eluted with 100 mM Glycine-HCl pH 2.8 containing 1% n-octyl-β-D-glucopyranoside, and 10 mM sodium iodoacetate. The eluted fractions were subjected to SDS-PAGE and analyzed by western blot and silver staining. The protein containing fractions were combined in a 10–14 kDa MWCO dialysis bag (Fisherbrand) and concentrated using polyvinylpyrrolidone (Sigma-Aldrich). The concentrated proteins were dialyzed with 3 exchanges of 500 ml PBS containing 0.05% n-octyl-β-D-glucopyranoside and stored at 4°C until use. The mAb clones used were A2F17107 (anti-mGM-CSF), YN1/1.7.4 (anti-mICAM-1), and C17.8 (anti-mIL-12 p40). Further detail is provided as supplementary materials. Purified GPI-ICAM-1 concentration was quantified using a micro BCA kit (Thermo Scientific). GPI-GM-CSF and GPI-IL-12 quantifications were performed by direct ELISA comparing to a standard curve obtained using recombinant soluble mGM-CSF (Peprotec) or mIL-12 p75 (eBioscience).

Incorporation of GPI-ISMs onto VLPs by protein transfer

H5 VLPs and H1 VLPs were made using the recombinant baculovirus (rBV) expression system and purified using sucrose gradient centrifugation as previously described (10, 24). rBVs expressing influenza M1 protein from H1N1 influenza A/PR/8/1934 virus and rBVs expressing influenza HA proteins from H5N1 influenza A/Indonesia/05/05 virus or from H1N1 influenza A/PR/8/1934 virus, were used to generate H5 or H1 VLPs, respectively. Purified GPI-ISMs were centrifuged at 13,200 rpm for 1 h at 4°C. Supernatant from the centrifuged GPI-ISM fractions was incubated with H5 VLPs in end-over-end rotation at 37°C for 6 h in the presence of 0.22 μm filtered PBS or PBS/0.1% ovalbumin. GPI-ISM-VLPs were pelleted at 13,200 rpm at 4°C for 30 min to remove unincorporated GPI-ISMs. VLP pellets were resuspended in PBS. Quantification of resulting protein transferred-VLPs was performed by a quantitative ELISA using known concentrations of VLPs as standards and quantification of incorporated GPI-ISMs was determined by quantitative ELISA using known concentrations of recombinant soluble GM-CSF or IL-12 as explained in supplementary materials.

Bone marrow-derived cell proliferation assay

Bone marrow (BM) cells were obtained from tibias and femurs of BALB/c mice and cultured in RPMI, 10% CCS, 1% PenStrep, 50 μM β-mercaptoethanol at 37°C, 5% CO2, as previously described (25). Mouse bone marrow cells (1 × 105) were cultured for 4 days in 100 μl of medium containing 1 μg/ml of protein transferred-VLP per well in triplicate in a 96-well flat bottom plate. Four days after culture, 20 μl of CellTiter 96® Aqueous One Solution (Promega) was added. The plate was placed in a 37°C, 5% CO2 incubator for 1 h and read at 490 nm.

Vaccination and challenge studies

BALB/c mice were immunized on day 0 and boosted on day 36 with 100 μl of 0.5 μg protein transferred-H5 VLPs in PBS on the left hind flank subcutaneously. Control groups received 100 μl PBS or mock protein transferred-H5 VLPs without GPI-ISMs. For influenza challenge, mice were challenged intranasally with influenza A/Vietnam/1203/2004 (rgH5N1), 1LD in 50 μl (11).

To determine viral titer, mice were euthanized 4 days post infection and lungs were individually homogenized. Supernatants of lung homogenates were serially diluted and inoculated into 9–11 day old fertilized eggs. The presence of virus was tested with the allantoic fluid 3 days later by hemagglutination assay. Fifty percent egg infectious dose (EID50) titers were calculated by the Reed and Muench method (26).

To determine the presence of virus-specific antibody secreting cells within different mouse organs after challenge, mice were sacrificed 4 days post challenge with influenza A/Vietnam/1203/2004 (rgH5N1). One day before sacrificing mice, H5 VLP or inactivated influenza virus (2 μg/ml) was used as a coating antigen on ELISA plates with overnight incubation at 4°C. Wells were washed with PBS and blocked with RPMI media containing 10% FBS for 2 h at 37°C. Spleens, lungs, and bone marrows from mice (n = 3) were pooled and homogenized. The homogenate was passed through a cell strainer (Fisher Scientific) and the cells were treated with red blood cell (RBC) lysis buffer (Sigma, St. Louis, MO) to remove RBCs. The collected cells were then added into the plates (0.5 × 106 spleen or bone marrow cells/200 μl/well; 0.2 × 106 lung cells/200 μl/well) and incubated for 1 or 5 days at 37°C, 5% CO2. The supernatant was removed and virus-specific antibody bound to the wells was quantified by ELISA using secondary anti-mouse-IgG-HRP antibody and TMB substrate.

Detection of serum IgG responses

ELISA was used to detect serum anti-viral IgG responses by coating 100 μl per well of 1 μg/ml VLP or inactivated virus to a 96-well flat bottom ELISA plate. Diluted serum was added as primary antibody followed by goat-anti-mouse IgG-HRP, rat-anti-mouse-IgG1-HRP, rat-anti-mouse-IgG2a-HRP, or goat-anti-mouse-IgG2b-HRP (Southern Biotech) as the secondary antibody.

Results

Protein transfer of GPI-ISMs onto influenza VLPs

GPI-ISMs (GPI-ICAM-1, GPI-IL-12, GPI-GM-CSF) were purified from CHO-K1 cell transfectants by affinity chromatography (Supplementary Figs. 1 and 2) and incubated with VLPs. Incorporation levels were directly proportional to the concentrations of GPI-ISMs added to H5 VLPs during protein transfer (Figure 2A). Protein transferred-GPI-GM-CSF constituted 8.9% to 38.5% of total VLP protein depending on the concentration GPI-GM-CSF added during protein transfer as determined by a quantitative ELISA using known concentrations of recombinant soluble GM-CSF as a standard. The kinetic analysis of protein transfer revealed that the level of GPI-ICAM-1 incorporation onto VLPs was dependent on incubation time with saturation reaching within 2 h (Figure 2B). Protein transfer was not limited to H5 VLPs but could also be extended to H1 VLPs derived from the expression of H1N1 influenza A/PR/8/1934 hemagglutinin and matrix 1 proteins (Figure 2C). These results show that the level of incorporation of GPI-ISMs onto influenza VLPs during protein transfer can be controlled by varying the concentration of purified GPI-ISMs and/or time. To determine whether more than one GPI-ISM could be incorporated simultaneously, H5 VLPs were tested for protein transfer with either a fixed concentration of GPI-ICAM-1 and varying concentrations of GPI-IL-12 (Supplemental Figure 3A) or with a fixed concentration of GPI-IL-12 and varying concentrations of GPI-ICAM-1 (Supplemental Figure 3B). Both GPI-ISMs incorporated onto influenza VLPs in a concentration dependent manner after protein transfer, and incorporation of one GPI-ISM did not affect incorporation levels of the other at the tested concentrations.

Figure 2. Protein transfer of GPI-ISMs onto influenza VLPs depends on the GPI-anchor.

Figure 2

Open in a new tab

(A) Concentration dependence of protein transfer. Varying concentrations of GPI-ICAM-1, GPI-IL-12, or GPI-GM-CSF were incubated with H5 VLPs at 37°C for 2 h in PBS/0.01% ovalbumin. The VLPs were washed twice with PBS by centrifugation and the resulting VLPs were analyzed by western blot. (B) Kinetics of protein transfer. H5 VLPs were protein transferred with GPI-ICAM-1 for different time periods at 37°C. ICAM-1 incorporation was analyzed by SDS PAGE and western blot analysis. (C) Protein transfer of GPI-ISMs onto H1 VLPs. GPI-ICAM-1 or GPI-GM-CSF was protein transferred onto H1 VLPs. (D) Effects of PI-PLC treatment. VLPs protein transferred with GPI-ICAM-1 were treated with PI-PLC and analyzed by western blot (top panel). PI-PLC treatment was also performed on GPI-ICAM-1 or GPI-IL-12 before subjecting to protein transfer with H5 VLPs (bottom two panels). (E) Fatty acid binding proteins compete with and inhibit incorporation of GPI-ISMs onto H5 VLPs. During protein transfer, 1% BSA or 1% orosomucoid was incubated with H5 VLPs and GPI-ICAM-1. Due to availability and ease of purification, most of the protein transfer optimization studies were performed with GPI-ICAM-1.

Protein transfer of GPI-ISMs onto influenza VLPs depends on an intact GPI-anchor and does not affect the integrity of VLPs

The GPI-anchor consists of two fatty acid chains that allow for insertion of the GPI-ISMs into the lipid bilayer by protein transfer (27–29). Phosphatidylinositol-specific phospholipase C (PI-PLC) cleaves the GPI-anchor from the ISM leaving the remaining protein domain intact (30). To determine if incorporation of GPI-ISMs onto influenza VLPs by protein transfer is GPI-anchor-dependent, GPI-ICAM-1 was subjected to PI-PLC digestion before or after protein transfer (Figure 2D). PI-PLC treatment decreased the level of VLP-incorporated-GPI-ICAM-1 and also abolished the ability of GPI-ICAM-1 to incorporate onto VLPs by protein transfer. Both of these actions of PI-PLC were blocked by ZnCl2, a potent inhibitor of the PI-PLC enzyme. A similar effect on GPI-IL-12 was observed. Moreover, recombinant soluble IL-12 that lacks a GPI-anchor did not incorporate onto the VLPs upon incubation. When fatty acid binding proteins, BSA or orosomucoid, were added to compete during protein transfer, the level of GPI-ICAM-1-incorporation onto VLPs decreased compared to the control (Figure 2E). These results show that protein transfer mediated incorporation of GPI-ISMs onto VLPs is a specific process dependent on an intact GPI-anchor lipid moiety. Further, neither the structural integrity of the VLPs as assessed by electron microscopy (Supplemental Figure 4A) nor the concentration of influenza-specific proteins on the VLPs (Supplemental Figure 4B) were affected by the protein transfer process. Physicochemical characterization of the VLPs before and after protein transfer showed that the size and zeta potential of VLPs slightly lowered after protein transfer (Supplemental Figure 4C). By the Malvern Zetasizer Nano ZS analysis, the average size of the VLPs was 214.5 ± 38.7 nm before and 191.0 ± 25.8 nm after protein transfer (Supplemental Figure 4C, left panel). Compared to electron microscopy in which VLPs appeared approximately 100 nm in diameter, the size is increased by Zetasizer analysis perhaps due to VLP aggregates. VLP aggregates were also seen by electron microscopy analysis supporting this idea. Further, the average polydispersity index (PDI) of the VLPs was 0.226 ± 0.043 before and 0.454 ± 0.069 after protein transfer with GPI-ICAM-1. The increase in PDI after protein transfer suggests that there is a broader range of particle sizes after protein transfer with GPI-ICAM-1. At present we do not know the reason for this increase in PDI but it is possible that incorporated ICAM-1 may interact with neighboring VLPs inducing aggregation of VLPs, and thus resulting in a polydispersion. On the other hand, the zeta potential measures the net charge around the particles. In general, a larger net charge is associated with more stable particles. After protein transfer, the zeta potential decreased from −10.6 mV to −12.7 mV (Supplemental Figure 4C, right panel) suggesting that incorporation of GPI-ISMs by protein transfer does not destabilize the VLPs. Further, endotoxin levels were undetectable in protein transfer-modified-VLPs that we used for administration (data not shown).

Protein transferred-influenza VLPs retain functional GPI-ISMs and are unaffected by storage

GM-CSF stimulates bone marrow (BM) cell proliferation (12); therefore, we tested whether GPI-GM-CSF-protein transferred-VLPs (GPI-GM-CSF-VLPs) retained ISM-functionality by inducing BM cell proliferation. After 4 days of culture, GPI-GM-CSF-VLPs induced 16-fold enhanced BM cell proliferation as compared to unmodified VLPs (Figure 3A). The co-incorporation of GPI-GM-CSF along with the cell adhesion molecule, GPI-ICAM-1, by protein transfer did not alter proliferation levels (data not shown) suggesting that ICAM-1 interaction with its receptor on BM cells does not further augment GPI-GM-CSF-VLP-induced proliferation. These results demonstrate that protein transferred-GPI-GM-CSF on VLPs retains its activity and is capable of stimulating cellular functions similar to soluble GM-CSF.

Figure 3. Functionality and storage of protein transferred VLPs.

Figure 3

Open in a new tab

(A) VLPs protein transferred with GPI-GM-CSF induce proliferation of BM cells as described in Methods section. The protein transferred GPI-GM-CSF constituted approximately 38.5% of total VLP protein in GPI-GM-CSF-VLPs. (B) Protein transferred VLPs retain incorporated GPI-GM-CSF after storage. VLPs protein transferred with GPI-GM-CSF were stored at 4°C for up to 4 weeks. Relative intensity was calculated by comparing the ratio of GM-CSF-to-VLP proteins at each storage time point to the ratio at day 0. (C) Protein transferred GPI-GM-CSF-VLPs stored for 4 weeks induce BM cell proliferation. BM cells were cultured in triplicate for 4 days with freshly prepared or stored GPI-GM-CSF-VLPs. Proliferation of BM cells was detected by CellTiter 96® Aqueous One Solution. (Statistical analysis: (A) One-way ANOVA – Tukey’s multiple comparisons test, p < 0.0001, (C) multiple t-test; p < 0.0001).

Vaccines often require storage, so we tested the ability of protein transferred-VLPs to retain functional GPI-ISMs after storage at 4°C, at which VLP vaccines remain stable long-term (31). Storage of protein transferred-GPI-GM-CSF-VLPs for 4 weeks at 4°C did not reduce the amount (Figure 3B) or the functionality (Figure 3C) of GM-CSF on the protein transferred-VLPs, as freshly prepared and stored GPI-GM-CSF-VLPs induced similar levels of BM cell proliferation.

Vaccination with protein transferred-influenza VLPs enhances anti-H5N1 viral serum antibody responses

Influenza VLP vaccination induces strong antibody responses against homologous viruses (10, 32, 33); however antibody responses against heterologous influenza viruses are weaker. We hypothesized that incorporating GPI-ISMs onto H5 VLPs by protein transfer would optimally activate immune cells leading to enhanced anti-heterologous viral immune responses. To test the in vivo efficacy of protein transferred influenza H5 VLPs, we analyzed serum antibody responses in vaccinated mice against either H5 VLPs, homologous inactivated H5N1 influenza A/Indonesia/05/2005 virus, or heterologous inactivated H5N1 influenza A/Vietnam/1203/2004 virus by ELISA. Mice vaccinated with GPI-GM-CSF-VLPs demonstrated significantly enhanced (6-fold higher) anti-homologous and anti-heterologous serum IgG levels against inactivated H5N1 virus compared to mice vaccinated with unmodified VLPs (Figure 4A) on d17 after boost. Interestingly, although GPI-ICAM-1 (Supplementary Figure 5) and GPI-IL-12 (data not shown) remained functional in vitro after purification, GPI-ICAM-1-VLPs and GPI-IL-12-VLPs did not augment humoral immunity against virus (Figure 4A). Further, VLPs simultaneously protein transferred with both GPI-GM-CSF and GPI-IL-12 enhanced anti-homologous and anti-heterologous viral IgG similar to GPI-GM-CSF-VLPs (Figure 4A) suggesting that the enhanced antibody response is contributed mostly by GM-CSF. Also, vaccination with 0.5 μg of GPI-GM-CSF-VLPs that express 0.029 μg GM-CSF/μg VLP or 0.089 μg GM-CSF/μg VLP resulted in similar homologous inactivated H5N1 influenza A/Indonesia/05/2005 virus and heterologous inactivated H5N1 influenza A/Vietnam/1203/2004 virus specific IgG responses (Supplemental Figure 6).

Figure 4. Vaccination of mice with H5 VLPs protein transferred with GPI-GM-CSF leads to enhanced anti-homologous and anti-heterologous H5N1 inactivated virus and anti-VLP serum IgG.

Figure 4

Figure 4

Figure 4

Open in a new tab

(A) Anti-homologous H5N1 influenza A/Indonesia/05/2005 and anti-heterologous H5N1 influenza A/Vietnam/1203/2004 specific total IgG in vaccinated mice. Serum was collected from groups of vaccinated mice (n = 5) 17 days after boost. Using 1:1000 diluted serum, an ELISA was performed to detect anti-homologous and anti-heterologous influenza virus specific total IgG. (B) H5 VLP-specific IgG subtype responses upon vaccination with protein transferred GPI-GM-CSF-VLPs. Serum was collected from groups of vaccinated mice (n = 5) 28 days after initial vaccination (left) and 7 days after boost (right). A 1:5000 diluted serum was used in an ELISA against coated H5 VLPs. (C) Inactivated homologous H5N1 influenza A/Indonesia/05/2005 virus-specific serum IgG subtypes. Serum was collected from groups of vaccinated mice (n = 5) 6 weeks after boost and 1:1000 dilution of serum was used in an ELISA against coated inactivated H5N1 influenza A/Indonesia/05/2005 virus. (D) Inactivated heterologous H5N1 influenza A/Vietnam/1203/2004 virus-specific serum IgG subtypes. Serum was collected from vaccinated mice (n = 5) 6 weeks after boost and 1:1000 diluted serum was used in an ELISA against coated inactivated H5N1 influenza A/Vietnam/1203/2004 virus. (Statistical analysis: (A) One-Way ANOVA – Tukey’s multiple comparisons test, (B–D) T-test. * p < 0.05, ** p < 0.01, *** p < 0.001, **** p < 0.0001).

Since humoral immune responses depend on T-cell help, IgG antibody subtype responses were examined to investigate the type of T-cell response elicited by VLP vaccination. IgG2a and IgG2b antibody subtypes correlate with a CD4+ T helper 1 (Th1) response whereas IgG1 correlates with a Th2 response (34). A single vaccination of mice with GPI-GM-CSF-VLPs led to a 3-fold enhancement of anti-VLP-specific IgG compared to unmodified VLPs, however after boost, the fold enhancement narrowed to 1.4 (Figure 4B). GPI-GM-CSF-VLP vaccination enhanced 3-fold anti-VLP-specific IgG2a and anti-H5N1 influenza A/Indonesia/05/05 (homologous) virus-specific IgG2a and IgG2b (Figure 4C) antibody responses as well as increased 2 to 3-fold anti-H5N1 influenza A/Vietnam/1203/2004 (heterologous) virus-specific IgG2a and IgG2b responses (Figure 4D). Interestingly, virus-specific IgG1 antibody responses were also elevated by 2-fold in mice vaccinated with GPI-GM-CSF-VLPs, suggesting an enhancement of both Th1 and Th2 responses in these mice.

VLPs protein transferred with GPI-GM-CSF provide complete protection against a heterologous virus challenge

Vaccination with unmodified VLPs provides protection against challenge from homologous virus (10, 13); however, complete protection is not detected against heterologous viral strains (13). To determine if the enhanced antibody response detected in mice vaccinated with GPI-GM-CSF-VLPs corresponded to better heterologous protection, an intranasal challenge with the heterologous influenza A/Vietnam/1203/2004 (rgH5N1) virus (1LD) was conducted. Mice vaccinated with GPI-GM-CSF-VLPs exhibited significantly minimal changes in body weight (Figure 5A) and complete (100%) protection against a heterologous challenge (Figure 5B), whereas unvaccinated mice or mice vaccinated with unmodified VLPs had marked weight loss resulting in only 60% survival. Mice vaccinated with GPI-GM-CSF-VLPs also displayed a decreased viral titer whereas unmodified VLP vaccinated mice showed a similar lung viral titer as control mice suggesting effective control of viral replication by GPI-GM-CSF-VLPs (Figure 5C).

Figure 5. GPI-GM-CSF-VLP vaccination induces complete protection after a heterologous H5N1 influenza A/Vietnam/1203/2004 virus challenge.

Figure 5

Open in a new tab

Groups of mice (n = 5) were vaccinated and boosted 36 days later with 0.5 μg GPI-GM-CSF-VLPs protein transferred to incorporate 0.089 μg GM-CSF/μg VLP. Mice were then challenged intranasally with heterologous influenza A/Vietnam/1203/2004 (rgH5N1) virus (1LD) and (A) body weight and (B) survival was measured. Mice that displayed <75% of initial body weight were sacrificed according to IACUC regulations. (C) Decreased lung viral titers in mice (n = 3) vaccinated with GPI-GM-CSF-VLPs 4 days post challenge with heterologous rgH5N1. Lungs homogenates were used to determine viral titers using 9–11 day old embryonated eggs. (D) Enhanced generation of virus-specific-IgG antibody responses by cells from spleen, bone marrow, and lungs of GPI-GM-CSF-VLP vaccinated mice. Spleen, bone marrow and lung cells of mice vaccinated with PBS, unmodified VLPs, or GPI-GM-CSF-VLPs were cultured in plates coated with either PBS, H5 VLPs, homologous H5N1 influenza A/Indonesia/05/2005 or heterologous H5N1 influenza A/Vietnam/1203/2004 inactivated virus as antigen and the production of antigen-specific IgG was detected by ELISA. (Statistical analysis: (A) Repeated Measures Two-Way ANOVA, (C) One-Way ANOVA – Dunnett’s multiple comparisons test; * p < 0.05, *** p < 0.001, **** p < 0.0001 (D) Two-Way ANOVA – Tukey’s multiple comparisons test; * p < 0.0001).

To determine if virus-specific long-lived memory B cells were present after vaccination with GPI-GM-CSF-VLPs, mice were challenged with rgH5N1 virus 21 weeks post boost. BM and lung cells derived from mice vaccinated with GPI-GM-CSF-VLPs displayed enhanced antibody production against H5 VLPs, inactivated H5N1 Indonesia virus, and Vietnam virus compared to VLP vaccinated mice after both 1 and 5 days of culture. However, spleen cells demonstrated enhanced anti-viral IgG responses only after 5 days of culture with antigen (Figure 5D).

Discussion

Safety issues, concerns about production in chicken eggs, and difficulty in manufacturing conventional whole inactivated or attenuated influenza virus-based vaccines severely limit the current vaccine production system to meet pandemic threats. Therefore VLP-based vaccines, which are easier to produce in large-scale quantities and safer due to the lack of viral genomes, are considered to be a strong alternative to currently used virus-based vaccines. VLPs, due to their nanoparticulate and repetitive antigenic nature induce a strong immune response to viral proteins upon vaccination (32). However, although VLP-based influenza vaccines lead to complete protection against challenge with homologous virus, protection against heterologous viral strains is weak (13, 35). Herein, we demonstrate a protein transfer approach to adjuvantate enveloped VLPs with ISMs in a controlled manner and within a relatively short amount of time. In this method, a simple incubation of influenza VLPs with purified GPI-ISMs led to spontaneous incorporation of the GPI-ISMs onto the surface of VLPs in a time and concentration dependent manner. This protein transfer method relied on the presence of the GPI-anchor, occurred within two hours, and allowed for more than one GPI-ISM to be incorporated simultaneously onto the VLPs. The incorporated protein was expressed stably, as protein transfer-modified-VLPs can be stored for a month without losing the function of the incorporated molecules.

Our results demonstrated that mice vaccinated with protein transfer-modified-influenza H5 VLPs had enhanced anti-VLP and anti-viral (homologous and heterologous) serum IgG responses compared to unmodified VLP-vaccinated mice. Subtype analysis of anti-viral IgG antibodies showed that vaccination with GPI-GM-CSF-VLPs augmented anti-viral IgG2a and IgG2b responses, suggestive of a protective Th1 phenotype, however anti-viral IgG1 subtype was also elevated demonstrating that the inclusion of GPI-GM-CSF onto these biodegradable nanoparticles enhances both Th1 and Th2-type immunity. Furthermore, this enhancement in antiviral IgG responses correlated with protection against a heterologous viral challenge. Skountzou et al. demonstrated that vaccination with genetically engineered SIV VLPs expressing GPI-GM-CSF leads to enhanced SIV-specific IgG and neutralizing antibodies compared to VLPs administered with similar amounts of recombinant soluble GM-CSF (12). This suggests that augmentation of the immune response by protein transferred GPI-GM-CSF-VLPs could be due to administering cytokines physically attached to the nanoparticulate VLP surface, which allows for localized effects of the cytokine that can further aid in activating an immune response against VLP-associated antigens. Cytokine immobilization onto the VLPs not only reduces systemic toxicity associated with administered soluble cytokines, but also may allow for a slow release depot of the cytokine at the vaccination site thus increasing the cytokine half-life. Furthermore, because the receptor for GM-CSF is expressed on antigen presenting cells, such as DCs and B cells, engulfment of VLPs by phagocytosis or micropinocytosis (36) and activation of these cells could also be further enhanced by the presence of GM-CSF on the VLP surface.

The advantage of protein transfer to modify VLPs over gene transfer is the rapid process and that the level of incorporation can be easily controlled. Previous studies show that GPI-GM-CSF or CD40L incorporation onto SIV VLPs by gene transfer resulted in only 0.1% or 0.14% expression of total VLP proteins, respectively (12), however by protein transfer we are able to achieve GPI-GM-CSF expression up to 38.5% under the conditions that we tested. We also found that vaccination with protein transfer-modified VLPs expressing 0.089 μg GM-CSF/μg VLP or 0.029 μg GM-CSF/μg VLP led to similar anti-viral IgG responses suggesting that low doses of GM-CSF incorporation can also induce anti-viral immune responses. Further, incorporating GPI-GM-CSF onto VLPs enhances immunity against viral-proteins, thereby decreasing the VLP dosage required for protection. This dose sparing outcome will increase vaccine availability and reduce production costs, which will be highly advantageous when quick large-scale vaccine production is required under the pressure of pandemic threats.

This method of incorporating novel antigens onto VLPs by protein transfer has the potential to be expanded to include additional viral proteins onto the VLPs. For example, cross protection against different viral influenza strains can be induced by protein transferring different viral strains of GPI-anchored-HA onto VLPs. Recent studies have shown that a higher density of antigen in VLPs increases antigen-specific immune responses (2, 37–40). Therefore, similarly the density of endogenously expressed HA antigen on the VLPs can be further increased by protein transfer of exogenously added GPI-HA for a better immune response.

It is well established that recombinant VLPs are easier to produce in large quantities and are as effective as currently used virus-based influenza vaccines produced with egg-based technologies. Also, production of influenza VLPs in insect cells does not have the safety concerns over handling highly pathogenic avian influenza viruses that are often lethal to embryos and can be a potential problem in the traditional egg-grown vaccine technologies. Our approach of adjuvanting VLPs with GPI-ISMs described in the current work further enhances the immunogenicity and breadth of the immune response and thus adds another layer of advantage to VLP-based influenza vaccines. In addition, purified GPI-ISM or GPI-HA stock preparations can be easily coupled with influenza VLP vaccines by protein transfer, especially with the possibility of a potential pandemic at any time regardless of season.

In summary, we have demonstrated that protein transfer of GPI-ISMs onto nanoparticulate influenza VLPs leads to stable incorporation of the GPI-ISMs on the VLPs in a time and concentration dependent manner, and the incorporated GPI-ISM remains functional even after storage. Moreover, we have demonstrated the enhanced efficacy of protein transfer-modified-VLP vaccines in an influenza viral model. Therefore, protein transfer of GPI-ISMs onto VLPs provides an easy means to adjuvantate the immunogenicity of viral proteins expressed on the VLPs and thus allows for the development of effective enveloped VLP-based vaccines.

Supplementary Material

supplement

Acknowledgments

Statement of Funding: This work was supported in part by NIH/NIAID grants 1 R01 CA138993-01A1 (P.S.), 1 F31 CA165632-01 (J.M.P.) and R01 AI093772 (S.M.K.) and R01 AI105170 (S.M.K.).

We thank Hong Yi at the Emory University Robert P. Apkarian Integrated Electron Microscopy Core for assistance with TEM, and we thank Erica Bozeman and Archana Boopathy for critical reading of the manuscript.

Footnotes

Conflict of Interest: Dr. Selvaraj is a co-founder & equity holder of Metaclipse Therapeutics Corporation - a startup company formed to develop therapeutic cancer vaccines for humans using the protein transfer technology described here for which he is a co-inventor.

Part of the work was presented as a poster at The American Association of Immunologists 2011, 2012, and 2013.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • 1.Kirchenbaum GA, Ross TM. Eliciting broadly protective antibody responses against influenza. Current opinion in immunology. 2014;28:71–6. doi: 10.1016/j.coi.2014.02.005. [DOI] [PubMed] [Google Scholar]
  • 2.Ulmer JB, Valley U, Rappuoli R. Vaccine manufacturing: challenges and solutions. Nature biotechnology. 2006;24(11):1377–83. doi: 10.1038/nbt1261. [DOI] [PubMed] [Google Scholar]
  • 3.Kang SM, Song JM, Quan FS, Compans RW. Influenza vaccines based on virus-like particles. Virus research. 2009;143(2):140–6. doi: 10.1016/j.virusres.2009.04.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Lopez-Macias C. Virus-like particle (VLP)-based vaccines for pandemic influenza: performance of a VLP vaccine during the 2009 influenza pandemic. Human vaccines & immunotherapeutics. 2012;8(3):411–4. doi: 10.4161/hv.18757. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Roldao A, Mellado MC, Castilho LR, Carrondo MJ, Alves PM. Virus-like particles in vaccine development. Expert review of vaccines. 2010;9(10):1149–76. doi: 10.1586/erv.10.115. [DOI] [PubMed] [Google Scholar]
  • 6.Grgacic EV, Anderson DA. Virus-like particles: passport to immune recognition. Methods. 2006;40(1):60–5. doi: 10.1016/j.ymeth.2006.07.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Cubas R, Zhang S, Kwon S, Sevick-Muraca EM, Li M, Chen C, et al. Virus-like particle (VLP) lymphatic trafficking and immune response generation after immunization by different routes. Journal of immunotherapy. 2009;32(2):118–28. doi: 10.1097/CJI.0b013e31818f13c4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Manolova V, Flace A, Bauer M, Schwarz K, Saudan P, Bachmann MF. Nanoparticles target distinct dendritic cell populations according to their size. European journal of immunology. 2008;38(5):1404–13. doi: 10.1002/eji.200737984. [DOI] [PubMed] [Google Scholar]
  • 9.Bright RA, Carter DM, Daniluk S, Toapanta FR, Ahmad A, Gavrilov V, et al. Influenza virus-like particles elicit broader immune responses than whole virion inactivated influenza virus or recombinant hemagglutinin. Vaccine. 2007;25(19):3871–8. doi: 10.1016/j.vaccine.2007.01.106. [DOI] [PubMed] [Google Scholar]
  • 10.Kang SM, Yoo DG, Lipatov AS, Song JM, Davis CT, Quan FS, et al. Induction of long-term protective immune responses by influenza H5N1 virus-like particles. PloS one. 2009;4(3):e4667. doi: 10.1371/journal.pone.0004667. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Song JM, Hossain J, Yoo DG, Lipatov AS, Davis CT, Quan FS, et al. Protective immunity against H5N1 influenza virus by a single dose vaccination with virus-like particles. Virology. 2010;405(1):165–75. doi: 10.1016/j.virol.2010.05.034. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Skountzou I, Quan FS, Gangadhara S, Ye L, Vzorov A, Selvaraj P, et al. Incorporation of glycosylphosphatidylinositol-anchored granulocyte- macrophage colony-stimulating factor or CD40 ligand enhances immunogenicity of chimeric simian immunodeficiency virus-like particles. Journal of virology. 2007;81(3):1083–94. doi: 10.1128/JVI.01692-06. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Wang BZ, Quan FS, Kang SM, Bozja J, Skountzou I, Compans RW. Incorporation of membrane-anchored flagellin into influenza virus-like particles enhances the breadth of immune responses. Journal of virology. 2008;82(23):11813–23. doi: 10.1128/JVI.01076-08. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Zhang R, Zhang S, Li M, Chen C, Yao Q. Incorporation of CD40 ligand into SHIV virus-like particles (VLP) enhances SHIV-VLP-induced dendritic cell activation and boosts immune responses against HIV. Vaccine. 2010;28(31):5114–27. doi: 10.1016/j.vaccine.2010.03.079. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Kaczmarczyk SJ, Sitaraman K, Young HA, Hughes SH, Chatterjee DK. Protein delivery using engineered virus-like particles. Proceedings of the National Academy of Sciences of the United States of America. 2011;108(41):16998–7003. doi: 10.1073/pnas.1101874108. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Ranka R, Petrovskis I, Sominskaya I, Bogans J, Bruvere R, Akopjana I, et al. Fibronectin-binding nanoparticles for intracellular targeting addressed by B. burgdorferi BBK32 protein fragments. Nanomedicine : nanotechnology, biology, and medicine. 2013;9(1):65–73. doi: 10.1016/j.nano.2012.05.003. [DOI] [PubMed] [Google Scholar]
  • 17.Haynes JR. Influenza virus-like particle vaccines. Expert review of vaccines. 2009;8(4):435–45. doi: 10.1586/erv.09.8. [DOI] [PubMed] [Google Scholar]
  • 18.Nagarajan S, Selvaraj P. Glycolipid-anchored IL-12 expressed on tumor cell surface induces antitumor immune response. Cancer research. 2002;62(10):2869–74. [PubMed] [Google Scholar]
  • 19.Poloso NJ, Nagarajan S, Bumgarner GW, Selvaraj P. Development of therapeutic vaccines by direct modification of cell membranes from surgically removed human tumor tissue with immunostimulatory molecules. Vaccine. 2001;19(15–16):2029–38. doi: 10.1016/s0264-410x(00)00424-2. [DOI] [PubMed] [Google Scholar]
  • 20.Poloso NJ, Nagarajan S, Mejia-Oneta JM, Selvaraj P. GPI-anchoring of GM-CSF results in active membrane-bound and partially shed cytokine. Molecular immunology. 2002;38(11):803–16. doi: 10.1016/s0161-5890(02)00005-6. [DOI] [PubMed] [Google Scholar]
  • 21.Carpen O, Pallai P, Staunton DE, Springer TA. Association of intercellular adhesion molecule-1 (ICAM-1) with actin-containing cytoskeleton and alpha-actinin. The Journal of cell biology. 1992;118(5):1223–34. doi: 10.1083/jcb.118.5.1223. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.McHugh RS, Ahmed SN, Wang YC, Sell KW, Selvaraj P. Construction, purification, and functional incorporation on tumor cells of glycolipid-anchored human B7-1 (CD80) Proceedings of the National Academy of Sciences of the United States of America. 1995;92(17):8059–63. doi: 10.1073/pnas.92.17.8059. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Nagarajan S, Selvaraj P. Human tumor membrane vesicles modified to express glycolipid-anchored IL-12 by protein transfer induce T cell proliferation in vitro: a potential approach for local delivery of cytokines during vaccination. Vaccine. 2006;24(13):2264–74. doi: 10.1016/j.vaccine.2005.11.045. [DOI] [PubMed] [Google Scholar]
  • 24.Quan FS, Sailaja G, Skountzou I, Huang C, Vzorov A, Compans RW, et al. Immunogenicity of virus-like particles containing modified human immunodeficiency virus envelope proteins. Vaccine. 2007;25(19):3841–50. doi: 10.1016/j.vaccine.2007.01.107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Lutz MB, Kukutsch N, Ogilvie AL, Rossner S, Koch F, Romani N, et al. An advanced culture method for generating large quantities of highly pure dendritic cells from mouse bone marrow. Journal of immunological methods. 1999;223(1):77–92. doi: 10.1016/s0022-1759(98)00204-x. [DOI] [PubMed] [Google Scholar]
  • 26.Reed LJ, Muench H. A simple method for estimating fifty percent endpoints. Am J Hyg. 1938;27:493–7. [Google Scholar]
  • 27.Cimino AM, Palaniswami P, Kim AC, Selvaraj P. Cancer vaccine development: protein transfer of membrane-anchored cytokines and immunostimulatory molecules. Immunologic research. 2004;29(1–3):231–40. doi: 10.1385/IR:29:1-3:231. [DOI] [PubMed] [Google Scholar]
  • 28.Poloso N, Nagarajan S, Bumgarner GW, Zampell JC, Selvaraj P. Designer cancer vaccines made easy: protein transfer of immunostimulatory molecules for use in therapeutic tumor vaccines. Frontiers in bioscience : a journal and virtual library. 2001;6:D760–75. doi: 10.2741/poloso. [DOI] [PubMed] [Google Scholar]
  • 29.Patel JMV, VF, Selvaraj P. Lipid-Mediated Cell Surface Engineering. In: Zhao JKW, editor. Micro- and Nanoengineering of the Cell Surface Micro and Nano Technologies. USA: Elsevier Inc; 2014. pp. 121–41. [Google Scholar]
  • 30.Low MG, Saltiel AR. Structural and functional roles of glycosyl-phosphatidylinositol in membranes. Science. 1988;239(4837):268–75. doi: 10.1126/science.3276003. [DOI] [PubMed] [Google Scholar]
  • 31.Le Tallec D, Doucet D, Elouahabi A, Harvengt P, Deschuyteneer M, Deschamps M. Cervarix, the GSK HPV-16/HPV-18 AS04-adjuvanted cervical cancer vaccine, demonstrates stability upon long-term storage and under simulated cold chain break conditions. Human vaccines. 2009;5(7):467–74. doi: 10.4161/hv.8485. [DOI] [PubMed] [Google Scholar]
  • 32.Chen Q, Lai H. Plant-derived virus-like particles as vaccines. Human vaccines & immunotherapeutics. 2013;9(1):26–49. doi: 10.4161/hv.22218. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Ludwig C, Wagner R. Virus-like particles-universal molecular toolboxes. Current opinion in biotechnology. 2007;18(6):537–45. doi: 10.1016/j.copbio.2007.10.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Lefeber DJ, Benaissa-Trouw B, Vliegenthart JF, Kamerling JP, Jansen WT, Kraaijeveld K, et al. Th1-directing adjuvants increase the immunogenicity of oligosaccharide-protein conjugate vaccines related to Streptococcus pneumoniae type 3. Infection and immunity. 2003;71(12):6915–20. doi: 10.1128/IAI.71.12.6915-6920.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Kodihalli S, Kobasa DL, Webster RG. Strategies for inducing protection against avian influenza A virus subtypes with DNA vaccines. Vaccine. 2000;18(23):2592–9. doi: 10.1016/s0264-410x(99)00485-5. [DOI] [PubMed] [Google Scholar]
  • 36.Win SJ, Ward VK, Dunbar PR, Young SL, Baird MA. Cross-presentation of epitopes on virus-like particles via the MHC I receptor recycling pathway. Immunology and cell biology. 2011;89(6):681–8. doi: 10.1038/icb.2010.161. [DOI] [PubMed] [Google Scholar]
  • 37.El Sahly HM, Davis C, Kotloff K, Meier J, Winokur PL, Wald A, et al. Higher antigen content improves the immune response to 2009 H1N1 influenza vaccine in HIV-infected adults: a randomized clinical trial. The Journal of infectious diseases. 2012;205(5):703–12. doi: 10.1093/infdis/jir837. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Hendrikx LH, Berbers GA, Veenhoven RH, Sanders EA, Buisman AM. IgG responses after booster vaccination with different pertussis vaccines in Dutch children 4 years of age: effect of vaccine antigen content. Vaccine. 2009;27(47):6530–6. doi: 10.1016/j.vaccine.2009.08.052. [DOI] [PubMed] [Google Scholar]
  • 39.Jing X, Phy K, Li X, Ye Z. Increased hemagglutinin content in a reassortant 2009 pandemic H1N1 influenza virus with chimeric neuraminidase containing donor A/Puerto Rico/8/34 virus transmembrane and stalk domains. Vaccine. 2012;30(28):4144–52. doi: 10.1016/j.vaccine.2012.04.073. [DOI] [PubMed] [Google Scholar]
  • 40.Wong SS, Webby RJ. Traditional and new influenza vaccines. Clinical microbiology reviews. 2013;26(3):476–92. doi: 10.1128/CMR.00097-12. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

supplement
NIHMS670595-supplement.pdf (670.9KB, pdf)

RESOURCES