Skip to main content
The AAPS Journal logoLink to The AAPS Journal
. 2010 Sep 28;12(4):699–707. doi: 10.1208/s12248-010-9229-6

Pulmonary Immunization of Guinea Pigs with Diphtheria CRM-197 Antigen as Nanoparticle Aggregate Dry Powders Enhance Local and Systemic Immune Responses

Pavan Muttil 1,5, Brian Pulliam 2,3, Lucila Garcia-Contreras 1, John Kevin Fallon 1, Chenchen Wang 1, Anthony James Hickey 1,3,, David A Edwards 2,3,4
PMCID: PMC2976983  PMID: 20878294

Abstract

This study establishes the immune response elicited in guinea pigs after pulmonary and parenteral immunizations with diphtheria CRM-197 antigen (CrmAg). Several spray-dried powders of formalin-treated/untreated CrmAg nanoaggregates with L-leucine were delivered to the lungs of guinea pigs. A control group consisting of alum with adsorbed CrmAg in saline was administered by intramuscular injection. Animals received three doses of powder vaccines containing 20 or 40 μg of CrmAg. The serum IgG titers were measured for 16 weeks after the initial immunization; IgA titers were measured at the time of sacrifice in the broncho-alveolar lavage fluid. Further, toxin neutralization tests in naïve guinea pigs were performed for a few select serum samples. Histopathology of the lung tissues was conducted to evaluate inflammation or injury to the lung tissues. While the highest titer of serum IgG antibody was observed in guinea pigs immunized by the intramuscular route, those animals immunized with dry powder formulation by the pulmonary route, and without the adjuvant alum, exhibited high IgA titers. A pulmonary administered dry powder, compared to parenteral immunization, conferred complete protection in the toxin neutralization test. Mild inflammation was observed in lung tissues of animals receiving dry powder vaccines by the pulmonary route. Thus, administering novel CrmAg as dry powders to the lungs may be able to overcome some of the disadvantages observed with the existing diphtheria vaccine which is administered by the parenteral route. In addition, these powders will have the advantage of eliciting a high mucosal immune response in the lungs without using traditional adjuvants.

Key words: CRM-197, diphtheria vaccine, dry powder formulation, guinea pigs, pulmonary delivery

INTRODUCTION

Diphtheria is a highly contagious upper respiratory tract disease which can spread from person-to-person by respiratory droplets through coughing and sneezing, showing the importance of mucosal colonization in transmission and initiation of the disease. It is essential to induce an effective immune response at these mucosal surfaces, through which pathogens enter, in addition to stimulating the systemic immunity. Diphtheria vaccine is administered as a suspension by the intramuscular (IM) route as part of the mass immunization program. The use of the IM route is accompanied by the problem of contamination and reuse of needles, a serious concern in many parts of the world where mass immunizations take place (1). Also, the probability of a local reaction at the site of injection is pronounced when multiple vaccines are administered simultaneously (2). The current diphtheria vaccine consists of four to five doses administered during infancy, followed by boosters every 10 years for life; the waning of immunity with age without constant re-exposure is a concern for the existing vaccine. Over the next few years, it is anticipated that the use of needles and syringes will wane in favor of alternatives (3). A non-injectable route and means of vaccine administration for mass immunization would likely improve safety and possibly efficacy of diphtheria vaccination.

The commercial diphtheria vaccine consists of toxin treated with formalin to mitigate potential toxicity and retain the immunogenicity of the native form of the antigen (4). Extrinsic factors such as freezing of the vaccine have been associated with decreased immune response; freezing dissociates the antigen from the alum, and, thus, interferes with the vaccine’s immunogenicity. None of the currently licensed adjuvants intended for use in humans are suitable for mucosal immunization (5).

The existing vaccine administered by the parenteral route usually fails to induce mucosal immunity which is desirable for reduction in disease dissemination (6). Mucosal IgA antibody would be advantageous especially in the case of diphtheria, as the antigen-specific IgA can bind to the exotoxin released by Corynebacterium diphtheriae, preventing it from entering and colonizing the mucosal membrane. Non-invasive vaccination does not require trained medical personnel, or needles and syringes, and could be an attractive route of delivery for mass immunization in developing countries (7). Besides, the development of a heat-stable dry powder vaccine formulation may help overcome the low temperature storage and distribution requirements for liquid vaccine formulations.

Antigen-presenting cells (APCs) such as alveolar macrophages and dendritic cells are located in the lungs for antigen sampling and subsequent presentation to T-cells, making pulmonary delivery of vaccines desirable (810). Increased vaccine efficacy when delivered by the pulmonary route may be explained by the prolonged residence time of the antigen in the alveoli and its subsequent migration to lymphoid tissues. This contrasts with vaccines administered by other routes where the antigen is transiently present before it is cleared from the body.

Given the possibility of an immunotoxicological response associated with the delivery of traditional adjuvants (alum is not approved for pulmonary delivery in humans) to the lungs, pulmonary formulation of vaccines must achieve adjuvancy by a more specific or localized mechanism, amplifying antigenicity while avoiding severe side-effects. Dry powder forms of vaccine for pulmonary delivery have been used in humans and animal models of disease for prevention of, for example, tuberculosis, influenza, diphtheria, hepatitis B, and measles (1114). One potential alternative approach is preparation of particles composed of biodegradable polymer, which can be used to encapsulate the vaccine. The preparation involves spray-drying poly(lactic-co-glycolic acid) (PLGA) nanoparticles containing CRM-197 antigen (CrmAg) with l-leucine to obtain the dry powder formulations. The selection of PLGA polymer is based on its extensive application in drug and vaccine delivery to the lung. Incorporation of l-leucine in the aerosol formulation during spray drying affects the surface properties of PLGA particles leading to improvement in powder flowability and aerosolization properties (15,16). This would in turn increase the respirable fraction of the total dose. The resulting nanoparticles containing the antigen need to form aggregates with a mass median aerodynamic diameter (MMAD) between 1 and 5 μm for effective delivery to the lungs, a challenge that can potentially be met using porous nanoparticle aggregate particle (PNAP) technology as previously developed (17,18).

The aim of this study was to deliver diphtheria vaccine as a dry powder for mass vaccination by the pulmonary route which has the potential to address the following critical issues: the need to increase efficacy that confers better local mucosal as well as systemic immunity; the need to demonstrate safety during administration by reducing the risk of contamination by sharps and needles; the need to eliminate powder reconstitution, thus increasing stability during transport, storage, and administration; and the need to improve cost effectiveness.

CrmAg is a non-toxic mutant of diphtheria toxin that cross-reacts immunologically with the toxin. It differs from diphtheria toxin by a single glycine to glutamic acid change in position 52 which makes it enzymatically inactive while retaining protective antibody titers against diphtheria (19); it binds more strongly to the lipid bilayer of the cell membrane than the diphtheria toxoid (20). After stabilization with formalin, CrmAg becomes immunogenic, and is able to induce protective antibody titers against diphtheria (21). Several studies have attempted to exploit this advantage by administering CrmAg to humans and animal models by the intranasal route for protection against diphtheria (2226). However, there are no published clinical or experimental data for pulmonary vaccination with CrmAg encapsulated in a dry powder formulation in which a series of immunizations were performed. Amidi et al. (12) examined the immune response generated after intra-tracheal administration of diphtheria toxoid encapsulated in chitosan-based polymers in guinea pigs. However, their study did not examine the effect of multiple doses of diphtheria vaccine on the immunity developed against the toxoid. This approach is important as new vaccines for diphtheria should induce not only high antibody titers but also maintain those high levels for prolonged periods of time.

This study consisted of administering three doses of CrmAg with PLGA to guinea pigs and examining the acquired immune response for 16 weeks. The immunogenicity of pulmonary vaccination of guinea pigs with CrmAg, delivered in dry powder forms, and determined by the IgG and IgA responses were addressed. Formulations were administered at different times and blood was withdrawn at regular intervals for estimation of serum IgG antibody titers. Lung IgA antibody titers were evaluated at the time of sacrifice in the broncho-alveolar lavage fluid (BAL) of the guinea pigs. Also, toxin-neutralizing antibody titers produced against the toxin were determined in a guinea pig passive challenge model.

MATERIALS AND METHODS

Materials

Goat anti-guinea pig IgG peroxidase conjugate was purchased from Sigma-Aldrich (St. Louis, MO, USA) and sheep anti-guinea pig IgA peroxidase conjugate procured from ICL Inc. for the in vivo enzyme-linked immunosorbent assay (ELISA). Diphtheria toxin (lot 35119) and antitoxin were procured from the Office of Vaccines Research and Review, CBER, FDA. All other chemicals used were of analytical grade.

Methods

PLGA nanoparticles were prepared by the emulsification solvent diffusion method (27) and the double-emulsion method (28). The particle sizes as measured by photon correlation spectroscopy were 200 ± 50 nm for all PLGA nanoparticles. The entrapment efficiency of CrmAg within the PLGA nanoparticles taking into account a theoretical loading of 1–2% was 87 ± 5% as determined by the bicinchoninic acid assay based on the non-entrapped amount of antigen in the supernatant.

Dry powder formulations with excellent aerosolization properties for pulmonary delivery were obtained by spray drying. Aqueous suspensions of CrmAg encapsulated in PLGA nanoparticles were spray dried (Niro atomizer, Columbia, MD, USA) with a solution of l-leucine. The ratio of nanoparticles to l-leucine was kept at 25/75. The inlet and outlet temperatures of the spray drier were maintained at 95°C and 38°C, respectively. The feed-rate of the solution was 30 mL/min and the air-flow rate 100 kg/h. The different forms of CrmAg spray dried with l leucine are given in Table I

Table I.

The Composition of the CRM-197 (CrmAg) Dry Powder Vaccines for Diphtheria, Their Dose, and Route of Administration (n = 6 Guinea Pigs per Group)

Nomenclature Composition Antigen loading (μg) Route of administration
1. Low-dose formalin-treated antigen and nanoparticle admixture Formalin-treated CrmAg in a physical mixture with PLGA nanoparticles 20 Insufflation
2. High-dose formalin-treated antigen and nanoparticle admixture Formalin-treated CrmAg in a physical mixture with PLGA nanoparticles 40 Insufflation
3. Low-dose formalin-treated antigen in nanoparticles Formalin-treated CrmAg encapsulated within PLGA nanoparticles 20 Insufflation
4. High dose-formalin treated antigen in nanoparticles Formalin-treated CrmAg encapsulated within PLGA nanoparticles 40 Insufflation
5. Low dose-native antigen in nanoparticles CrmAg encapsulated within PLGA nanoparticles 20 Insufflation
6. High dose-native antigen in nanoparticles CrmAg encapsulated within PLGA nanoparticles 40 Insufflation
7. Low dose-alum with adsorbed antigen Alum with adsorbed Ag 20 Intramuscular

The volume median diameter of the spray-dried powders as measured by laser-light diffraction (Sympatec, Germany) with powder dispersion (Rodos, Sympatec, Germany) was ∼7.0 μm. The MMAD and fine particle fractions determined by the Andersen eight stage non-viable cascade impactor was ∼5.0 μm and 50%, respectively, based on a particle size cut-off of <5.8 μm. Tap density was obtained using a Varian tap density machine by measuring the volume of a known amount of powder after tapping it for 500 times and was between 0.11 and 0.17 g/cm3 for all the spray-dried powders. Water content of the powders as measured by Karl Fisher method was less than 1%.

Scanning electron microscope was used to elucidate the structures of the PLGA nanoparticles after spray drying with l-leucine. Hollow- and low-density particles with a rough inner surface probably due to the presence of intact nanoparticles were obtained when PLGA nanoparticles were spray dried.

A Western blot analysis was carried out on the antigen obtained from the spray-dried PLGA particles to ensure that epitopes present on CrmAg were preserved. Results showed that antigenic epitopes were preserved during the spray-drying process. The various dry powder vaccines were stored in a desiccator at 4°C in-between dosing schedules to prevent moisture uptake and antigen degradation.

Vaccination and Immunization Schedule

All animal procedures were approved by the University of North Carolina at Chapel Hill Institutional Animal Care and Use Committee. Male guinea pigs weighing between 240 and 280 g (Hilltop Laboratory Animals, Inc., Scottdale, PA, USA) were housed in a 12-h light/12-h dark cycle and constant temperature of 22°C. A standard diet and water were supplied ad libitum. Animals were randomly assigned to seven groups (n = 6 each) and immunized with novel CrmAg formulations administered as dry powders as shown in Table I. Table I shows the various formulations administered to guinea pigs, their composition, antigen loading, and route of administration. Guinea pigs were anesthetized with ketamine, xylazine, and acepromazine (3:1:0.5) administered by the subcutaneous route in the nape of the neck. Dry powder formulations containing CrmAg were administered to anesthetized animals by insufflation. Approximately 10 mg of the powder, corresponding to 20 or 40 μg of CrmAg (Table I), were administered with a DP-4 insufflator (Penn Century, Inc. Philadelphia, PA, USA). Briefly, anesthetized guinea pigs were placed on their abdomen on a flat surface and the tracheal opening located by inserting a fiber optic laryngoscope (Fiber-Lite system 181-1 Dolan-Jenner Industries, Inc., Woburn, MA, USA) into the mouth of the animal. The DP-4 insufflator was loaded with the powder and inserted into the tracheal opening until the insufflator reached a few millimeters from the carina of the lung for proper aerosolization of the powder. The dry powder was delivered into the lungs of the guinea pigs by pumping air with a 5-mL disposable plastic syringe attached to the other end of the insufflator. The delivered dose was determined by subtracting the weight of the insufflator after powder administration from that of the insufflator before administration. The loss of powder adhering inside the insufflator was ≤1 mg; consequently, the dose +1 mg was loaded to compensate this loss. IM immunization for the control group was performed in the calf muscle of anesthetized guinea pigs with the antigens suspended in sterile phosphate-buffered saline; a volume of 100 μl was injected. In all cases, the immune responses were followed for 16 weeks after the first immunization.

Measurement of In vivo Antibody Response

ELISA

Blood was collected at regular intervals, from anesthetized animals by the saphenous vein, and serum recovered by centrifugation; final bleeding was carried out at week 16 after the first immunization. After the final bleed, BAL was performed on the guinea pigs. Sera and lavage fluids were stored in aliquots at −80°C prior to analysis and frozen samples thawed only once before analysis.

Anti-CrmAg titer was measured in pooled and individual serum samples by indirect ELISA. Briefly, 96 well flat-bottom immuno plates (MaxiSorp, Nalge NUNC International, Rochester, NY, USA) were coated with CrmAg in 50 μl/well of coating buffer (50 mM carbonate buffer, pH 9.6) at a concentration of 2.5 μg/mL at 4°C overnight. The plates were washed four times with wash buffer (PBS, 0.05% Tween 20 [Sigma, St. Louis, MO, USA]) and blocked to prevent non-specific binding at 37°C for 2 h with 150 μl/well blocking buffer (PBS, 0.05% Tween 20, 1% bovine serum albumin). The blocking buffer was aspirated and the plates washed four times with wash buffer. Pooled and individual serum samples diluted in blocking buffer were added to the plates at a starting dilution of 1:200 and further diluted twofold across the plates. The plates were incubated at 37°C for 1 h with 50 μl/well of each serum dilution. After aspirating the serum dilutions and washing four times, the plates were incubated for another 1 h at 37°C with 50 μl/well of anti-guinea pig IgG peroxidase conjugate diluted 1/8,000 in blocking buffer. After aspirating the conjugate and performing the final 4 washes, the plates were developed with the substrate o-phenylenediamine (OPD, Sigma, Saint Louis, MO, USA) for 30 min at 37°C in the dark. The substrate consisted of one tablet (10 mg) of OPD, 20 mL of citrate buffer (50 mM, pH 4.9), and 15 μl of 30% H2O2. The reaction was stopped after 30 min by adding 50 μl/well of 3 M HCL solution to the plates and optical density measured at 485 nm with 570 nm as the reference wavelength (Plate CHAMELEON, Multilabel Detection Platform, Hidex). Relative IgG antibody titers were determined by comparison to the curve generated with sera from alum adsorbed with antigen-immunized guinea pigs and expressed as the reciprocal of the endpoint dilution which yielded an absorbance value at least three times the background levels. All samples were analyzed in duplicate.

IgA antibody titers were evaluated in the BAL fluid at the end of the study (16 weeks post-first immunization) when the guinea pigs were sacrificed. Guinea pigs were anesthetized as described earlier and the trachea isolated by a midline incision. Animals were tracheostomized, and a 14-gauge stainless steel catheter was secured with a suture. The animals were exsanguinated, and the lungs gently perfused three times with 5 mL of saline at room temperature. Recovery volume was ≥80%. The lavage fluid was centrifuged and the saline supernatant was assayed for IgA antibody titers by ELISA. The protocol followed was the same as for the evaluation of IgG titers, except that the anti-guinea pig IgG secondary antibody was replaced with sheep anti-guinea pig IgA peroxidase conjugate at a dilution of 1/5,000 in blocking buffer. Non-diluted BAL samples from individual guinea pigs were added to the plates and diluted twofold across the plates as before.

Toxin Neutralization Assay

The in vivo toxin-neutralizing test was examined on a few select serum samples obtained from vaccinated guinea pigs for specific anti-diphtheria toxin-neutralizing antibodies as described previously (29). The pooled serum samples were obtained from guinea pigs vaccinated with low-dose formalin-treated antigen and nanoparticle admixture, and high-dose formalin-treated antigen in nanoparticles, dry powders by the pulmonary route, and the alum adsorbed with antigen group administered by the IM route. These serum samples were chosen based on their high IgG antibody titers obtained after performing ELISA; only three groups were selected due to the high cost and endpoint sacrifice of the animals in this assay. The toxin-neutralizing test requires L+ control toxin dose and is defined as the minimum amount of diphtheria toxin (0.450 mg/mL) which when mixed with one international unit (IU) of diphtheria antitoxin kills a 240–280 g guinea pig in greater than 48 h and less than 96 h; the L+ dose of toxin is equivalent to 1.67 limes flocculation and was verified in four guinea pigs. The diphtheria toxin-specific antitoxin levels in the serum of vaccinated guinea pigs were tested at three levels equivalent to 2.0, 1.0, and 0.5 IU/mL; two naïve guinea pigs were used at each level. Briefly, 2.0 mL of serum from each of the four immunized guinea pigs was pooled. The serum was stored at −80°C for later use and was not thawed more than once.

The preparation of the diphtheria toxin–antitoxin mixture is given in Table II. The concentration of the test dose of diphtheria toxin was 0.91 mg/mL in saline-G (saline containing 2% gelatin). The stock diphtheria antitoxin (6 units/mL in glycerin) was diluted to 1 U/mL in saline-G. The preparation of the L+ control, negative control and the three values of 0.5, 1.0, and 2.0 IU of antitoxin/mL were tested in naïve guinea pigs (female) and are reported in Table II. The toxin–antitoxin mixtures were mixed thoroughly and allowed to stand at room temperature for 1 h before injecting into guinea pigs. The mixtures were injected into guinea pigs in the order mixed to prevent degradation due to storage. Of the mixture, 3 mL was injected into each guinea pig subcutaneously.

Table II.

Preparation of the Diphtheria Toxin–Antitoxin Mixture Tested in Naïve Guinea Pigs at Three Levels Equivalent to 2.0, 1.0, and 0.5 International Unit (IU) of Diphtheria Antitoxin/mL solution (n = 4 for L + control and n = 2 for all other Groups)

Groups Saline Ga (mL) Serum pool (mL) Antitoxin (mL) Toxin (mL) Total volume (mL)
L + Control 3 3 3 9
Negative control 6 3 9
0.5 IUb 4.5 2.25 6.75
1.0 IUb 2.25 2.25 2.25 6.75
2.0 IUb 3.37 1.13 2.25 6.75

a2% gelatin in saline

bPooled serum of vaccinated guinea pigs administered low-dose formalin-treated antigen and nanoparticle admixture, high dose formalin treated antigen in nanoparticles and alum with adsorbed antigen were tested for neutralizing antibodies

Animal Response in Toxin-Neutralizing Assay

Guinea pigs were kept alive for 8 days and graded three times daily according to the degree of diphtheria toxin symptoms. The test animals (who were administered the pooled serum obtained from vaccinated guinea pigs along with the toxin) should survive for more than 96 h in order for the CrmAg-induced antitoxin to completely neutralize the toxin.

Lung Histopathological Analysis

A few guinea pigs died during the second and third dosing in some groups administered dry powder vaccines by the pulmonary route. The cause of death was investigated to determine whether it was due to an acute vaccine response or the degradation/endotoxin contamination of the stored dry powder vaccine. Naïve guinea pigs (two for each group) received the dry powder vaccines by the pulmonary route and were sacrificed after 48 h. Lung tissues were collected in 10% formalin. Likewise, the lung tissues of guinea pigs that had died during the second/third dosing of the vaccine were preserved in formalin. Any inflammatory or microscopic changes occurring in the lungs following multiple dosing with the dry powder vaccine were noted. Tissues were embedded in paraffin, sectioned at 5 μm, mounted on slides, and stained with H&E. Slides were examined by a board-certified veterinary pathologist at Charles River laboratories-Pathology Associates.

STATISTICAL ANALYSIS

One-way ANOVA was used to compare the data. Post hoc pair-wise multiple comparisons were conducted using Bonferonni t test when the differences in the means were significant (Sigmaplot version 11, Systat Software, Inc., CA, USA). P values of less than 0.05 were considered statistically significant.

RESULTS

Humoral Antibody Response

Various dry powder formulations containing CrmAg encapsulated in PLGA were administered to guinea pigs by the pulmonary route and the immune response elicited, based on the serum IgG and BAL IgA, assessed. As shown in Fig. 1, an increase in the IgG titers was observed in all of the immunized groups 15 days after the second immunization except the high-dose native antigen in nanoparticles group. The high-dose native antigen in nanoparticle-immunized animals achieved antibody titers similar to other pulmonary immunized animals 30 days after the second immunization. Guinea pigs immunized with alum adsorbed with antigen by the IM route achieved higher initial antibody titers (30 days after the first dose) compared to the groups immunized by the pulmonary route. The high titer values in the alum adsorbed with antigen group were further boosted after the second immunization. Surprisingly, the third dose, administered 12 weeks after the first dose, boosted the immunity marginally in all groups including the alum-adsorbed-with-antigen group (Fig. 1). At the time of sacrifice (16 weeks after the first immunization), animals treated with alum adsorbed with antigen (by the IM route) elicited significantly (p < 0.001) higher IgG titers compared to groups receiving dry powder vaccines by the pulmonary route (Fig. 1). Guinea pigs receiving various dry powder vaccines by the pulmonary route raised similar IgG responses, regardless of the different antigenic formulations.

Fig. 1.

Fig. 1

Anti-CRM-197 antigen-specific serum IgG titers in guinea pigs after three doses of immunization with dry powder formulations administered by the pulmonary route. Responses are compared to those determined for guinea pigs administered with CRM-197 antigen-adsorbed alum by the intramuscular route. Error bars represent the mean ± standard error. Arrows indicate the immunization schedule which corresponds to 0, 6, and 12 weeks. Antibody titers are expressed as the reciprocal of the sample dilution corresponding to at least three times the background level. #NP nanoparticles. Asterisk denotes results are significantly different than all other treatment groups, P < 0.001

IgA antibody titers at week 16 in the lungs of guinea pigs were higher in the groups receiving the novel dry powder formulations by the pulmonary route compared to those receiving alum adsorbed with antigen administered by the IM route (Fig. 2). IgA titers for low-dose native antigen in nanoparticles and high-dose native antigen in nanoparticle-immunized guinea pigs were significantly higher (P < 0.001 and P < 0.003, respectively) than for all of the other immunized groups. Dry powder formulations in which the CrmAg was treated with formalin did not elicit a substantial IgA response in the lungs. However, an immunological advantage is clearly afforded by pulmonary administration of dry powder vaccines with respect to eliciting mucosal immunity.

Fig. 2.

Fig. 2

The last time-point (week 16) CRM-197 antigen-specific broncho-alveolar lavage IgA antibody titers in guinea pigs after three doses of immunization with novel dry powder formulations administrated by the pulmonary routes. Responses are compared to those determined for guinea pigs administered with CRM-197 antigen-adsorbed alum by the intramuscular route. Error bars represent the mean ± standard error. Antibody titers are expressed as the reciprocal of the sample dilution corresponding to at least three times the background level. ** and * denotes results are significantly different than all other treatment groups, P < 0.001 and P < 0.003, respectively. No last time period antibody titers for the high-dose formalin-treated antigen and nanoparticle admixture group as all of the guinea pigs in this group died during the study. #NP nanoparticles

Toxin-Neutralizing Assay

The toxin-neutralizing tests were performed for serum samples obtained from vaccinated guinea pigs. The serum samples were tested at three levels equivalent to 2.0, 1.0, and 0.5 IU/mL of diphtheria antitoxin. After administering the serum-toxin mixture, the guinea pigs were monitored three times daily and weighed once a day, and given a score for any diphtheria toxin symptoms (data not shown).

As shown in Fig. 3, naïve guinea pigs injected with high dose formalin-treated antigen in nanoparticles serum-toxin mixture survived at all three levels of antitoxin. The low-dose formalin-treated antigen and nanoparticle-admixture serum obtained from the pulmonary vaccinated guinea pigs could not neutralize the toxin at any of the doses examined and the animals administered with this serum mixture died within 96 h. In the guinea pigs administered the alum-adsorbed-with-antigen-serum sample, the toxin was completely neutralized at the 0.5 and 1.0 IU/mL level. A partial neutralization of the toxin was obtained for the 2.0 IU/mL level as the animals survived beyond 48 h but not 96 h (Fig. 3).

Fig. 3.

Fig. 3

Toxin-neutralizing test performed on the serum samples of vaccinated guinea pigs at three levels equivalent to 2.0, 1.0, and 0.5 international unit (IU) of diphtheria antitoxin/mL. Nabs-neutralizing antibodies; L-AlumAg low-dose alum with adsorbed antigen, H-FAgN high-dose formalin-treated antigen in nanoparticles, L-FAgNA low-dose formalin-treated antigen and nanoparticle admixture

Histopathology

The deaths of guinea pigs receiving the second and third immunizations were investigated. Since the guinea pigs in the study were exposed to three doses of CrmAg encapsulated in PLGA for 12 weeks, we sought to identify the potential inflammatory or toxicity reaction occurring in the lungs due to multiple dosing. Naïve guinea pigs receiving dry powder vaccines and sacrificed after 48 h had mild acute inflammation. The inflammation was located in the central portion of the lungs, particularly around the main airways, and consisted of a diffuse infiltrate of a small number of neutrophils that were present within alveolar lumens and the lumens of some small airways. There were no other signs of toxicity observed in these lung tissues. Guinea pigs that died after the second and third doses had mild multifocal chronic inflammation in a few tissues, with most of the lung tissues showing no remarkable changes. The chronic inflammation consisted of areas of the alveoli that were filled with alveolar macrophages. In addition, small lymphoid nodules (dense areas) were present within the foci of the chronic inflammation. These observations were not specific to any particular group vaccinated by the pulmonary route (Fig. 4a and b correspond to low-dose formalin-treated antigen in nanoparticles and low-dose formalin-treated antigen and nanoparticle admixture administered guinea pigs).

Fig. 4.

Fig. 4

Microscopic evaluation of the lung tissue of guinea pigs after administering a low-dose formalin-treated antigen in nanoparticles and b low-dose formalin-treated antigen and nanoparticle admixture. Both magnifications were at 4×. Arrows show dense regions consisting of lymphoid nodules

DISCUSSION

Immunization by the pulmonary route in the prophylaxis and therapy of infectious diseases is still in its early stages (30). Diphtheria was the first disease in which passive antibody therapy was shown to be effective. The extracellular toxin secreted by the bacteria can be neutralized by reacting with antibodies before the toxin can bind to host cells; the tertiary structure of the toxin is altered by these antibodies. Neutralizing antibodies are thus protective responses that mimic responses to a natural infection. Vaccines based on CrmAg induce the production of anti-CrmAg antibodies; these antibodies also recognize diphtheria toxin as the CrmAg differs from the native toxin by only one amino acid (31).

Nanoparticles have been employed in drug and vaccine delivery because of their ability to enhance absorption and migration through tissues and cells. Since nanoparticles tend to agglomerate due to their high surface-to-volume ratio, they have been incorporated into porous microparticles through the spray-drying process to form structures that may contain varying proportions of nanoparticles. Once delivered to the deep lung, these appropriately sized particles will collapse to release the nanoparticles in the alveolar region.

In the present study, dry powders intended for vaccine delivery were prepared based on the PNAP technology (17,18). In the form of nanoparticles, it was feasible to entrap significant quantities of CrmAg without damaging the epitopes when analyzed by Western blot. CrmAg was incorporated into various spray-dried formulations containing nanoparticles that influenced the aerosol particle properties by increasing their geometric diameter and lowering their density while maintaining an aerodynamic diameter that allows for deposition in the deep lung (MMAD between 1 and 5 μm). These nanoparticles contained the CrmAg either within their structures or as free antigen in a physical mixture, and thus helped maintain its stability. The volume median diameter, density, and MMAD of the dry powders were appropriate for deposition in the deep lung.

The guinea pig model was used in this study because it is known to correlate with humans with respect to the toxin-neutralizing test in vitro and in vivo. Guinea pigs are used for potency evaluation of novel diphtheria vaccines because, unlike mice, they are responsive to the lethal effects of diphtheria toxin and show protection against challenge after immunization (32). Guinea pigs are also the animal of choice for evaluating adjuvant formulations as mice have previously shown inconsistent responses to adjuvants (33). However, the scarcity of immunological reagents for guinea pigs and the high cost of maintaining them have acted as impediments to their widespread use in new vaccine studies.

Conventional diphtheria vaccines consists of ‘inactivated toxin’ administered as a series of four to five doses, and then as a single dose every 10 years to boost the waning antibody titers. The initial design for our study had a series of three doses administered 12 weeks apart. However, Fig. 1 illustrates that two doses are sufficient to provide and maintain high IgG titers in the systemic circulation for 45 days after the second dose (prior to the last immunization) in guinea pigs. After two immunizations, the antibody responses reached a plateau and only increased slightly after the third dose in all of the immunized animals irrespective of the route of administration (Fig. 1); the last immunization may have helped to sustain the titer levels. Since the guinea pigs were sacrificed 4 weeks after the last dose (16 weeks after the first immunization), we could not anticipate the level of the antibody titers beyond this point. Future studies should consider the long-term effect of multiple immunization regimens on the antibody titers.

The significantly high serum IgG response elicited by the alum-adsorbed-with-antigen group (administered by the IM route) compared to dry powder vaccines administered by the pulmonary route may have been due to the presence of alum (Fig. 1); it has been shown that antigen delivered with alum in parenteral vaccines, rather than in a mucosal vaccine, form persistent areas of high local antigen concentration at the injection site which may allow the host immune system to interact for an extended period of time with the antigen (34).

However, unlike serum IgG titers, IgA antibodies generated in the lungs at week 16 after the first immunization were higher for guinea pigs receiving the dry powder vaccines by the pulmonary route compared to those subjected to IM delivery (Fig. 2). Low-dose native antigen in nanoparticles and high-dose native antigen in nanoparticles vaccines administered to guinea pigs by the pulmonary route elicited a significantly higher IgA response than the alum-adsorbed-with-antigen group administered by the IM route, suggesting that alum-containing parenteral vaccines are poor generators of mucosal immune response (35). Indeed, guinea pigs receiving alum adsorbed with antigen by the IM route, which elicited a high systemic IgG response, had low IgA titers in the BAL fluid. Thus, the administration of parenteral vaccines containing alum has the propensity to elicit high IgG titers in the serum but may not be sufficient to evoke an IgA mucosal response in the lungs which is desirable to prevent dissemination of diphtheria infection from the upper respiratory tract. We did not examine the IgG titers in the BAL fluid though they are evoked significantly in response to infection or vaccination. The IgG in BAL fluid approximates the serum titer, with the majority of the (BAL) IgG coming from transudation from the plasma compartment (36).

Interestingly, the low-dose native antigen in nanoparticles and high-dose native antigen in nanoparticles groups, that elicited significantly higher IgA titers compared to other immunized groups (Fig. 2) including the IM-administered alum-adsorbed-with antigen group, did not contain formalin-treated CrmAg. Formalinization of CrmAg of diphtheria toxin may sometimes contribute to alteration of protein epitopes leading to decrease in antigenicity and immunogenicity (32), although there have been reports where potency has increased after formalin treatment (37). However, this does not explain the formalin-treated CrmAg eliciting equivalent IgG titers to those of untreated antigen groups in the serum of animals in our study. In other studies, native CrmAg has been shown to be less immunogenic in guinea pigs and formalin-treated antigen has led to improvement in its immunogenicity by stabilization of the molecule (21,32). However, these studies did not examine the effect of formalin treatment of the antigen on the lung IgA titers which is critical to give protection in the respiratory tract; our study showed significantly lower IgA titers in the BAL fluid (p < 0.001) and similar IgG titers in serum for antigens treated with formalin. Low IgA titers in animals administered formalin-treated CrmAg may have been due to toxicity of formalin to APCs in the lung. Further studies are required to examine the effect of formalin-treated antigen on the mechanism of IgA and IgG antibody production, and are beyond the scope of this work.

The antitoxin titers in vaccinated animals against diphtheria can be estimated by in vitro methods such as Vero cell method and D ELISA, and by the in vivo toxin-neutralizing assay. The ELISA method does not distinguish between non-neutralizing and neutralizing antibodies whereas the toxin-neutralizing assay measures only the protective antibodies (38). In our study, the toxin-neutralizing test in naïve guinea pigs showed that the serum from the pulmonary vaccinated high-dose formalin-treated antigen in nanoparticles group afforded protection at all concentrations of antitoxin used (0.5, 1.0 and 2.0 IU/mL; Fig. 3; >0.01 IU/mL is protective in man according to the WHO) (39). On the contrary, the low-dose formalin-treated antigen and nanoparticle admixture group failed to provide protection at any level of antitoxin tested. Interestingly, the alum-adsorbed-with-antigen group that had elicited significantly higher IgG levels compared to other immunized groups was able to neutralize the toxin only at the 0.5 and 1.0 IU level of antitoxin/mL. This demonstrates that the high IgG titers in the serum do not necessarily correspond to high titers of protective neutralizing antibodies (38).

The potential inflammatory reaction occurring in the lungs after multiple dosing of the dry powder vaccine was investigated. Mild acute inflammation was observed in guinea pigs administered a single dose of vaccine. Animals exposed to multiple doses of dry powder had multifocal chronic inflammation in a few lung tissues. Presence of lymphoid nodules within the foci confirms the presence of a chronic inflammatory response (Fig. 4 a and b). There remains a possibility that some of the deaths observed after multiple vaccination may have been due to the presence of formalin in the dry powders.

Intranasal immunization with CrmAg has been shown to induce systemic antibody responses comparable to, or greater than, those achieved with parenteral immunization (25). Amidi et al. showed that diphtheria toxoid encapsulated in chitosan was able to elicit high IgA antibody titers in guinea pigs at the respiratory epithelium and mucosal sites (12). The lungs, therefore, may prove a promising alternative route for administration of novel dry powder diphtheria vaccines.

The requirement for adjuvancy was indicated by the absence of significant IgG titers in the serum of guinea pigs immunized by the pulmonary route with the dry powders not containing any adjuvant. Interestingly, IgA titers in the lungs were higher when these powder formulations were administered by the pulmonary route compared to CrmAg administered by the IM route along with alum (Fig. 2). Currently, alum is the only approved adjuvant for human vaccines including multiple diphtheria vaccine which is administered by the parenteral route.

The mechanism by which serum antibody titers are developed after delivering antigens to the lungs is not clearly known, but may be due to the presence of APCs and their proximity to mediastinal lymph nodes. Other factors may also be involved and undoubtedly, further studies are required to explore the relevant mechanisms. Future studies should also look for a correlation between systemic IgG and mucosal IgA titers, and the protective effect of these antibodies after pulmonary vaccination.

CONCLUDING REMARKS

The studies described here demonstrate that pulmonary immunization with dry powder vaccines against diphtheria may lead to a high mucosal immune response in the respiratory tract and sufficient neutralizing antibodies in the systemic circulation to provide protection against this disease. It may be concluded that the lungs could be a suitable route of administration for novel dry powder vaccines that may offer not only clinical advantages but also the pharmaceutical advantages of absence of needles in a mass immunization program and absence of a cold chain for delivery globally.

Acknowledgments

The authors thank Christine Anderson and Dr. Rajesh Gupta from the Office of Vaccines Research & Review, CBER, FDA for kindly providing the diphtheria toxin and antitoxin for the toxin-neutralizing assay.

References

  • 1.WHO Fact Sheet Number 231. Injection safety, in facts and figures. 2006.
  • 2.Rennels MB, Deloria MA, Pichichero ME, Losonsky GA, Englund JA, Meade BD, Anderson EL, Steinhoff MC, Edwards KM. Extensive swelling after booster doses of acellular pertussis-tetanus-diphtheria vaccines. Pediatrics. 2000;105(1):e12. doi: 10.1542/peds.105.1.e12. [DOI] [PubMed] [Google Scholar]
  • 3.Varmus H, Klausner R, Zerhouni E, Acharya T, Daar AS, Singer PA. Public health. Grand challenges in global health. Science. 2003;302(5644):398–399. doi: 10.1126/science.1091769. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Janoff EN, Briles DE, Rubins JB, et al. In: Respiratory bacterial vaccines, in mucosal immunology. Mestecky J, et al., editors. MA: Elsevier; 2005. [Google Scholar]
  • 5.Moschos SA, Bramwell VW, Somavarapu S, Alpar HO. Adjuvant synergy: the effects of nasal coadministration of adjuvants. Immunol Cell Biol. 2004;82(6):628–637. doi: 10.1111/j.0818-9641.2004.01280.x. [DOI] [PubMed] [Google Scholar]
  • 6.McCluskie MJ, Wen YM, Di Q, Davis HL. Immunization against hepatitis B virus by mucosal administration of antigen–antibody complexes. Viral Immunol. 1998;11(4):245–252. doi: 10.1089/vim.1998.11.245. [DOI] [PubMed] [Google Scholar]
  • 7.Jaganathan KS, Vyas SP. Strong systemic and mucosal immune responses to surface-modified PLGA microspheres containing recombinant hepatitis B antigen administered intranasally. Vaccine. 2006;24(19):4201–4211. doi: 10.1016/j.vaccine.2006.01.011. [DOI] [PubMed] [Google Scholar]
  • 8.Lambrecht BN, Prins JB, Hoogsteden HC. Lung dendritic cells and host immunity to infection. Eur Respir J. 2001;18(4):692–704. [PubMed] [Google Scholar]
  • 9.von Garnier C, Filgueira L, Wikstrom M, Smith M, Thomas JA, Strickland DH, Holt PG, Stumbles PA. Anatomical location determines the distribution and function of dendritic cells and other APCs in the respiratory tract. J Immunol. 2005;175(3):1609–1618. doi: 10.4049/jimmunol.175.3.1609. [DOI] [PubMed] [Google Scholar]
  • 10.Holt PG. Pulmonary dendritic cells in local immunity to inert and pathogenic antigens in the respiratory tract. Proc Am Thorac Soc. 2005;2(2):116–120. doi: 10.1513/pats.200502-017AW. [DOI] [PubMed] [Google Scholar]
  • 11.Garcia-Contreras L, Wong YL, Muttil P, Padilla D, Sadoff J, Derousse J, Germishuizen WA, Goonesekera S, Elbert K, Bloom BR, Miller R, Fourie PB, Hickey A, Edwards D. Immunization by a bacterial aerosol. Proc Natl Acad Sci USA. 2008;105(12):4656–4660. doi: 10.1073/pnas.0800043105. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Amidi M, Pellikaan HC, Hirschberg H, de Boer AH, Crommelin DJ, Hennink WE, Kersten G, Jiskoot W. Diphtheria toxoid-containing microparticulate powder formulations for pulmonary vaccination: preparation, characterization and evaluation in guinea pigs. Vaccine. 2007;25(37–38):6818–6829. doi: 10.1016/j.vaccine.2007.05.064. [DOI] [PubMed] [Google Scholar]
  • 13.Cutts FT, Clements CJ, Bennett JV. Alternative routes of measles immunization: a review. Biologicals. 1997;25(3):323–338. doi: 10.1006/biol.1997.0103. [DOI] [PubMed] [Google Scholar]
  • 14.Muttil P, Prego C, Garcia-Contreras L, Pulliam B, Fallon J, Wang C, Hickey A, et al. Immunization of guinea pigs with novel hepatitis B antigen as nanoparticle aggregate powders administered by the Pulmonary Route. AAPS journal, 2010 (in press). [DOI] [PMC free article] [PubMed]
  • 15.Najafabadi AR, Gilani K, Barghi M, Rafiee-Tehrani M. The effect of vehicle on physical properties and aerosolisation behaviour of disodium cromoglycate microparticles spray dried alone or with l-leucine. Int J Pharm. 2004;285(1–2):97–108. doi: 10.1016/j.ijpharm.2004.07.027. [DOI] [PubMed] [Google Scholar]
  • 16.Li HY, Neill H, Innocent R, Seville P, Williamson I, Birchall JC. Enhanced dispersibility and deposition of spray-dried powders for pulmonary gene therapy. J Drug Target. 2003;11(7):425–432. doi: 10.1080/10611860410001659786. [DOI] [PubMed] [Google Scholar]
  • 17.Sung JC, Pulliam BL, Edwards DA. Nanoparticles for drug delivery to the lungs. Trends Biotechnol. 2007;25(12):563–570. doi: 10.1016/j.tibtech.2007.09.005. [DOI] [PubMed] [Google Scholar]
  • 18.Pulliam B, Sung JC, Edwards DA. Design of nanoparticle-based dry powder pulmonary vaccines. Expert Opin Drug Deliv. 2007;4(6):651–663. doi: 10.1517/17425247.4.6.651. [DOI] [PubMed] [Google Scholar]
  • 19.Orr N, Galen JE, Levine MM. Expression and immunogenicity of a mutant diphtheria toxin molecule, CRM(197), and its fragments in Salmonella typhi vaccine strain CVD 908-htrA. Infect Immun. 1999;67(8):4290–4294. doi: 10.1128/iai.67.8.4290-4294.1999. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Papini E, Colonna R, Schiavo G, Cusinato F, Tomasi M, Rappuoli R, Montecucco C. Diphtheria toxin and its mutant crm 197 differ in their interaction with lipids. FEBS Lett. 1987;215(1):73–78. doi: 10.1016/0014-5793(87)80116-3. [DOI] [PubMed] [Google Scholar]
  • 21.Porro M, Saletti M, Nencioni L, Tagliaferri L, Marsili I. Immunogenic correlation between cross-reacting material (CRM197) produced by a mutant of Corynebacterium diphtheriae and diphtheria toxoid. J Infect Dis. 1980;142(5):716–724. doi: 10.1093/infdis/142.5.716. [DOI] [PubMed] [Google Scholar]
  • 22.Illum L, Jabbal-Gill I, Hinchcliffe M, Fisher AN, Davis SS. Chitosan as a novel nasal delivery system for vaccines. Adv Drug Deliv Rev. 2001;51(1–3):81–96. doi: 10.1016/S0169-409X(01)00171-5. [DOI] [PubMed] [Google Scholar]
  • 23.McNeela EA, Jabbal-Gill I, Illum L, Pizza M, Rappuoli R, Podda A, Lewis DJ, Mills KH. Intranasal immunization with genetically detoxified diphtheria toxin induces T cell responses in humans: enhancement of Th2 responses and toxin-neutralizing antibodies by formulation with chitosan. Vaccine. 2004;22(8):909–914. doi: 10.1016/j.vaccine.2003.09.012. [DOI] [PubMed] [Google Scholar]
  • 24.McNeela EA, O’Connor D, Jabbal-Gill I, Illum L, Davis SS, Pizza M, Peppoloni S, Rappuoli R, Mills KH. A mucosal vaccine against diphtheria: formulation of cross-reacting material (CRM(197)) of diphtheria toxin with chitosan enhances local and systemic antibody and Th2 responses following nasal delivery. Vaccine. 2000;19(9–10):1188–1198. doi: 10.1016/S0264-410X(00)00309-1. [DOI] [PubMed] [Google Scholar]
  • 25.Mills KH, Cosgrove C, McNeela EA, Sexton A, Giemza R, Jabbal-Gill I, Church A, Lin W, Illum L, Podda A, Rappuoli R, Pizza M, Griffin GE, Lewis DJ. Protective levels of diphtheria-neutralizing antibody induced in healthy volunteers by unilateral priming-boosting intranasal immunization associated with restricted ipsilateral mucosal secretory immunoglobulin a. Infect Immun. 2003;71(2):726–732. doi: 10.1128/IAI.71.2.726-732.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Huo Z, Sinha R, McNeela EA, Borrow R, Giemza R, Cosgrove C, Heath PT, Mills KH, Rappuoli R, Griffin GE, Lewis DJ. Induction of protective serum meningococcal bactericidal and diphtheria-neutralizing antibodies and mucosal immunoglobulin A in volunteers by nasal insufflations of the Neisseria meningitidis serogroup C polysaccharide-CRM197 conjugate vaccine mixed with chitosan. Infect Immun. 2005;73(12):8256–8265. doi: 10.1128/IAI.73.12.8256-8265.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Murakami H, Kobayashi M, Takeuchi H, Kawashima Y. Preparation of poly(DL-lactide-co-glycolide) nanoparticles by modified spontaneous emulsification solvent diffusion method. Int J Pharm. 1999;187(2):143–152. doi: 10.1016/S0378-5173(99)00187-8. [DOI] [PubMed] [Google Scholar]
  • 28.Zambaux MF, Bonneaux F, Gref R, Dellacherie E, Vigneron C. Preparation and characterization of protein C-loaded PLA nanoparticles. J Control Release. 1999;60(2–3):179–188. doi: 10.1016/S0168-3659(99)00073-5. [DOI] [PubMed] [Google Scholar]
  • 29.Department of Health, Education and Welfare. United States minimum requirements: for tetanus and diphtheira toxoid combined precipitated adsorbed (August 25, 1953).
  • 30.Lu D, Hickey AJ. Pulmonary vaccine delivery. Expert Rev Vaccines. 2007;6(2):213–226. doi: 10.1586/14760584.6.2.213. [DOI] [PubMed] [Google Scholar]
  • 31.Baraldo K, Mori E, Bartoloni A, Norelli F, Grandi G, Rappuoli R, Finco O, Del Giudice G. Combined conjugate vaccines: enhanced immunogenicity with the N19 polyepitope as a carrier protein. Infect Immun. 2005;73(9):5835–5841. doi: 10.1128/IAI.73.9.5835-5841.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Gupta RK, Collier RJ, Rappuoli R, Siber GR. Differences in the immunogenicity of native and formalinized cross-reacting material (CRM197) of diphtheria toxin in mice and guinea pigs and their implications on the development and control of diphtheria vaccine based on CRMs. Vaccine. 1997;15(12–13):1341–1343. doi: 10.1016/S0264-410X(97)00034-0. [DOI] [PubMed] [Google Scholar]
  • 33.Edelman R. In: An overview of adjuvant use, in vaccine adjuvants: preparation methods and research protocols. O’ Hagan DT, editor. New Jersey: Humana; 2000. pp. 1–28. [Google Scholar]
  • 34.HogenEsch H. Mechanisms of stimulation of the immune response by aluminum adjuvants. Vaccine. 2002;20(Suppl 3):S34–S39. doi: 10.1016/S0264-410X(02)00169-X. [DOI] [PubMed] [Google Scholar]
  • 35.Singh M, O’ Hagan DT. In: Microparticles as vaccine adjuvants and delivery systems, in novel vaccination strategies. Kaufmann SHE, editor. Weinheim: Wiley-VCH; 2004. pp. 148–172. [Google Scholar]
  • 36.Reynolds HY. In: Host defense mechanisms in the respiratory tract, in internal medicine. Stein JH, Eisenberg JM, editors. Philadelphia: Elsevier Health Sciences; 1998. pp. 365–370. [Google Scholar]
  • 37.Metz B, Jiskoot W, Hennink WE, Crommelin DJ, Kersten GF. Physicochemical and immunochemical techniques predict the quality of diphtheria toxoid vaccines. Vaccine. 2003;22(2):156–167. doi: 10.1016/j.vaccine.2003.08.003. [DOI] [PubMed] [Google Scholar]
  • 38.Manghi MA, Pasetti MF, Brero ML, Deluchi S, di Paola G, Mathet V, Eriksson PV. Development of an alternative method for testing the immunogenicity of diphtheria vaccines. Vaccine. 1995;13(6):597–601. doi: 10.1016/0264-410X(94)00047-Q. [DOI] [PubMed] [Google Scholar]
  • 39.Department of Health and Human Services, Food and Drug Administration. Biological products; bacterial vaccines and toxoids; implementation of efficacy review; proposed rule. Federal Register. 1985. p. 51002–117 [PubMed]

Articles from The AAPS Journal are provided here courtesy of American Association of Pharmaceutical Scientists

RESOURCES