Skip to main content
NCBI home page
Poultry Science logoLink to Poultry Science
. 2023 Aug 22;102(11):103057. doi: 10.1016/j.psj.2023.103057

Effects of the in ovo injection of an Escherichia coli vaccine on the hatchability and quality characteristics of commercial layer hatchlings

SA Fatemi *,1, LL Lindsey *, JD Evans †, KEC Elliott †, SA Leigh †, KJ Robinson †, A Mousstaaid *, PD Gerard ‡, ED Peebles *
PMCID: PMC10507230  PMID: 37690369

Abstract

In the commercial egg industry, avian pathogenic Escherichia coli (APEC) can lead to significant economic loss. The Poulvac E. coli vaccine (PECV) is a commercially available attenuated live vaccine commonly applied via spray or drinking water to protect against losses associated with colibacillosis. The PECV has not been tested in layer hatching eggs using in ovo injection. Therefore, the purpose of this experiment was to determine the effects of injecting 50 μL of different doses of the PECV into Hy-Line W-36-layer hatching eggs on the hatchability and quality characteristics of hatchlings. At 18 d of incubation (DOI), treatments included 1 noninjected and 1 diluent-injected control. Furthermore, PECV treatments included a full dose (4.4 × 108E. coli CFU) or serial dilutions of the full dose to produce 4.4 × 106, 4.4 × 104, or 4.4 × 102 CFU doses of E. coli. In ovo injections targeted the amnion. Percent hatchability of live embryonated eggs (HI), percent residue eggs, hatchling mortality, and female chick whole and yolk-free BW, relative yolk sac weight, and body length were among the variables examined. Treatment significantly (P < 0.0001) affected HI, with HI being highest in the control groups (97.3% in the noninjected and 94.2% in the diluent-injected), and with HI values being 89.0, 88.9, 84.4, and 71.2% in the 4.4 × 102, 4.4 × 104, 4.4 × 106, and 4.4 × 108 CFU E. coli dose treatments, respectively. The percentage of live embryos that did not complete hatch but that pipped internally (P = 0.024) or externally (P < 0.0001) were significantly affected by treatment, with percentages being highest in the 4.4 × 108 CFU treatment. Female chick body length was significantly (P < 0.0001) affected by treatment and was longer in both control groups and in the 1 × 102 CFU E. coli treatment in comparison to all other treatments. Yolk-free female chick BW was significantly (P = 0.034) affected by treatment and was lower in the 4.4 × 106 CFU and 4.4 × 108 CFU treatments when compared to the diluent-injected control group. An increase in the E. coli concentration administered in the amnion of embryonated layer hatching eggs at 18 DOI decreased hatch success and female chick yolk-free BW and body length.

Key words: chick, E. coli, hatchability, in ovo, layer

INTRODUCTION

Avian pathogenic Escherichia coli (APEC) is a pathogenic strain of E. coli that can cause significant economic losses within the table egg industry (Barnes et al., 2008). Disease transmission and incidence can increase if birds are housed in undesirable conditions in which they are exposed to contaminated feces, are provided contaminated water, or are subjected to insufficient ventilation and high air humidity (Dho-Moulin and Fairbrother, 1999; Barnes et al., 2008). Infection in a layer facility can further be spread by the inhalation of dust particles (Dho-Moulin and Fairbrother, 1999). An infection of APEC can manifest itself in multiple ways, including yolk sac infection, cellulitis, swollen-head syndrome, salpingitis, acute fatal septicemia, peritonitis, airsacculitis, and orchitis (Barnes et al., 2008). Affected birds will normally show clinical signs such as the inability to eat and drink and can exhibit complications with their balance and gait (Barnes et al., 2008).

Early APEC infections can originate and spread in hatcheries, especially if floor eggs are used, and can subsequently lead to poor chick quality (Dho-Moulin and Fairbrother, 1999; Barnes et al., 2008). Avian pathogenic E. coli is one of the leading causes of first week mortality of chickens (Swelum et al., 2021). Due to this, the Poulvac E. coli vaccine (PECV; Zoetis, Durham, NC) was developed, and is the only modified live E. coli vaccine available to protect and improve the health of all major types of commercial poultry in the United States (Zoetis, 2021a,b,c). In commercial layer operations, it is usually administered to pullets through drinking water or coarse spray at 1 d of age and again at 14 to 16 wk of age (Zoetis, 2021a). Upon testing the effects of the PECV in laying hens at 5 d of age, and again at 14 and 30 wk of age, Christensen and Nielsen (2020) reported that it was safe and had no negative effects on egg production. Śmialek et al. (2020) have also shown that the PECV can minimize subclinical E. coli infection incidences and improve the production results of broilers.

The process of injecting a hatching egg with a substance intended to confer a benefit to the embryo and subsequent hatchling is known as in ovo injection (Peebles, 2018). This process usually takes place before incubation is completed and has been determined to be a safe and effective process, with little to no effect on chick hatchability (Peebles, 2018). In addition, to maximize the profitability of in ovo injection it is recommended that injections be given at 18 d of incubation (DOI) and that they target the amnion (AM; Williams and Hopkins, 2011). Utilizing in ovo injection can reduce the risk of bird stress, lower labor costs, and give reassurance that an equal dose is being distributed to each chick vaccinated (Ricks et al., 1999). In ovo injection of vaccines for other ailments besides APEC, such as those for coccidiosis or IBDV, have been found to confer immune protection when birds were challenged later in life (Moura et al., 2007; Sokale et al., 2018).

In a previous study conducted by Lindsey et al. (2022), it was shown that although the in ovo vaccination of 6.5 × 103 and 6.5 × 104 CFU of PECV provided a 100% level of E. coli presence in the AM, those levels led to significantly higher embryonic mortalities and a subsequently lower percentage hatchability of eggs that contained live embryos at the time of injection when it was administrated in the AM rather than air cell. For a better understanding of the effects of the in ovo administration of the PECV in commercial layers, in addition to a comprehensive determination of the effects of different doses of the PECV on the hatching process, a more extensive subsequent evaluation of the various quality characteristics of hatchlings is necessary. Therefore, the first objective of this study was to analyze the hatchery residue and hatchability percentage of Hy-Line W-36-layer embryos when vaccinated in ovo with various doses of the PECV at 18 DOI. A second objective was to determine the effects of these treatments on the subsequent quality characteristics of the hatchlings including yolk-free BW (YFBW), relative yolk sac weight, and body length.

MATERIALS AND METHODS

Egg Incubation and Treatment Assignments

All handling and care of the layer hatchlings were conducted under the approval of the Mississippi State University Institutional Animal Care and Use Committee (Protocol # IACUC-20-351). At 0 DOI, 3,240 Hy-Line W-36 (Hy-Line International, 2020) layer hatching eggs from a 46-wk-old parent flock were randomly allocated to 3 single-stage NatureForm NMC 1080 incubators (NatureForm Hatchery Systems, Jacksonville, FL). The parent flock had been vaccinated for E. coli based on the recommendations of the commercial source (Hy-Line North America, Mansfield, GA). The incubators served as both the setter and hatcher units for the study. Settable eggs were preassigned to 6 treatment groups, with treatments assigned to incubation trays containing a total of 90 eggs designated for an individual treatment. Two replicate blocks of incubation trays having a similar random arrangement of all 6 treatments were represented in each of the 3 incubators. Therefore, there were a total of 6 replicate groups (blocks) for each treatment. Eggs were incubated at standard set temperatures (37.5°C dry bulb and 29.8°C wet bulb temperatures between 0 and 18 DOI, and 36.9°C dry bulb and 29.3°C wet bulb temperatures between 18 and 22 DOI). Eggs were turned once every 2 h. At 12 and 18 DOI, eggs were candled to verify live embryonation. Nonembryonated eggs and eggs with dead embryos were discarded. Mean percent egg weight loss (PEWL) for each treatment-replicate group of eggs was calculated between 0 and 12 DOI and between 12 and 18 DOI. This was determined to establish the lack of an incubational vertical stratification effect on the arrangement of the treatment groups. All chicks were pulled from the hatcher at 22 DOI (hatch).

Vaccine Preparation and In Ovo Injection at 18 DOI

All serial dilutions were performed in sterile Marek's disease commercial diluent (Merial, Inc., Athens, GA), and all titer determinations were determined from duplicate plates and were performed as described by Lindsey et al. (2022). Upon plating the full dose of the PECV, pre- and postinjection titers (CFU/mL) were observed to be 8.8 × 109 and 3.4 × 109, respectively. The full dose of the PECV was diluted using 10-fold serial dilutions to create 1 × 10−4, 1 × 10−5, 1 × 10−6, and 1 × 10−7 dilutions. In correspondence with the serial dilutions, the amount of E. coli delivered in a 50 μL volume to each egg were 4.4 × 108, 4.4 × 106, 4.4 × 104, and 4.4 × 102 CFU, respectively.

At 18 DOI, a 50 μL volume of the diluent was in ovo-injected alone (DI) or in combination with the PECV containing 1 of the 4 CFU doses of E. coli. These E. coli doses are subsequently described respectively as 102, 104, 106, and 108 CFU levels. A noninjected (NI) control group was also included. In the dispensing of diluent alone or in combination with PECV, injections penetrated the inner air cell membrane and targeted the AM. Eggs belonging to different treatment groups were kept separate and remained in the hatcher unit until they were injected. Treatments were injected using an Embrex Inovoject m injection machine (Zoetis, Durham, NC). Injection of diluent alone was administered first, followed by the consecutive administration of the lowest to highest E. coli doses. The injection process took a total of 1.5 h to complete. Cross-contamination prevention between treatments was as described by Lindsey et al. (2022). One egg from each tray (6 per treatment) was randomly selected to determine the accuracy of site of injection (Elliott et al., 2018) and embryonic developmental stage (Avakian, 2006). Once injection was complete, eggs (including the NI control) were returned to their same incubators for the hatcher phase and were placed in stacked hatching baskets with each basket divided in half by a divider, with each half of the basket containing equal numbers of eggs of the same treatment. All treatments were represented in each of the 6 total replicate blocks across the 3 incubators. In order to prevent possible contamination at hatch, the NI and DI treatment eggs were stacked respectively in the top 2 incubator levels, followed by the lowest to highest dose treatments being sequentially placed in the succeeding levels of the incubator.

Hatchery Residue Analysis and Hatchling Evaluation at 22 DOI

According to the methods described by Lindsey et al. (2022), at hatch (22 DOI) no chicks were culled, and all hatchlings belonging to each treatment-replicate group were weighed and counted to determine mean hatchling BW and the percentage of viable chicks hatched from eggs that contained live embryos at the time of injection (18 DOI: HI). Embryo residue analysis included the percentages of contaminated eggs (PCT); mid-incubation (8–17 DOI, PMD) and late-incubation (18–19 DOI, PLD) embryonic deaths; embryos that were alive in the shell but had not attempted to pip (PLNP); embryos that were dead but that had pipped internally or externally (PDP); and live embryos that did not complete hatch but had pipped internally (PLIP) or externally (PLEP). Visible assessment of egg contamination (Peebles et al., 1998) and an analysis of hatched chicks at 22 DOI included determinations of hatchling mortality (free from the shell) and culled chicks (possessing dry, rough navels, or externalized intestines).

Hatchling Quality Characteristics at 22 DOI

Female hatchlings were selected via feather sexing, and all females within a treatment (all 6 replicate groups) were pooled together. From 6 chicks in each pool, blood was collected via decapitation for serum ELISA analysis. In addition, within each pool of female hatchlings in each treatment group, 35 live female hatchlings per treatment were randomly selected for body length measurement. In an elongated position, body length was measured from the tip of the beak to the end of the extended third toe of the left leg (Joseph et al., 2006; Van den Brand et al., 2019). After body length measurement, the hatchlings were euthanized and the total BW and yolk sac weight of each of the 35 females per treatment were subsequently determined. Yolk sac dry matter and moisture contents were determined according to the procedures of Peebles et al. (1998). Yolk sac weight was expressed as a percentage of total BW, and yolk sac dry matter and moisture contents were expressed as percentages of total yolk sac weight. Using the total BW and yolk sac weight data, chick YFBW was calculated by subtracting yolk sac weight from total BW. Furthermore, BW-to-length ratio (BWTLR) and percentage body mass (PBM) were also calculated for each sampled female hatchling. The BWTLR was calculated by dividing the BW of the chick by its length. Mean PBM was calculated by dividing the yolk-free BW of the chick by its whole BW and multiplying the result by 100. Any remaining chicks (male and female) were euthanized.

Serum ELISA Analysis

Indirect ELISA was used to determine serum anti-E. coli IgY levels in day-of-hatch birds. An overnight culture of E. coli (ATCC - 25922) was diluted 1:10 in ELISA coating buffer (Bio-Rad Laboratories, Hercules, CA) and used to coat 96-well plates by incubating overnight at 4.0°C. Following incubation, plates were washed 3 times with ELISA wash buffer (Bio-Rad Laboratories, Hercules, CA) and blocked overnight (4.0°C) with ELISA BSA Block (Bio-Rad Laboratories, Hercules, CA). Plates were washed twice with wash buffer, sealed, and stored at 4.0°C for no more than 2 wk prior to analysis. Serum samples were diluted 1:500 in blocking buffer, added to the prepared plate in triplicate, and incubated at room temperature for 3 h. Serum was removed by washing 4 times in wash buffer. Goat anti-Chicken IgG conjugated to HRP (Bio-Rad Laboratories, Hercules, CA) was diluted 1:10,000 in blocking buffer and incubated on the plate for 1 h and removed by washing the plate 4 times with wash buffer. Antibody signal was detected by adding TMB Core+ (Bio-Rad Laboratories, Hercules, CA) and incubating for 30 min followed by the addition of stop solution (2M H2SO4). Plates were read at 450 nm using an Epoch microplate spectrophotometer (Agilent Technologies, Santa Clara, CA). A standard curve was run using unlabeled chicken IgY (Southern Biotech, Birmingham, AL) and used to interpolate antibody concentrations.

Statistical Analysis

A 1-way ANOVA was utilized to analyze the data. In the hatcher phase, a complete block design was utilized. Hatch basket section served as the experimental unit in the analyses of hatched chicks, residue eggs, and average hatchling BW. Individual female chick quality characteristics and serum ELISA and antibody concentration data were analyzed with individual bird representing the experimental unit. Least squares means were compared in the event of significant global effects. Global effects and least squares means differences were considered significant at P ≤ 0.05. All data were analyzed using the MIXED procedure of SAS software 9.4 (SAS Institute, 2013; Fatemi et al., 2022). Treatment means and LS means comparisons for variables that were not significantly affected by treatment are not described or discussed.

RESULTS

Egg Weight Loss, Site of Injection, and Embryo Staging

There were no significant differences between treatments for PEWL between 0 and 12 (P = 0.555) and between 12 and 18 (P = 0.975) DOI (Table 1). Of the 36 dye injections administered at 18 DOI, 31 (86.1%) were observed to be successfully administered in the AM. The rest of the injections were in the allantois (13.9%). The average embryonic developmental stage score at 18 DOI was 1.86, which was prior to a pipping response and prior to the positioning of the head of the embryo under its right wing.

Table 1.

Effects of in ovo injection treatments on percentage egg weight loss (PEWL) between 0 and 12, and 0 and 18 d of incubation (DOI), hatchability of injected live embryonated eggs (HI), hatchling BW, and hatch residue variables at 22 DOI.

In ovo injection treatments3 PEWL 0–12 PEWL 12–18 PCT1 PLD1 PLNP 1 PDP1 PLIP1 PLEP1 Hatchling mortality2 HI Hatchling BW
% g
NI 7.44 4.28 0b 1.40b 0.22 0.22 0b 0.63c 0.22 97.3a 39.0
DI 7.42 4.24 0b 4.26a 0 0.87 0b 0.67c 0 94.2ab 39.0
4.4 × 102 7.32 4.3 0b 5.99a 0.22 0.87 0.87ab 3.05bc 0 89.0bc 39.0
4.4 × 104 7.61 4.3 0.63b 4.16a 0.22 1.45 1.68a 2.97bc 0 88.9bc 38.8
4.4 × 106 7.48 4.28 0.20b 5.14a 1.25 1.05 0.85ab 6.87b 0.20 84.4c 38.2
4.4 × 108 7.48 4.29 2.06a 4.35a 1.58 1.50 2.23a 17.72a 0.42 71.2d 38.6
SEM 0.146 0.081 0.348 1.182 0.479 0.720 0.710 2.455 0.228 3.13 0.360
P value 0.555 0.975 <0.0001 0.017 0.156 0.528 0.024 <0.0001 0.373 <0.0001 0.191
Open in a new tab
a–d

Means in the variable column with no common superscript differ significantly (P ≤ 0.05).

1

Percentages of contaminated eggs (PCT); late-incubation (18–19 DOI) embryonic deaths (PLD); embryos that were alive in the shell but had not attempted to pip (PLNP); embryos that were dead but that had pipped internally or externally (PDP); and live embryos that did not complete hatch but had pipped internally (PLIP) or externally (PLEP).

2

Hatchling mortality based on percentage of chicks observed dead in hatching basket sections.

3

NI = noninjected control treatment. DI = Marek's disease commercial diluent-injected control treatment. Poulvac vaccine E. coli CFU administered per egg are designated as: 4.4 × 102, 4.4 × 104, 4.4 × 106, and 4.4 × 108.

N = 6 replicate hatching basket sections in each treatment, with each hatching basket section containing a maximum of 90 eggs (minus eggs removed for dye injection).

Residue Eggs and Hatchability

Mean PMD (none observed), PLNP (P = 0.156), and PDP (P = 0.528) were not significantly affected by treatment. However, mean PCT (P < 0.0001) and PLD (P = 0.017) were significantly affected by treatment (Table 1). In the 108 PECV dose treatment there was a significantly higher percentage of contaminated eggs when compared to all other treatments, with no significant differences between all the other treatments. Mean PLD was significantly lower in the NI treatment compared to all other treatments, with all other treatments being not significantly different from each other.

Mean PLIP (P = 0.024) and PLEP (P < 0.0001) were significantly affected by treatment (Table 1). In the 104 and 108 PECV dose treatments, there were significantly higher PLIP values than in the NI and DI treatments, with those in the 102 and 106 PECV dose treatments being intermediate. A significant higher PLEP was observed in the 108 PECV dose treatment in comparison to all the other treatments. The PLEP in the 106-dose treatment was also higher than that in the NI and DI controls, with that in the 102 and 104 PECV dose treatments being intermediate.

Mean HI was significantly (P < 0.0001) affected by treatment (Table 1). In the NI treatment, HI was significantly higher than all the other treatments except for the DI control group. A significantly higher HI was also observed in the DI treatment in comparison to that in the 106 PECV dose treatment, with that in the 102 and 104 treatments being intermediate. Furthermore, HI in the 106 PECV dose treatment was higher than that in the 108 PECV dose treatment. There were no significant differences between the treatments for percentage hatchling mortality (P = 0.373) or for average hatchling BW (P = 0.191).

Female Chick Quality Characteristics

Mean whole female chick BW was not significantly (P = 0.101) different between treatments. However, mean female YFBW was significantly (P = 0.034) affected by treatment (Table 2). Female YFBW in the DI and 104 dose treatments was significantly higher than the YFBW in the 106 and 108 PECV dose treatments but was not significantly different from the NI and 102 dose treatments. The NI and 102 PECV treatments were intermediate to and not significantly different from any of the other treatment groups. Mean absolute (YSW) and percent (PYSW) yolk sac weight of the female chicks were also significantly (P < 0.0001) affected by treatment (Table 2). Those in the 108 PECV dose treatment had a higher YSW in comparison to that in all the other treatment groups except for the 104 treatment which was intermediate to the 108 and 106 treatments. The YSW in the 106 treatment was greater than that in the NI and 102 treatments with that in the DI treatment being intermediate. The PYSW value in the 108 PECV dose treatment was greater than that in the NI and DI controls as well as the 102 and 106 PECV dose treatments. The PYSW values were also higher in the 104-dose treatment in comparison to both control groups and the 102 PECV treatment. Furthermore, the PYSW value in the 106 PECV dose treatment was higher than that in the NI and 102 dose treatments, with that in the DI control group being intermediate (Table 2).

Table 2.

Effects of in ovo injection treatment on female hatchling body weight (BW) and hatchling quality characteristics at 22 d of incubation (DOI).

In ovo injection treatments2 BW YFBW1 YSW1 DYSW1 PYSW1 PYSM1 PDYSW1 PBM1 Body length BWTLR1
g g g g % cm g/cm
NI 37.8 35.9abc 3.74c 1.85c 9.69d 50.7abc 49.3abc 95.2a 18.36a 2.06c
DI 38.8 36.7a 4.19bc 2.11bc 10.77cd 50.4bc 49.7ab 94.7ab 18.38a 2.11bc
4.4 × 102 37.4 35.6abc 3.73c 1.81c 9.83d 51.7a 48.3c 95.2a 18.23a 2.05c
4.4 × 104 38.6 36.2ab 5.01ab 2.45ab 12.85ab 51.3ab 48.7bc 93.7cd 17.52b 2.21a
4.4 × 106 37.0 34.8c 4.49b 2.21b 11.97bc 51.3ab 48.7bc 94.1bc 17.56b 2.09bc
4.4 × 108 37.8 35.1c 5.32a 2.69a 14.01a 49.9c 50.2a 92.9d 17.73b 2.15ab
SEM 0.72 0.63 0.326 0.175 0.757 0.641 4.16 0.42 0.122 0.042
P value 0.101 0.034 <0.0001 <0.0001 <0.0001 0.040 <0.0001 <0.0001 <0.0001 0.002
Open in a new tab
a–d

Means in the variable column with no common superscript differ significantly (P ≤ 0.05).

1

Yolk-free BW (YFBW; BW-YSW), absolute yolk sac weight (YSW), absolute dry yolk sac weight (DYSW), percent yolk sac weight [PYSW; (YSW/BW) × 100], percent yolk sac moisture [PYSM; ((YSW − DYSW)/YSW) × 100], percent dry yolk sac weight [PDYSW; (DYSW/YSW) × 100], percent body mass [PBM; (YFBW/BW) × 100], and body weight-to-length ratio (BWTLR; BW/body length).

2

NI = noninjected control treatment. DI = Marek's disease commercial diluent-injected control treatment. Poulvac vaccine E. coli CFU administered per egg is designated as: 4.4 × 102, 4.4 × 104, 4.4 × 106, and 4.4 × 108.

N = 35 individual chicks from a pool of 6 replicate hatching basket sections in each treatment.

Mean absolute (DYSW) and percentage (PDYSW) dry yolk sac weight of female chicks were significantly (P < 0.0001) affected by treatment (Table 2). Mean DYSW in the 108 PECV dose treatment was significantly higher than that in all the other treatments, except for that in the 104 PECV dose treatment which was intermediate to the 108 and 106 dose treatments. The DYSW of birds in the 104 and 106 PECV dose treatments was higher than those in the NI and 102 PECV dose treatments, with that in the DI control treatment being intermediate. The PDYSW of birds in the 108 PECV dose treatment was significantly higher than that in all the other treatments, except for the NI and DI control groups. The PDYSW of those in the DI treatment was higher than those in the 102 PECV dose treatment, with that in the 104 and 106 PECV dose treatment groups being intermediate. The PDYSW of the NI control group was intermediate to and not significantly different from any of the other treatment groups. Mean percent yolk sac moisture (PYSM) in female chicks was also significantly (P = 0.040) affected by treatment (Table 2). The PYSM of the female chicks was significantly higher in the 102 PECV dose treatment in comparison to the DI and 108 PECV dose treatments. The PYSM in the 104 and 106 PECV dose treatments were also higher than that in the 108 PECV dose treatment, whereas the NI and DI groups were intermediate. In addition, the NI control group was intermediate to and not significantly different from any of the treatments (Table 2).

Mean PBM of female chicks was significantly (P < 0.0001) affected by treatment (Table 2). Birds in the NI and 102 PECV dose treatments had a significantly higher PBM than those in the 106 PECV dose treatment, with those in the DI treatment being intermediate. Birds in the 106-dose treatment had a higher PBM than those in the 108 PECV dose treatment, with those in the 104-dose treatment being intermediate (Table 2). Mean body length of female chicks was significantly (P < 0.0001) different between treatments (Table 2). The birds in the NI, DI, and 102 PECV dose treatments had significantly longer body lengths compared to those in the 104, 106, and 108 PECV dose treatments. Body weight to length ratio (BWTLR) of female chicks was also significantly (P = 0.002) different between treatments (Table 2). The BWTLR of birds in the 104-dose treatment was significantly higher than that in all the other treatments except for that in the 108-dose treatment. The BWTLR in the 108 PECV dose treatment was higher than that in the NI and 102 PECV dose treatments, within that in the DI and 106 dose treatments being intermediate (Table 2).

Female Chick Serum IgY Response at Hatch

Most of the sampled female chicks were found to be positive for serum IgY against E. coli at hatch (Table 3). Analysis of the antibody concentration of the positive samples showed no significant difference between the treatments (P = 0.675; Figure 1).

Table 3.

Percentage of ELISA positive chicks at hatch by treatment.

In ovo injection treatments1 Number of female chicks sampled2 Number of female chicks testing positive Percentage positive chicks
NI 6 3 50
DI 6 2 33
4.4 × 102 6 4 67
4.4 × 104 6 4 67
4.4 × 106 6 5 83
4.4 × 108 6 3 50
Open in a new tab
1

NI = noninjected control treatment. DI = Marek's disease commercial diluent-injected control treatment. Poulvac vaccine E. coli CFU administered per egg is designated as: 4.4 × 102, 4.4 × 104, 4.4 × 106, and 4.4 × 108.

2

Female chicks were randomly selected from each treatment.

Figure 1.

Figure 1

Open in a new tab

Effect of in ovo injection on circulating IgY at hatch. NI = noninjected control treatment. DI = Marek's disease commercial diluent-injected control treatment. Poulvac vaccine E. coli CFU administered per egg are designated as: 4.4 × 102, 4.4 × 104, 4.4 × 106, and 4.4 × 108. Serum was collected from 6 birds per treatment immediately following hatch. Circulating IgY was measured using an indirect ELISA and quantified against a chicken IgY standard curve. No significant difference in antibody titer was found among treatments (P = 0.675).

DISCUSSION

Across the 6 tray levels and the 3 incubators, there were no significant differences between treatments for PEWL between both the 0 to 12 and 12 to 18 DOI periods, indicating that the eggs in all the treatment groups experienced similar incubational conditions, thereby ensuring that the noted effects of treatment on the variables examined were not confounded by environmental influences during incubation. Moreover, with only 6 tray levels in each incubator, a possible stratification effect in each incubator remained minimal.

A comparison of the pre- and postplating results in which 8.8 × 109 and 3.4 × 109 CFU/mL, respectively, were observed, indicates that a negligible number of bacteria were lost during the injection process, and that an appreciable number of bacteria would have been administered within the treatment groups. The efficacy of the administered PECV is further supported by the embryo developmental stage and site of injection results. In the current study, 86.1% of the injections were successfully administered in the AM. The mean embryo stage score at 18 DOI of 1.86 was like the 2.09 score reported by Sokale et al. (2018) for Ross 708 broiler embryos at 18.5 DOI. It was also like the 1.65 score reported by Elliott et al. (2022) for layer embryos at 18 DOI. According to Avakian (2006), embryos injected at 18 DOI with an Inovoject machine have a greater than 90% chance of their injections occurring in the AM, with approximately 5.79% of their injections occurring in the allantoic sac, air cell, or yolk sac. The current results of this and previous studies indicate that most of the injections administered at 18 DOI were successfully administered in the AM. Furthermore, employing the same amniotic injection procedure as in the current study and using PCR results from swabs of the amnion membrane at 19 DOI, Lindsey et al. (2022) reported that all diluent-injected control eggs were negative for Poulvac E. coli presence, whereas eggs injected with 6.5 × 101 or 6.5 × 102 CFU were 50% positive and those injected with 6.5 × 103 or 6.5 × 104 CFU were 100% positive for E. coli presence.

Like the decrease in HI and increase in PLIP and PLEP in response to the increased dosage of the in ovo-injected PECV in this study, Lindsey et al. (2022) reported that HI decreased, and embryonic mortalities increased in response to the in ovo injection of 6.5 × 104 CFU of the PECV into the AM. These results are also like those of Elliott et al. (2017), who investigated the injection of the F-strain of Mycoplasma gallisepticum in Hy-Line W-36-layer embryos at 18 DOI. In that study, in which 5.5 × 106, 5.5 × 104, 5.5 × 102, and 5.5 CFU concentrations of the Poulvac Myco F vaccine were used, it was found that HI decreased as dose increased, with an approximate 60% HI observed in the full dose treatment. More specifically, there was a significantly higher percentage of embryos that died while pipping after receiving a 1 × 104 or 1 × 106 CFU dose of the Poulvac Myco F vaccine. These combined results indicate that when injected at 18 DOI into the AM, the E. coli of the PECV at the dosages delivered in this study, particularly between 104 and 108 CFU, compromised late embryonic viability, thereby preventing the ability of the embryo to complete a normal hatching process. The increases in PLIP and PLEP were evidently the major cause of the decrease in HI. It was also noted that the highest percentages of PLIP and PLEP corresponded with the highest percentages of PDP in the 108 CFU treatment, suggesting that given a longer observational period after 22 DOI, the PLIP and PLEP embryos would eventually die. A possible reason for the dose-dependent effects of the in ovo injection of the PECV could be linked to an increase in E. coli infection after the injection of the 108 CFU level of the PECV. Furthermore, the current results showed that in comparison to all the other treatments, higher percentages of contaminated eggs were observed when a full dose of the PECV was injected.

Maternal antibodies, transferred from the breeder hen to the embryo via the egg yolk, are a chick's leading defense against pathogens until they can produce their own immune cells (Hincke et al., 2019). The hens that laid the hatching eggs used in the current study were vaccinated for E. coli according to company recommendations (Hy-Line International, 2020), and most of the resulting progeny were positive for serum IgY against E. coli at hatch as expected (Malkinson, 1965; Heller et al., 1990). There were, however, no differences in antibody concentration among the positive samples regardless of if the embryos had been administered the E. coli vaccine in ovo or not. These findings hint that the maternal antibodies may not have interfered with or neutralized the Poulvac E. coli vaccine strain within the embryonated egg (Negash et al., 2004). Additionally, the levels of antibodies that they possessed, if they acted to neutralize the vaccine strain of E. coli, may not have been sufficient to protect them from the large amounts of E. coli administered at 18 DOI. In a study conducted by Weber and Rodenberg (2019), 1-day-old specific-pathogen-free chicks were vaccinated with PECV by coarse spray and then challenged with APEC at either 7, 14, or 21 d of age (DOA). While control chicks showed a high percentage of colibacillosis lesions at 7, 14, and 21 d of growout, chicks that had been vaccinated with the PECV had a significantly lower presence of lesions at 14 and 21 d of growout. Blood samples were taken on day of hatch to test for the presence of maternal antibodies in chicks that later served as unvaccinated controls or that were vaccinated at 1 d of age with the PECV. While it was found that the specific-pathogen-free chicks did indeed carry maternal antibodies for E. coli, it was also concluded that the presence of maternal antibodies alone were not sufficient in protecting chicks from an APEC infection.

Another basis for a defective hatching process and increased rate of embryonic mortality could be due to the immaturity of the immune systems of the embryos. It is also well documented that the immunity of chickens is age-related (van Ginkel et al., 2015). By 18 DOI, chicken embryos can activate an immune response against pathogens that they come into contact with (Hincke et al., 2019 ). However, because immune competence tends to increase with age, older birds tend to have a better immune response to pathogens than embryos or young chicks (Alkie et al., 2019). Furthermore, an increase in mortality is not seen when the PECV is administered to chickens via coarse spray or drinking water at 1 d of age (Zoetis, 2021a), or to pullets via drinking water at 5 d of age, or at 14 and 30 wk of age (Christensen and Nielsen, 2020). The adaptive immunity of the gut-associated lymphoid tissue matures toward the end of the second week of life (Bar-Shira and Friedman, 2006). Therefore, it is likely that only an innate immune response was activated when the embryos were vaccinated at 18 DOI in the current study, and that the response was not strong enough to overcome the large amounts of E. coli deposited in the AM. This indicates that chicken embryos are more prone to enteric pathogens such as E. coli due to immaturity of the adaptive immune response.

Triplett et al. (2018) administered 3 different probiotic strains of bacteria (Lactobacillus acidophilus, Bacillus subtilis, and Bifidobacterium animalis) by in ovo injection to broiler hatching eggs at 18 DOI. In the Bacillus subtilis treatment, there was a significant increase in the percentage of embryos that died during the pipping process. Upon consideration of various possible reasons for the increase in the number of embryos that died while pipping, Triplett et al. (2018) suggested that the embryos in that treatment group may have lacked sufficient energy reserves to immunologically resist the bacterial challenge. This same type of response to the E. coli bacterial challenge, involving insufficient energy reserves as well as an immature immunity, may also have occurred in the current study.

The results of the determinations of the various subsequent quality characteristics of the hatchlings in response to the imposed treatments revealed that the adverse posthatch effects of the higher in ovo PECV dosages on the embryo carried over into the posthatch period. In comparison to the DI control, female chick YFBW and body length were decreased in the 106 and 108 PECV dose treatments, and those in the 108-dose treatment had a lower PBM. Chick body length has been viewed as a variable that can be a predictor of later chick quality, with a longer length being associated with improved posthatch performance (Molenaar et al., 2008). Although compensatory growth cannot be precluded, a higher dose of the PECV may lead to lower chick quality in the growout phase. It is well documented that metabolic enzyme activity increases activity between 17 and 20 DOI (Rinaudo et al., 1979). Their activities then become low in the early posthatch phase (Moriuchi and Deluca, 1974). Additionally, the development of tissues linked to humoral immunity (spleen and bursa of Fabricius) and cell-mediated immunity (thymus) occur by 19 DOI (Moriuchi and Deluca, 1974). Furthermore, small intestine development continues through the first wk of posthatch life (Uni et al., 1995a,b; Applegate et al., 1999; Geyra et al., 2001). It is worth mentioning that during the fast-growing stage of embryogenesis (the last 3–4 DOI), the embryo requires sufficient nutrients to support a proper hatch as well as hatchling quality. Additionally, this stage of embryogenesis is highly associated with posthatch performance (Peebles, 2018). Not only may in ovo injection of the PECV negatively affect embryo development and result in a delay in the proper development of immune-related organs, but it may also further lead to changes in the morphology of the small intestine, which is linked to lower nutrient absorption (Onderci et al., 2006). However, further research is needed to determine histomorphological changes in the small intestine and immune-related organs that may occur in response to the in ovo injection of the PECV.

In comparison to both controls, YSW, PYSW, and DYSW were significantly higher in the 108 PECV dose treatment. This indicates that the hatched chicks from the higher PECV treatment group retained heavier yolk sacs. Having a larger residual yolk sac could be another result of a compromised physiology of the embryos in response to the injection of the PECV. A mounted immunological resistance to the bacterial insult could have led to a delay in development, thus causing the affected chicks to retain more yolk. These results, however, differ from those of Elliott et al. (2017), who found no significant differences in YFBW, YSW, PYSW, DYSW, or PDYSW between controls and embryos that were in ovo-vaccinated with the Poulvac Myco F vaccine at 18 DOI. Nevertheless, the results of the current study were like the results reported in a study by Montgomery et al. (1999), who injected EcNAL, an adapted form of E. coli, into the chorioallantoic sac of broiler embryos at 12 DOI. Of the chicks that hatched, it was discovered that the chicks injected with EcNAL had heavier yolks and a higher yolk mass relative to body mass when compared to control groups. When birds were reared until 21 DOA, it was further found that the mean BW of the chicks that had been injected with EcNAL were significantly lower than the mean BW of the control groups (Montgomery et al., 1999).

In conclusion, it was shown in this study that the PECV can be successfully delivered to the AM of Hy-Line W-36-layer hatching eggs at 18 DOI. Nevertheless, the injection of the PECV into the AM at 18 DOI at the doses administered in this study may incur detrimental effects on late embryo livability, subsequent HI, and hatchling quality characteristics. Furthermore, in ovo injection of the vaccine to embryos rather than by spray or drinking water to hatchlings, as is currently recommended, appears to have an overwhelming effect on the newly forming immune system of the embryo, yielding potential undesired results. Further exploration into the growout period to examine possible compensatory growth should be considered.

ACKNOWLEDGMENTS

The authors express their appreciation for the financial support of the United States Department of Agriculture (USDA; agreement No. 58-6064-9-014), Zoetis Animal Health Co. (for donation of the vaccine), and for the help and assistance from the graduate and undergraduate students in the Mississippi State University Poultry Science Department.

DISCLOSURES

There is no conflict of interest.

Footnotes

This publication is a contribution of the Mississippi Agricultural and Forestry Experiment Station.

This material is based on work that is supported by the National Institute of Food and Agriculture, U.S. Department of Agriculture, Hatch project under accession number 1011797.

Use of trade names in this publication does not imply endorsement by Mississippi Agricultural and Forestry Experiment Station of these products, nor similar ones not mentioned.

REFERENCES

  1. Alkie T.N., Yitbarek A., Hodgins D.C., Kulkarni R.R., Taha-Abdelaziz K., Sharif S. Development of innate immunity in chicken embryos and newly hatched chicks: a disease control perspective. Avian Pathol. 2019;48:288–310. doi: 10.1080/03079457.2019.1607966. [DOI] [PubMed] [Google Scholar]
  2. Applegate T.J., Dibner J., Kitchell M.L., Uni Z., Lilburn M.S. Effect of turkey (Meleagridis gallopavo) breeder hen age and egg size on poult development. 2. Intestinal villus growth, enterocyte migration, and proliferation of the turkey poult. Comp. Biochem. Physiol. B Biochem. Mol. Biol. 1999;124:381–389. doi: 10.1016/s0305-0491(99)00140-6. [DOI] [PubMed] [Google Scholar]
  3. Avakian A. Understanding in ovo vaccination. Int. Hatchery Pract. 2006;20:15–17. [Google Scholar]
  4. Barnes, J. B., L. K. Nolan, and J. Vaillancourt. 2008. Colibacillosis. Pages 691–732 in Diseases of Poultry. Y. M. Saif, ed. 12th ed. Iowa State Press, Ames, IA.
  5. Bar-Shira E., Friedman A. Development and adaptations of innate immunity in the gastrointestinal tract of the newly hatched chick. Dev. Comp. Immunol. 2006;30:930–941. doi: 10.1016/j.dci.2005.12.002. [DOI] [PubMed] [Google Scholar]
  6. Christensen H., Nielsen C. Safety of the live Escherichia coli vaccine Poulvac® E. coli in layer parent stock in a field trial. Vet. Microbiol. 2020;240 doi: 10.1016/j.vetmic.2019.108537. [DOI] [PubMed] [Google Scholar]
  7. Dho-Moulin M., Fairbrother J.M. Avian pathogenic Escherichia coli (APEC) Vet. Res. 1999;30:299–316. [PubMed] [Google Scholar]
  8. Elliott K.E.C., Branton S.L., Evans J.D., Gerard P.D., Peebles E.D. Layer chicken embryo survival to hatch when administered an in ovo vaccination of strain F Mycoplasma gallisepticum and locations of bacteria prevalence in the newly hatched chick. Poult. Sci. 2017;96:3879–3884. doi: 10.3382/ps/pex196. [DOI] [PubMed] [Google Scholar]
  9. Elliott K.E.C., Branton S.L., Evans J.D., Magee C.L., Peebles E.D. Onset of the humoral response of layer chicks vaccinated in ovo with strain F Mycoplasma gallisepticum vaccine and evidence of male-biased mortality. Poult. Sci. 2022;101 doi: 10.1016/j.psj.2022.101761. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Elliott K.E.C., Branton S.L., Evans J.D., Peebles E.D. Early post-hatch survival and humoral immune response of layer chickens when in ovo vaccinated with strain F Mycoplasma gallisepticum. Poult. Sci. 2018;97:3860–3869. doi: 10.3382/ps/pey282. [DOI] [PubMed] [Google Scholar]
  11. Fatemi S., Macklin A.K.S., Zhang L., Mousstaaid A., Poudel S., Poudel I. Improvement in the immunity- and vitamin D3 activity-related gene expression of coccidiosis-challenged Ross 708 broilers in response to the in ovo injection of 25-hydroxyvitamin D3. Animals (Basel) 2022;12:2517. doi: 10.3390/ani12192517. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Geyra A., Uni Z., Sklan D. Enterocyte dynamics and mucosal development in the posthatch chick. Poult. Sci. 2001;80:776–782. doi: 10.1093/ps/80.6.776. [DOI] [PubMed] [Google Scholar]
  13. Heller E.D., Leitner G., Drabkin N., Melamed D. Passive immunisation of chicks against Escherichia coli. Avian Pathol. 1990;19:345–354. doi: 10.1080/03079459008418685. [DOI] [PubMed] [Google Scholar]
  14. Hincke M.T., Da Silva M., Guyot N., Gautron J., McKee M.D., Guabiraba-Brito R., Réhault-Godbert S. Dynamics of structural barriers and innate immune components during incubation of the avian egg: critical interplay between autonomous embryonic development and maternal anticipation. J. Innate Immun. 2019;11:111–124. doi: 10.1159/000493719. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Hy-Line International. 2020. Hy-Line W-36 Commercial Layers Management Guide. Accessed Feb. 2022.https://www.hyline.com/filesimages/Hy-Line-Products/Hy-Line-Product-PDFs/W-36/36%20COM%20ENG.pdf.
  16. Joseph N.S., Lourens A., Moran Jr E.T. The effects of suboptimal eggshell temperature during incubation on broiler chick quality, live performance, and further processing yield. Poult. Sci. 2006;85:932–938. doi: 10.1093/ps/85.5.932. [DOI] [PubMed] [Google Scholar]
  17. Lindsey L.L., Elliott K.E.C., Fatemi S.A., Evans J.D., Mousstaaid A., Gerard P.D., Peebles E.D. Variable effects of the in ovo administration of an Escherichia coli vaccine in the amnion or air cell on commercial layer embryo and hatchling development. Poultry. 2022;1:278–290. [Google Scholar]
  18. Malkinson M. The transmission of passive immunity to Escherichia coli from mother to young in the domestic fowl (Gallus domesticus) Immunology. 1965;9:311–317. [PMC free article] [PubMed] [Google Scholar]
  19. Molenaar R., Reijrink I.A.M., Meijerhof R., Van den Brand H. Relationship between hatchling length and weight on later productive performance in broilers. Worlds Poult. Sci. J. 2008;64:599–604. [Google Scholar]
  20. Montgomery R.D., Boyle C.R., Lenarduzzi T.A., Jones L.S. Consequences to chicks hatched from Escherichia coli-inoculated embryos. Avian Dis. 1999;43:553–563. [PubMed] [Google Scholar]
  21. Moura L., Vakharia V., Liu M., Song H. In ovo vaccine against infectious bursal disease. Int. J. Poult. Sci. 2007;11:770–775. [Google Scholar]
  22. Moriuchi S., Deluca H.F. Metabolism of vitamin D3 in the chick embryo. Arch. Biochem. Biophys. 1974;164:165–171. doi: 10.1016/0003-9861(74)90018-6. [DOI] [PubMed] [Google Scholar]
  23. Negash T., Al-Garib S.O., Gruys E. Comparison of in ovo and post-hatch vaccination with particular reference to infectious bursal disease. A review. Vet. Q. 2004;26:76–87. doi: 10.1080/01652176.2004.9695170. [DOI] [PubMed] [Google Scholar]
  24. Onderci M., Sahin N., Sahin K., Cikim G., Aydìn A., Ozercan I., Aydìn S. Efficacy of supplementation of α-amylase producing bacterial culture on the performance, nutrient use, and gut morphology of broiler chickens fed a corn-based diet. Poult. Sci. 2006;85:505–510. doi: 10.1093/ps/85.3.505. [DOI] [PubMed] [Google Scholar]
  25. Peebles E.D. In ovo applications in poultry: a review. Poult. Sci. 2018;97:2322–2338. doi: 10.3382/ps/pey081. [DOI] [PubMed] [Google Scholar]
  26. Peebles E.D., Pansky T., Doyle S.M., Boyle C.R., Smith T.W., Latour M.A., Gerard P.D. Effects of dietary fat and eggshell cuticle removal on egg water loss and embryo growth in broiler hatching eggs. Poult. Sci. 1998;77:1522–1530. doi: 10.1093/ps/77.10.1522. [DOI] [PubMed] [Google Scholar]
  27. Ricks C.A., Avakian A., Bryan T., Gildersleeve R., Haddad E., Ilich R., King S., Murray L., Phelps P., Poston R., Whitfill C., Williams C. In ovo vaccination technology. Adv. Vet. Med. 1999;41:495–515. [PubMed] [Google Scholar]
  28. Rinaudo M.T., Curto M., Bruno R. Lipase and alpha amylase of chick embryo pancreas and of chickens of various ages. Boll. Soc. Ital. Biol. Sper. 1979;55:86–92. [PubMed] [Google Scholar]
  29. SAS Institute Inc . SAS Inst. Inc; Cary, NC: 2013. SAS Proprietary Software Release 9.4. [Google Scholar]
  30. Śmialek M., Kowalczyk J., Koncicki A. Influence of vaccination of broiler chickens against Escherichia coli with live attenuated vaccine on general properties of E. coli population, IBV vaccination efficiency, and production parameters - a field experiment. Poult. Sci. 2020;99:5452–5460. doi: 10.1016/j.psj.2020.08.039. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Sokale A.O., Williams C.J., Cummings T.S., Gerard P.D., Bello A., Peebles E.D. Effects of in ovo injection of different doses of coccidiosis vaccine and turn-out times on broiler performance. Poult. Sci. 2018;97:1891–1898. doi: 10.3382/ps/pey028. [DOI] [PubMed] [Google Scholar]
  32. Swelum A.A., Elbestawy A.R., El-Saadony M.T., Hussien E.O.S., Alhotan R., Suliman G.M., Taha A.E., Ba-Awadh H., El-Tarabily K.A., El-Hack M.E.A. Ways to minimize bacterial infections, with special reference to Escherichia coli, to cope with the first-week mortality in chicks: an updated overview. Poult. Sci. 2021;100 doi: 10.1016/j.psj.2021.101039. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Triplett M.D., Zhai W., Peebles E.D., McDaniel C.D., Kiess A.S. Investigating commercial in ovo technology as a strategy for introducing probiotic bacteria to broiler embryos. Poult. Sci. 2018;97:658–666. doi: 10.3382/ps/pex317. [DOI] [PubMed] [Google Scholar]
  34. Uni Z., Noy Y., Sklan D. Development of the small intestine in heavy and light strain chicks before and after hatching. Br. Poult. Sci. 1995;36:63–71. doi: 10.1080/00071669608417837. [DOI] [PubMed] [Google Scholar]
  35. Uni Z., Noy Y., Sklan D. Posthatch changes in morphology and function of the small intestine in heavy- and light-strain chicks. Poult. Sci. 1995;74:1622–1629. doi: 10.3382/ps.0741622. [DOI] [PubMed] [Google Scholar]
  36. Van den Brand H., van de Kraats S.J.F., Sözcü A., Jöerissen R., Heetkamp M.J.W., van den Anker I., Ooms M., Kemp B. Both the rooster line and incubation temperature affect embryonic metabolism and hatching quality in laying hen crossbreds. Poult. Sci. 2019;98:2632–2640. doi: 10.3382/ps/pez007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. van Ginkel F.W., Padgett J., Martinez-Romero G., Miller M.S., Joiner K.S., Gulley S.L. Age-dependent immune responses and immune protection after avian coronavirus vaccination. Vaccine. 2015;33:2655–2661. doi: 10.1016/j.vaccine.2015.04.026. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Weber R., Rodenberg J. Proceedings of XXIst World Veterinary Poultry Association Congress, Bangkok, Thailand, September 16-19, 2019. 2019. Maternal antibodies against Escherichia (E.) coli in newly hatched SPF chickens do not protect against infection and do not interfere with a live E. coli vaccine. [Google Scholar]
  39. Williams C.J., Hopkins B.A. Field evaluation of the accuracy of vaccine deposition by two different commercially available in ovo injection systems. Poult. Sci. 2011;90:223–226. doi: 10.3382/ps.2010-00759. [DOI] [PubMed] [Google Scholar]
  40. Zoetis. 2021a. Poulvac E. coli, dosing and administration. Zoetisus.com. Accessed July 2021. https://www.zoetisus.com/products/poultry/poulvac-e_coli/directions.aspx.
  41. Zoetis. 2021b. Poulvac E. coli, strengthening broiler flocks. Zoetisus.com. Accessed July 2021. https://www.zoetisus.com/products/poultry/poulvac-e_coli/broiler.aspx.
  42. Zoetis. 2021c. Poulvac E. coli, broad-spectrum E. coli protection. Zoetisus.com. Accessed July 2021. https://www.zoetisus.com/products/poultry/poulvac-e_coli/layer.aspx.

Articles from Poultry Science are provided here courtesy of Elsevier

RESOURCES