Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Oct 1.
Published in final edited form as: J Neurosci Methods. 2019 Jul 27;326:108376. doi: 10.1016/j.jneumeth.2019.108376

A Mouse Model for Chronic Intermittent Electronic Cigarette Exposure Exhibits Nicotine Pharmacokinetics Resembling Human Vapers

Xuesi M Shao 1,2, Briana Lopez 1,4, David Nathan 1,4, Julian Wilson 1, Emmanuel Bankole 1, Hayk Tumoyan 1, Alexandra Munoz 1, Jorge Espinoza-Derout 1, Kamrul M Hasan 1, Scarlett Chang 1, Christina Du 1, Amiya P Sinha-Hikim 1,2, Kabirullah Lutfy 1,3, Theodore C Friedman 1,2,4
PMCID: PMC6717674  NIHMSID: NIHMS1536412  PMID: 31361999

Abstract

Electronic cigarettes (E-cig) use is increasing rapidly, particularly among youths. Animal models for E-cig exposure with pharmacokinetics resembling human E-cig users are lacking. We developed an E-cig aerosol exposure system for rodents and a chronic intermittent delivery method that simulates E-cig users who vape episodically during wakefulness and abstain during sleep. Mice were exposed to E-cig in a programmed schedule at very low, low, medium, and high doses defined by duration of each puff, number of puffs per delivery episode and frequency of episodes in the dark phase of a 12/12-h circadian cycle for 9 consecutive days. The plasma nicotine/cotinine levels and their time courses were determined using LC/MS-MS. We assessed the body weight, food intake and locomotor activity of Apolipoprotein E null (ApoE−/−) mice exposed to chronic intermittent E-cig aerosol. Plasma nicotine and cotinine levels were positively correlated with exposure doses. Nicotine and cotinine levels showed a circadian variation as they increased with time up to the maximum nicotine level of 21.8 ± 7.1 ng/mL during the daily intermittent E-cig exposure in the 12-h dark phase and then declined during the light phase when there was no E-cig delivery. Chronic E-cig exposure to ApoE−/− mice decreased body weight, food intake and increased locomotion. Our rodent E-cig exposure system and chronic intermittent exposure method yield clinically relevant nicotine pharmacokinetics associated with behavioral and metabolic changes. The methodologies are essential tools for in vivo studies of the health impacts of E-cig exposure on CNS, cardiovascular, pulmonary, hepatic systems, metabolism and carcinogenesis.

Keywords: E-cigarette, aerosol, chronic intermittent exposure, nicotine, pharmacokinetics

Graphical Abstract

graphic file with name nihms-1536412-f0005.jpg

1. Introduction

Electronic cigarettes (E-cigs) are increasing dramatically in popularity, especially among young adults in the US (The Surgeon General, 2016). E-cigs initially emerged in 2003 as an alternative to conventional cigarettes and have become widely accessible internationally, particularly over the Internet. As a result of this popularity, E-cig sales are part of a multi-billion dollar industry. E-cigs are battery-powered devices that use a resistive heating coil to aerosolize E-liquid containing propylene glycol, glycerin, artificial flavors, typically nicotine and sometimes ethanol as well as water (The Surgeon General, 2016). E-cig devices come in varying nicotine concentrations that typically range from 0 to 36 mg/mL and in many flavors that often appeal to children and young adults. E-cigs generate a visible and inhalable aerosol, not a simple nicotine vapor as incorrectly claimed by the E-cig industry. Mainstream and second-hand E-cig aerosols contain, in addition to nicotine, detectable levels of toxins including carcinogens and heavy metals such as formaldehyde, benzene, acetaldehyde, acrolein, nitrosamines, nickel, cadmium and lead (Goniewicz et al., 2014; Salamanca et al., 2018; Williams et al., 2013). More research is urgently needed to examine whether E-cigs are a safer alternative to combustible cigarettes (Flint and Jones, 2018).

In human vapers, exposure to E-cigs is chronic and intermittent with episodic vapings during wakefulness and abstinence during sleep leading to circadian variations of blood nicotine levels. As with conventional cigarette smoking, the episodic inhalation patterns of E-cig users produce a distinct nicotine Pharmacokinetics (PK) profile (Lopez et al., 2016; Ramoa et al., 2016) that induces cycles of activation, desensitization and resensitization of nicotinic acetylcholine receptors (nAChRs) that contribute to a number of different aspects of nicotine effects, such as addiction and cardiac and blood pressure effects compared to acute or continuous exposure (Benowitz et al., 2002; Matta et al., 2007). For animal models to be relevant to the health of human vapers, the route of administration, the animal exposure doses and the patterns of E-cig exposure should be similar to those of human vapers. We propose that when the nicotine levels in the animal blood are similar to that of human vapers, animal models would have similar exposure to, not only nicotine, but entire E-cig aerosol containing the E-liquid major components propylene glycol and/or glycerin, flavors, other chemicals and toxins (Goniewicz et al., 2014; Salamanca et al., 2018; Williams et al., 2013).

In vivo and in vitro studies recently reported that exposure to E-cig aerosol boosts carcinogen-bioactivating enzymes and increases oxygen free radical production, induces DNA damage in different tissues and inhibits DNA repair as well as induces a dose-dependent increase in the number of revertants in Ames test. This suggests that E-cig exposure enhances the risk of cancers (Canistro et al., 2017; Lee et al., 2018). The American Association for Cancer Research (AACR., 2015) and the American Society of Clinical Oncology (ASCO, 2016) both recognize that E-cigs can have detrimental consequences as tobacco use and should be further researched to understand their harmful effects (England et al., 2015; The Surgeon General, 2016). With this regard, clinically-relevant in vivo E-cig exposure methods and animal models to mimic the effects of E-cigs on human health have been limited. We developed and filed a patent on an E-cig aerosol generation and exposure system that delivers aerosol to rodents with characteristics comparable to those inhaled by human vapers. We developed a mouse chronic intermittent E-cig aerosol delivery method correlating blood nicotine/cotinine levels and chronic E-cig exposure doses. We examined the nicotine circadian PK in mice to resemble that of human vapers. To test the methodology, we examined the effects of chronic intermittent E-cig exposure on body weight, food intake and local motor activity using ApoE−/− mice on a western diet (WD) that have been well characterized for studying the cardiovascular effects of conventional cigarettes (Lo Sasso et al., 2016). We aimed to develop a mouse model for studying the effects of complex E-cig aerosol as that in genuine E-cigs obtained in the market. This model can thus be used for different research needs such as the consequences and the pathophysiology of chronic E-cig exposure on cardiovascular, pulmonary and hepatic systems, CNS, metabolism as well as carcinogenesis, toxicology and teratogenicity.

2. Materials and Methods

2.1. Animal handling

Animal handling and experimentation were in accordance with the recommendation of the American Veterinary Medical Association and were approved by the Charles R. Drew University School of Medicine and Science Institutional Animal Care and Use Committee. Male adult (8 - 12 weeks old) C57BL/6J wild-type and ApoE−/− mice on a C57BL/6J background were purchased from the Jackson Laboratory (Bar Harbor, ME). Mice were housed 5 per cage under controlled temperature (22°C) and photoperiod (12-h light and 12-h dark cycle).

2.2. E-cig aerosol generation and exposure system

We developed an E-cig aerosol generation and rodent exposure system (patent number PCT/US17/54133) (Fig. 1). The product EcigAero (3 eCig mouse model) is available from our collaborator at AutoMate Scientific, Inc. (Berkeley, CA). The system includes an aerosol exposure chamber that holds up to 5 unrestrained, unanesthetized, freely-moving mice, E-cig holders and programmable E-cig activation control unit. The device was connected to a pressurized air source. The air pressure activates the E-cigs and generates an appropriate flow of E-cig aerosol to the mouse exposure chamber. The E-cig aerosol concentrations in air in the animal exposure chamber were controlled by the number of activated E-cigs, the air pressure and air-flow rate as well as puff duration and frequency of puffs. The air pressure and flow can be readily regulated and were continuously monitored with pressure gauges and flow meters during experiments. We adjusted the flow to 0.8 to 1.0 L/min when E-cigs were activated. When the E-cigs were not activated, fresh air-flow was maintained at a rate of ~40 air change per hour (ACH) based on the volume of the rodent exposure chamber. This air flow eliminates residue aerosol from the chamber and provides fresh air for the mice to breath. Classic tobacco flavor bluCig PLUS E-cig tanks containing high (2.4%) nicotine were used in all experiments. All E-cigs and supplies were purchased on the bluCig website. Batteries were charged every day and were replaced every other week for the chronic experiments. During the light phase of 12-h time (0900 to 2100), mice were returned to their home cages, no aerosol was delivered, and the E-cig chambers were cleaned. Food and water were provided ad lib during both light and dark phases.

Fig. 1:

Fig. 1:

A. E-cigarette aerosol generation and exposure system for mice. Chamber with mouse without E-cig aerosol (B) and with E-cig aerosol (C). D. Blu-cig E-cigs used in the study.

2.3. Dose-response and time-course of plasma nicotine/cotinine levels

The E-cig system was programmed to generate E-cig aerosol exposure at the following doses during the dark phase of 12/12-h dark/light cycles for 9 days: Very low dose: Puff duration was 3 seconds with 2 puffs with an inter-puff interval of 30 sec per episode, 1 episode per hour amounting to a total 12 episodes during the 12-h dark phase of 12/12-h dark/light cycles. Low dose: Puff duration was for 4 seconds with 3 puffs with an inter-puff interval of 30 sec per episode, 1 episode per hour amounting to a total of 12 episodes during the 12-h dark phase. Medium dose: Puff duration was for 4 seconds with 3 puffs with an inter-puff interval of 30 sec per episode, 1 episode per 30 min amounting to a total 24 episodes during the 12-h dark phase. High dose: Puff duration was for 4 seconds with 8 puffs with an inter-puff interval of 25 sec per episode, 1 episode per 20 min amounting to a total of 36 episodes during the 12-h dark phase. Blood was collected 30 to 60 min after the last exposure.

For the time-course experiment, we used the medium dose as defined above. Mice were exposed to E-cig aerosol every day for 9 days from 0 to 12 h (the dark phase of 12/12-h dark/light cycles) and the blood was collected at 3, 6, 12, 18 and 24 h.

2.4. Chronic Long-term Exposure Studies

For the chronic long-term exposure studies, eight-week old ApoE−/− male mice were fed with a WD containing 40% of calories from fat, 43% calories from carbohydrate, and 17% of calories from protein (D12079B; Research Diets, New Brunswick, NJ) and exposed to aerosol from bluCig PLUS Classic tobacco E-cigs containing 2.4% nicotine for 12 weeks. Our chronic intermittent E-cig exposure protocol was: puff duration, 4 sec; 8 puffs per vaping episode with an inter-puff interval of 25 sec; one vaping episode every 30 min during the 12-h dark phase of 12/12-h dark/light cycle. Control mice were exposed to saline aerosol (Shao et al., 2013) with an aerosol generation and exposure device (AeroDeliver™ Afasci Inc., Burlingame, CA) in a chronic intermittent schedule comparable to the E-cig exposure.

2.5. Plasma Nicotine and Cotinine Assays

Thirty to 60 min after the last exposure (Fig. 2) and at the time points indicated in Fig. 3, blood samples were collected into an EDTA-containing tube by cardiac puncture and centrifuged within 15 min. Plasma was collected and stored at −20° C until they were shipped to NMS Labs (Willow Grove, PA) for measurements of nicotine and its metabolite, cotinine. The analysis was performed on a Waters TQD MS with ACQUITY UPLC system equipped with an ACQUITY HSS T3 2.1 × 50mm, 1.8 μm analytical column, with in-line filter and positive-ion electrospray mass spectrometry (LC-MS/MS). The detection limit for nicotine is 2.5 ng/mL and for cotinine is 5 ng/mL.

Fig. 2:

Fig. 2:

Plasma nicotine (left panel) cotinine (right panel) concentrations (mean ± SE) as a function of E-cig dose. Mice were exposed to different doses of E-cigs as defined in the Methods section during the dark phase of 12/12-h circadian dark/light cycles for 9 days. Mice were sacrificed 30-60 min following the last exposure and plasma nicotine and cotinine were measured (n = 4-5 mice per group, *P < 0.05 vs very low dose with MANOVA analysis).

Fig. 3:

Fig. 3:

Plasma nicotine and cotinine concentrations (mean ± SE) as a function of time. Mice were exposed to intermittent E-cig aerosol for 0 to 12 h in the dark phase of 12/12-h circadian dark/light cycle (shaded area indicates exposure to E-cigs) for 9 consecutive days. The exposure dose: puff duration = 4 sec; 3 puffs per episode with an inter-puff interval = 30 sec; one episode per 30 min amounting to a total 24 episodes during the 12-h dark phase. Mice were returned to their home cages without E-cig exposure during the 12-h light phase of the dark/light cycles. Blood samples were taken in day 9 by the end of the h time point indicated. Time point 0 is the time right before the first episode of daily 12 h exposure of day 9. It is equivalent to time point 24 h of day 8. Plasma nicotine and cotinine levels were measured (n = 3 - 5 mice per time point).

2.6. Locomotion Assay

Mice were tested for locomotor activity in the open field following a 10-week E-cig exposure at a chronic intermittent E-cig exposure protocol. Two h after the last exposure, mice were placed in a clean mouse cage that used as the open field arena. The distance traveled, used as a measure of motor activity, was recorded for 5 min using a video camera placed above the arena. The videos were then analyzed using the TopScan software (Clever Sys. Inc., Reston, VA, USA).

2.7. Statistical analyses

Statistical analyses were performed using GraphPad Prism 5 (GraphPad Software, Inc., San Diego, CA). Data are expressed as mean ± SD in text. The data of plasma nicotine and cotinine levels vs. doses were analyzed with the multivariate analysis of variance (MANOVA) method since there are two dependent variables. For the effect of E-cigs on body weight (Fig. 4A), data were analyzed using two-way repeated measures analysis of variance (ANOVA) followed by Holm-Sidak method for multiple comparison (Aickin and Gensler, 1996). For the effect of E-cigs on cumulative food intake (Fig. 4B), data were analyzed using multiple linear regression and the slopes of the two groups were compared. Student t test was used for analyzing the locomotion activities in Fig 4C. P ≤ 0.05 was considered statistically significant. For PK parameters, the area under concentration-time curve (AUC) was estimated with the linear trapezoidal method. The half-life (t1/2) was estimated by the linear regression of the semi-logarithmic concentrations versus time data.

Fig. 4.

Fig. 4.

Effects of E-cigs exposure on body weight, food intake and locomotion in Apo E null mice. A) Body weight; data were analyzed using repeated measures ANOVA, P = 0.0046 for Group x time interaction term showing a significant difference in the time trend between the two groups: E-cig exposed vs. saline aerosol exposed as control with an aerosol generation and exposure device. e-cig: E-cig. B) Cumulative food intake; data were analyzed with multiple linear regression, showing a highly significant difference in slope between the two groups, P < 0.00001. C) Locomotor Activity (travel distance) analyzed by Student t test. All values are means ± SE (n = 5 mice per group, *P < 0.05).

3. Results

Since rodents are nocturnal animals, we exposed mice to E-cig aerosol when they are active during the dark phase of their circadian cycle. No E-cig aerosol was delivered during the 12-h light phase when mice slept.

We adjusted the chronic intermittent E-cig aerosol delivery parameters to produce different exposure doses (refer to Materials and Methods) that resulted in very low, low, medium and high plasma nicotine and cotinine levels (Fig. 2). The medium dose yielded nicotine/cotinine levels comparable to the blood nicotine concentrations found in E-cigs vapers. These results suggest that our system can produce mouse models with various nicotine/cotinine levels that positively correlate to E-cig exposure doses defined by duration of each puff, number of puffs per delivery episode and frequency of episodes in the dark phase of a 12/12-h circadian cycle including a medium dose that generates clinically relevant nicotine/cotinine levels.

Fig. 3 shows the time course of nicotine and cotinine levels at 0, 3, 6, and 12 h during chronic intermittent E-cig exposure and then 6 and 12 h after the exposure on the 9th day of the experiment. Nicotine and cotinine levels increased during E-cig exposure from 3 h to the nicotine levels (at 12 h) of 18.4 ± 5.9 ng/mL with cotinine levels rising to 85.4 ± 35 ng/mL and then declined to baseline following removal from the E-cig exposure system. We analyzed the PK parameters based on the nicotine concentration vs. time data. The maximum nicotine concentration Cmax was 21.75 ± 7.1 ng/mL, the time to reach Cmax (Tmax) was 10.5 ± 3.0 h, the AUC from 0 to 24 h (AUC0-24 h) was 239 ± 56 ng/mL and the t1/2 was 2.1 h. These data suggest that our chronic intermittent E-cig dosing method that simulates human vaping behavioral pattern can produce the PK in mouse models with nicotine and cotinine levels similar to human vapers.

Fig. 4 shows the effects of E-cigs (with 2.4% nicotine for 12 weeks) on body weight, food intake and locomotion in ApoE−/− mice. Chronic exposure to E-cigs led to decreased body weight (4A) and food intake (4B) compared to saline aerosol-treated mice. Fig. 4C shows that mice with chronic exposure to E-cig aerosol had increased locomotor activity, as evidenced by more distance traveled by these mice compared to saline aerosol-exposed controls. These results demonstrate that chronic intermittent E-cig exposure alters physiologic parameters of caloric intake, weight gain and locomotor activity similar to nicotine in long-term chronic exposure studies (Welzl et al., 1988).

4. Discussion

Although previous rodent nicotine inhalation exposure models have been developed such as conventional smoking machines where rodents are exposed to tobacco cigarettes (Beven, 1976; Moir et al., 2008), nicotine vapor (George et al., 2010) and nicotine aerosol (Shao et al., 2018; Shao et al., 2013), our E-cig aerosol generation and exposure system is specifically designed for studying the effects of E-cigs in vivo. The exposure system is efficient, programmable and non-invasive that minimizes distress in mice during exposure. The exposure doses for rodent models are regulated by varying the number of E-cig simultaneously activated, puff duration, inter-puff interval and puff frequency using software. The system ensures an adequate and controllable amount of exposure of E-cig aerosol components including nicotine that enters the circulation and the brain quickly thereby achieving blood nicotine concentrations/PK in rodents comparable to human E-cig users making it clinically relevant. E-cig holders to fit different brands and different sizes of E-cigs can be provided to meet investigators’ needs for studies of, in addition to nicotine, potential toxins contained in a variety of “real” E-cigs available in the market. The system provides an important methodology for studying both acute effects (e.g., vaping one E-cig) and chronic effects of E-cigs, such as multiple E-cigs a day for a long period of time in a home cage-like environment.

The typical peak venous plasma nicotine concentrations in chronic conventional cigarette smokers range between 19 to 50 ng/mL (Hukkanen et al., 2005) and typical plasma cotinine levels are 100 to 300 ng/mL (Benowitz et al., 2009). The peak venous plasma nicotine concentrations in E-cig vapers range between 8 to 30 ng/mL (Farsalinos et al., 2014; St Helen et al., 2016; Vansickel and Eissenberg, 2013) depending on the nicotine concentrations in the E-liquid, puff topography, and whether vapers are experienced or inexperienced. Cotinine levels reported in E-cig users were 60.6 ± 34 ng/mL (Flouris et al., 2013). These studies are, however, for a short duration, e.g., within 1-3 h exposure (Farsalinos et al., 2014; St Helen et al., 2016; Vansickel and Eissenberg, 2013). Circadian PK for chronic E-cig exposure have not been reported, although there are reports for PK of chronic tobacco cigarette smokers (Benowitz et al., 1983) and we reported circadian PK in rats exposed to chronic intermittent nicotine aerosol (Shao et al., 2018). Here we report circadian nicotine PK in mice exposed to chronic intermittent E-cig aerosol and show a pattern similar to tobacco smokers with the peak nicotine/cotinine levels within the range of E-cig vapers. We designed our experiment to expose mice to E-cigs at night when mice are active, and use the term circadian PK to refer to daily variation of plasma nicotine levels with the higher levels at night as others have (Satoh et al., 2006).

Elimination half-life t1/2 is a measure of the rate of removal of drug from the systemic circulation, t1/2 changes as a function of both clearance and volume of distribution. Drug concentrations in plasma often follow a multicomponent pattern (mostly bi-exponential) of decline. When the dosing stops, the drug will be initially cleared from plasma as expected while the drug distributes into secondary body compartments (tissues), but will eventually drop to a point at which net diffusion from the secondary compartments to the blood begins. This slow equilibration produces a prolongation of the half-life of the drug, referred to as the terminal half-life (Laurence L. Brunton et al., 2018). It has been reported that plasma nicotine t1/2 is much shorter in rodents than in primates; 45 min in the rat and 6-7 min in the mouse vs 2 h in humans and nonhuman primates (Matta et al., 2007). This notion is based on PK parameters estimated by acute single dose administration (i.v. i.p. or i.c.) of nicotine in rats (Harris et al., 2008; Lesage et al., 2007) and in mice (Alsharari et al., 2015; Damaj et al., 2007; Zhou et al., 2010). These analyses did not differentiate initial (α phase) t1/2 (t1/2α) and terminal t1/2 (t1/2β). Jung and colleagues (Jung et al., 2001) differentiated t1/2α and terminal t1/2β, and found t1/2β to be 144 min in rats if nicotine base is injected (i.v.). In our chronic intermittent E-cig delivery conditions, the decline phase is a function of clearance and redistribution of nicotine from the tissue to the blood which would start immediately after the end of the dosing. Therefore, we expect a prolonged t1/2 as t1/2β, without t1/2α. The volume of distribution of nicotine in rats is 4.7 to 5.7 L/Kg (Craig et al., 2014; Kyerematen et al., 1988) compared to human values of 2.2 to 3.3 L/Kg with the volume of distribution in mice not reported in the literature. A high volume of distribution shortens the initial t1/2α in the blood due to quick distribution into the tissue and prolongs terminal t1/2β due to slow equilibration as nicotine distributes out of storage tissue (Benowitz et al., 1990). Therefore, it enlarges the difference in t1/2 between rodents and humans if t1/2α and t1/2β are not differentiated in an acute single bolus dose of nicotine delivery in rodents. In contrast, in chronic intermittent exposure conditions, the high volume of distribution reduces the difference in t1/2 between rodents and humans. By inspecting Fig 1 in Benowitz et al. (Benowitz et al., 1982), t1/2 in human chronic smoking conditions is ~ 4 h. We showed that t1/2 in chronic intermittent nicotine aerosol exposure conditions in rats is 4.5 h (Shao et al., 2018). In this study, we report t1/2 to be 2.1 h in chronic intermittent E-cig exposure in mice. Another PK parameter AUC represents total drug exposure across time. We found the AUC from 0 to 24 h (AUC0-24 h) of 239 ± 56 ng/mL in daily chronic intermittent E-cig exposure. This value is lower than those of smokers who smoked their usual brands of tobacco cigarettes and higher than those who smoked low-nicotine tobacco cigarettes (Benowitz et al., 1982). This is consistent with published data in the literature that blood nicotine levels are lower in E-cig users than conventional cigarette smokers (Farsalinos et al., 2014; St Helen et al., 2016; Vansickel and Eissenberg, 2013). Our PK study in chronic intermittent E-cig exposure conditions in rodents is more clinically relevant as the nicotine exposure pattern of tobacco and E-cig users is chronic and intermittent. Our findings of PK circadian variation and estimated PK parameters are similar to those of tobacco and E-cig users.

In most previous E-cig exposure animal studies, blood or urine cotinine levels were reported (Lerner et al., 2015; Smith et al., 2015; Sussan et al., 2015). Lerner and colleagues reported on using a commercially available Teague smoking machine modified to generate E-cig “vapor” (Lerner et al., 2015). Cotinine levels of 10.8 ng/mL were reported in mice exposed to bluCig E-cigs on the 3rd day after a 5 h exposure/day for 3 consecutive days, while nicotine levels were not reported. As blood cotinine levels in E-cig users are 60.6 ± 34 ng/ml (Flouris et al., 2013), their cotinine result suggests that their method of E-cig exposure does not lead to relevant levels of cotinine. Alasmari and colleagues (Alasmari et al., 2018) described a nicotine exposure system developed by SCIREQ Scientific Respiratory Equipment Inc., capable of delivering E-cigs to rodents, but did not report nicotine and cotinine levels. Sussan and colleagues (Sussan et al., 2015) used a modified Jaeger-Baumgartner Cigarette Smoke Machine 2080 and found that serum cotinine reached 267±17 ng/mL immediately after the final E-cig exposure. Nicotine levels were not reported. Nicotine is the ligand of nAChRs that mediates most of its actions on health. For E-cig exposure in animal models, exposure dose is affected by many experimental conditions such as nicotine concentrations in the E-liquid, components in the E-liquid from different brands of E-cigs, the power of the E-cig batteries, volume of the animal exposure chamber, patterns and timing of E-cig activations, air flow that activates E-cigs and emission flow from E-cigs as well as air flow between E-cig activations. Reporting only cotinine levels and correlating them to human E-cig users is probably inaccurate to reflect nicotine exposure, and therefore, E-cig exposure. To evaluate E-cig exposure in animal models, blood nicotine levels need to be reported so that clinical relevance of the study can be evaluated and compared among research laboratories.

In previous studies, investigators used room air flow as a control for E-cig aerosol (Lerner et al., 2015; Sussan et al., 2015). In the current study, we used saline aerosol as a control. Aerosol and vapor are physically different. Based on the particle size distribution, aerosol deposits in different parts of the respiratory tracts and that is of important significance in pulmonary physiology and in hemodynamics (Patton and Byron, 2007; Shao et al., 2017). As E-cig emission is an aerosol, not a simple vapor (Ji et al., 2016; Mikheev et al., 2016), saline aerosol is a physiologically more compatible control for studies of the health effects of E-cigs.

Our findings of decreased body weight and food intake in mice exposed to E-cigs are consistent with our previous findings of a decrease in these parameters found when mice were given intraperitoneal injections of nicotine (Mangubat et al., 2012). The nicotine-induced changes in body weight may be due to decreased energy intake, increased energy expenditure, or a combination of the two processes (Tweed et al., 2012). Increased locomotion reflects an increase in dopamine in the reward pathways (Welzl et al., 1988), which is implicated in the rewarding and reinforcing actions of nicotine (Bromberg-Martin et al., 2010; Clarke, 1990). Together, these results on body weight, food intake and locomotion confirm that our E-cig exposure system is capable of delivering adequate amount of nicotine that has effects on various biological systems.

Potential limitations to the current study are that in freely moving whole body E-cig exposure, E-cig aerosol could condense on the fur of mice or inner surface of the chamber that leads to possible dermal and/or oral exposure, e.g., by licking their fur. We estimate that this contributes insignificantly to the blood nicotine levels as we obtained a circadian PK pattern similar to human smokers and as the blood nicotine was undetectable toward the end of the 12-h light phase. We used ApoE−/− mice, a well characterized model to study the effects of conventional cigarettes on cardiovascular diseases, to study body weight, food intake and locomotion. We plan to perform these experiments in wild-type mice, but expect similar results. Another limitation to the study was the exclusion of other brands of E-cigs that may give different levels. Nevertheless, our model is relevant as we used the bluCig E-cigs that are a widely used brand of E-cigs (Wagener et al., 2014). We plan to study other types of E-cigs, including those in the form of pods with Juul being the most popular brand (Kavuluru et al., 2019).

We have reported on the effects of long-term E-cig exposure using our unique E-cig exposure system on cardiac and cardiovascular diseases (Espinoza-Derout et al., 2019a) and hepatic steatosis (Espinoza-Derout et al., 2019b; Hasan et al., 2019). We also are studying the effects of chronic E-cig exposure on breast cancer proliferation. Our E-cig exposure system is capable of studying the effects of gestational exposure of E-cigs on neonatal mice and its longterm cardiac consequences in the adults.

5. Conclusions

In summary: (i) we developed an E-cig aerosol generation and exposure system that ensures adequate and well controlled E-cig aerosol delivery to free-moving rodents through inhalation, (ii) our chronic intermittent E-cig exposure method simulates the circadian smoking pattern of E-cig users, with exposure when mice are awake, (iii) the plasma nicotine/cotinine levels are positively correlated with E-cig exposure doses that is determined by programmed E-cig aerosol generation, (iv) circadian PK in mice with nicotine/cotinine concentrations resembling human vapers can be achieved using our chronic intermittent E-cig exposure method, and (v) chronic intermittent E-cig exposure alters metabolism and locomotion. The rodents in our exposure chamber are not only exposed to nicotine, but also other chemicals contained in commercially available E-cigs, such as propylene glycol and/or glycerin, flavors, and other harmful and potentially harmful chemicals. Our rodent model can be used to study the health effects of these other components of E-cigs. The E-cig aerosol system and the chronic intermittent E-cig exposure method developed in current study can generate clinically relevant rodent models for studies of the impact and the underlying mechanisms of E-cig exposure on addiction, the brain, cardiovascular, pulmonary, and hepatic systems as well as metabolism, carcinogenesis, toxicology and fetal development during pregnancy.

Highlights.

  • An E-cigarette aerosol exposure system for rodents is developed.

  • A chronic intermittent delivery method for mice simulates human vaping behavior.

  • Plasma nicotine/cotinine levels are positively correlated with exposure doses.

  • Clinically relevant circadian pharmacokinetics in mice is achieved.

  • E-cigarette exposure induces metabolic and behavioral changes.

  • The methods can be used for diverse in vivo studies of health impacts of E-cigarettes.

Acknowledgments

Grants: This work was supported by the Diversity-Promoting Institution Drug Abuse Research Program (DIDARP) grant R24DA017298 (to TCF), California Tobacco-Related Disease Research Program (TRDRP) Grant 251P003 (to TCF), TRDRP grant 24RT-0023H (to KL), the NIMHD/NIH Accelerating Excellence in Translational Science (AXIS) grant 2U54MD007598 (to Charles Drew University), NIMHD S21-MD000103 grant (to JE), NHLBI/NIH grant 1R01HL135623-01 (to XMS) and NIDA/NIH grant 1R41DA044788-01 (to XMS).

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Disclosures: No conflict of interest declared.

References

  1. AACR. Electronic Cigarettes: What You Don’t Know Can Hurt You. American Association for Cancer Research [Online], 2015. [Google Scholar]
  2. Aickin M, Gensler H. Adjusting for multiple testing when reporting research results: the Bonferroni vs Holm methods. Am. J. Public Health, 1996; 86: 726–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Alasmari F, Crotty Alexander LE, Drummond CA, Sari Y. A computerized exposure system for animal models to optimize nicotine delivery into the brain through inhalation of electronic cigarette vapors or cigarette smoke. Saudi Pharm J, 2018; 26: 622–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Alsharari SD, King JR, Nordman JC, Muldoon PP, Jackson A, Zhu AZ, Tyndale RF, Kabbani N, Damaj MI. Effects of Menthol on Nicotine Pharmacokinetic, Pharmacology and Dependence in Mice. PLoS One, 2015; 10: e0137070. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. ASCO. ASCO Applauds Food and Drug Administration (FDA) for Regulating E-Cigarettes. American Society of Clinical Oncology: [Online], 2016. [Google Scholar]
  6. Benowitz NL, Bernert JT, Caraballo RS, Holiday DB, Wang J. Optimal serum cotinine levels for distinguishing cigarette smokers and nonsmokers within different racial/ethnic groups in the United States between 1999 and 2004. Am. J. Epidemiol, 2009; 169: 236–48. [DOI] [PubMed] [Google Scholar]
  7. Benowitz NL, Hansson A, Jacob P 3rd. Cardiovascular effects of nasal and transdermal nicotine and cigarette smoking. Hypertension, 2002; 39: 1107–12. [DOI] [PubMed] [Google Scholar]
  8. Benowitz NL, Kuyt F, Jacob P 3rd. Circadian blood nicotine concentrations during cigarette smoking. Clin. Pharmacol. Ther, 1982; 32: 758–64. [DOI] [PubMed] [Google Scholar]
  9. Benowitz NL, Kuyt F, Jacob P 3rd, Jones RT, Osman AL. Cotinine disposition and effects. Clin. Pharmacol. Ther, 1983; 34: 604–11. [DOI] [PubMed] [Google Scholar]
  10. Benowitz NL, Porchet H, Jacob Pr. Pharmacokinetics, metabolism, and pharmacodynamics of nicotine In Wonnacott S, Russell MAH, Stolerman IP, editors. Nicotine psychopharmacology. Oxford University Press: Oxford, 1990: 112–57. [Google Scholar]
  11. Beven JL. Inhalation toxicity studies on cigarette smoke. I. A versatile exposure system for inhalation toxicity studies on cigarette smoke. Toxicology, 1976; 6: 189–96. [DOI] [PubMed] [Google Scholar]
  12. Bromberg-Martin ES, Matsumoto M, Hikosaka O. Dopamine in motivational control: rewarding, aversive, and alerting. Neuron, 2010; 68: 815–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Canistro D, Vivarelli F, Cirillo S, Babot Marquillas C, Buschini A, Lazzaretti M, Marchi L, Cardenia V, Rodriguez-Estrada MT, Lodovici M, Cipriani C, Lorenzini A, Croco E, Marchionni S, Franchi P, Lucarini M, Longo V, Della Croce CM, Vornoli A, Colacci A, Vaccari M, Sapone A, Paolini M. E-cigarettes induce toxicological effects that can raise the cancer risk. Sci. Rep, 2017; 7: 2028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Clarke PB. Mesolimbic dopamine activation--the key to nicotine reinforcement? Ciba Found. Symp, 1990; 152: 153–62; discussion 62-8. [DOI] [PubMed] [Google Scholar]
  15. Craig EL, Zhao B, Cui JZ, Novalen M, Miksys S, Tyndale RF. Nicotine pharmacokinetics in rats is altered as a function of age, impacting the interpretation of animal model data. Drug Metab. Dispos, 2014; 42: 1447–55. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Damaj MI, Siu EC, Sellers EM, Tyndale RF, Martin BR. Inhibition of nicotine metabolism by methoxysalen: Pharmacokinetic and pharmacological studies in mice. J. Pharmacol. Exp. Ther, 2007; 320: 250–7. [DOI] [PubMed] [Google Scholar]
  17. England LJ, Bunnell RE, Pechacek TF, Tong VT, McAfee TA. Nicotine and the Developing Human: A Neglected Element in the Electronic Cigarette Debate. Am. J. Prev. Med, 2015; 49: 286–93. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Espinoza-Derout J, Hasan KM, Shao XM, Jordan MC, Sims C, Lee DL, Sinha S, Simmons Z, Mtume N, Liu Y, Roos KP, Sinha-Hikim AP, Friedman TC. Chronic Intermittent Electronic Cigarette Exposure Induces Cardiac Dysfunction and Atherosclerosis in Apolipoprotein E (ApoE) Knockout Mice. Am. J. Physiol. Heart Circ. Physiol, 2019a. doi: 10.1152/ajpheart.00738.2018. [Epub ahead of print] [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Espinoza-Derout J, Shao XM, Bankole E, Hasan KM, Mtume N, Liu Y, Sinha-Hikim AP, Friedman TC. Hepatic DNA Damage Induced by Electronic Cigarette Exposure Is Associated With the Modulation of NAD+/PARP1/SIRT1 Axis. Front. Endocrinol. (Lausanne), 2019b; 10: 320. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Farsalinos KE, Spyrou A, Tsimopoulou K, Stefopoulos C, Romagna G, Voudris V. Nicotine absorption from electronic cigarette use: comparison between first and new-generation devices. Sci. Rep, 2014; 4: 4133. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Flint SW, Jones AW. The irresponsible promotion of e-cigarettes and Swaptober. Lancet Respir Med, 2018; 6: e3–e4. [DOI] [PubMed] [Google Scholar]
  22. Flouris AD, Chorti MS, Poulianiti KP, Jamurtas AZ, Kostikas K, Tzatzarakis MN, Wallace Hayes A, Tsatsakis AM, Koutedakis Y. Acute impact of active and passive electronic cigarette smoking on serum cotinine and lung function. Inhal. Toxicol, 2013; 25: 91–101. [DOI] [PubMed] [Google Scholar]
  23. George O, Grieder TE, Cole M, Koob GF. Exposure to chronic intermittent nicotine vapor induces nicotine dependence. Pharmacol Biochemistry Behavior, 2010; 96: 104–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Goniewicz ML, Knysak J, Gawron M, Kosmider L, Sobczak A, Kurek J, Prokopowicz A, Jablonska-Czapla M, Rosik-Dulewska C, Havel C, Jacob P 3rd, Benowitz N. Levels of selected carcinogens and toxicants in vapour from electronic cigarettes. Tob. Control, 2014; 23: 133–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Harris AC, Burroughs D, Pentel PR, LeSage MG. Compensatory nicotine self-administration in rats during reduced access to nicotine: an animal model of smoking reduction. Exp. Clin. Psychopharmacol, 2008; 16: 86–97. [DOI] [PubMed] [Google Scholar]
  26. Hasan KM, Friedman TC, Shao X, Parveen M, Sims C, Lee DL, Espinoza-Derout J, Sinha-Hikim I, Sinha-Hikim AP. E-cigarettes and Western Diet: Important Metabolic Risk Factors for Hepatic Diseases. Hepatology, 2019; 69: 2442–54. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Hukkanen J, Jacob P 3rd, Benowitz NL. Metabolism and disposition kinetics of nicotine. Pharmacol. Rev, 2005; 57: 79–115. [DOI] [PubMed] [Google Scholar]
  28. Ji EH, Sun B, Zhao T, Shu S, Chang CH, Messadi D, Xia T, Zhu Y, Hu S. Characterization of Electronic Cigarette Aerosol and Its Induction of Oxidative Stress Response in Oral Keratinocytes. PLoS One, 2016; 11: e0154447. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Jung BH, Chung BC, Chung SJ, Shim CK. Different pharmacokinetics of nicotine following intravenous administration of nicotine base and nicotine hydrogen tartarate in rats. J. Control. Release, 2001; 77: 183–90. [DOI] [PubMed] [Google Scholar]
  30. Kavuluru R, Han S, Hahn EJ. On the popularity of the USB flash drive-shaped electronic cigarette Juul. Tob. Control, 2019; 28: 110–2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Kyerematen GA, Taylor LH, deBethizy JD, Vesell ES. Pharmacokinetics of nicotine and 12 metabolites in the rat. Application of a new radiometric high performance liquid chromatography assay. Drug Metab. Dispos, 1988; 16: 125–9. [PubMed] [Google Scholar]
  32. Brunton Laurence L., Hilal-Dandan Randa, Knollmann Bjorn C.. Pharmacokinetics: The Dynamics of Drug Absorption, Distribution, Metabolism, and elimination In Brunton Laurence L., Hilal-Dandan Randa, Knollmann Bjorn C., editors. Goodman and Gilman’s: The Pharmacological Basis of Therapeutics. McGraw Hill: New York, 2018. [Google Scholar]
  33. Lee HW, Park SH, Weng MW, Wang HT, Huang WC, Lepor H, Wu XR, Chen LC, Tang MS. E-cigarette smoke damages DNA and reduces repair activity in mouse lung, heart, and bladder as well as in human lung and bladder cells. Proc. Natl. Acad. Sci. U. S. A, 2018; 115: E1560–E9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Lemer CA, Sundar IK, Yao H, Gerloff J, Ossip DJ, McIntosh S, Robinson R, Rahman I. Vapors produced by electronic cigarettes and e-juices with flavorings induce toxicity, oxidative stress, and inflammatory response in lung epithelial cells and in mouse lung. PLoS One, 2015; 10: eOl16732. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Lesage MG, Keyler DE, Burroughs D, Pentel PR. Effects of pregnancy on nicotine self-administration and nicotine pharmacokinetics in rats. Psychopharmacology (Berk), 2007; 194: 413–21. [DOI] [PubMed] [Google Scholar]
  36. Lo Sasso G, Schlage WK, Boue S, Veljkovic E, Peitsch MC, Hoeng J. The Apoe(−/−) mouse model: a suitable model to study cardiovascular and respiratory diseases in the context of cigarette smoke exposure and harm reduction. J. Transl. Med, 2016; 14: 146. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Lopez AA, Hiler MM, Soule EK, Ramoa CP, Karaoghlanian NV, Lipato T, Breland AB, Shihadeh AL, Eissenberg T. Effects of Electronic Cigarette Liquid Nicotine Concentration on Plasma Nicotine and Puff Topography in Tobacco Cigarette Smokers: A Preliminary Report. Nicotine Tob Res, 2016; 18: 720–3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Mangubat M, Lutfy K, Lee ML, Pulido L, Stout D, Davis R, Shin CS, Shahbazian M, Seasholtz S, Sinha-Hikim A, Sinha-Hikim I, O’Dell LE, Lyzlov A, Liu Y, Friedman TC. Effect of nicotine on body composition in mice. J. Endocrinol, 2012; 212: 317–26. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Matta SG, Balfour DJ, Benowitz NL, Boyd RT, Buccafusco JJ, Caggiula AR, Craig CR, Collins AC, Damaj MI, Donny EC, Gardiner PS, Grady SR, Heberlein U, Leonard SS, Levin ED, Lukas RJ, Markou A, Marks MJ, McCallum SE, Parameswaran N, Perkins KA, Picciotto MR, Quik M, Rose JE, Rothenfluh A, Schafer WR, Stolerman IP, Tyndale RF, Wehner JM, Zirger JM. Guidelines on nicotine dose selection for in vivo research. Psychopharmacology (Berl.), 2007; 190: 269–319. [DOI] [PubMed] [Google Scholar]
  40. Mikheev VB, Brinkman MC, Granville CA, Gordon SM, Clark PI. Real-Time Measurement of Electronic Cigarette Aerosol Size Distribution and Metals Content Analysis. Nicotine Tob Res, 2016; 18:1895–902. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Moir D, Rickert WS, Levasseur G, Larose Y, Maertens R, White P, Desjardins S. A comparison of mainstream and sidestream marijuana and tobacco cigarette smoke produced under two machine smoking conditions. Chem. Res. Toxicol, 2008; 21: 494–502. [DOI] [PubMed] [Google Scholar]
  42. Patton JS, Byron PR. Inhaling medicines: delivering drugs to the body through the lungs. Nature reviews. Drug discovery, 2007; 6: 67–74. [DOI] [PubMed] [Google Scholar]
  43. Ramoa CP, Hiler MM, Spindle TR, Lopez AA, Karaoghlanian N, Lipato T, Breland AB, Shihadeh A, Eissenberg T. Electronic cigarette nicotine delivery can exceed that of combustible cigarettes: a preliminary report. Tob. Control, 2016; 25: e6–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Salamanca JC, Meehan-Atrash J, Vreeke S, Escobedo JO, Peyton DH, Strongin RM. E-cigarettes can emit formaldehyde at high levels under conditions that have been reported to be non-averse to users. Sci. Rep, 2018; 8: 7559. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Satoh S, Tada H, Murakami M, Tsuchiya N, Li Z, Numakura K, Saito M, Inoue T, Miura M, Hayase Y, Suzuki T, Habuchi T. Circadian pharmacokinetics of mycophenolic Acid and implication of genetic polymorphisms for early clinical events in renal transplant recipients. Transplantation, 2006; 82: 486–93. [DOI] [PubMed] [Google Scholar]
  46. Shao XM, Liu S, Lee ES, Fung D, Pei H, Liang J, Mudgway R, Zhang J, Feldman JL, Zhu Y, Louie S, Xie XS. Chronic intermittent nicotine delivery via lung alveolar region-targeted aerosol technology produces circadian pharmacokinetics in rats resembling human smokers. J Appl Physiol (1985), 2018; 125: 1555–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Shao XM, Lopez-Valdes HE, Liang J, Feldman JL. Inhaled nicotine equivalent to cigarette smoking disrupts systemic and uterine hemodynamics and induces cardiac arrhythmia in pregnant rats. Sci. Rep, 2017; 7: 16974. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Shao XM, Xu B, Liang J, Xie XS, Zhu Y, Feldman JL. Nicotine delivery to rats via lung alveolar region-targeted aerosol technology produces blood pharmacokinetics resembling human smoking. Nicotine Tob Res, 2013; 15: 1248–58. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Smith D, Aherrera A, Lopez A, Neptune E, Winickoff JP, Klein JD, Chen G, Lazarus P, Collaco JM, McGrath-Morrow SA. Adult Behavior in Male Mice Exposed to E-Cigarette Nicotine Vapors during Late Prenatal and Early Postnatal Life. PLoS One, 2015; 10: e0137953. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. St Helen G, Havel C, Dempsey DA, Jacob P 3rd, Benowitz NL. Nicotine delivery, retention and pharmacokinetics from various electronic cigarettes. Addiction, 2016; 111: 535–44. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Sussan TE, Gajghate S, Thimmulappa RK, Ma J, Kim JH, Sudini K, Consolini N, Cormier SA, Lomnicki S, Hasan F, Pekosz A, Biswal S. Exposure to electronic cigarettes impairs pulmonary anti-bacterial and anti-viral defenses in a mouse model. PLoS One, 2015; 10: e0116861. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. The Surgeon General. E-Cigarette Use Among Youth and Young Adults: A Report of the Surgeon General. U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES, Public Health Service, Office of the Surgeon General: Rockville, MD, USA, 2016. [Google Scholar]
  53. Tweed JO, Hsia SH, Lutfy K, Friedman TC. The endocrine effects of nicotine and cigarette smoke. Trends Endocrinol. Metab, 2012; 23: 334–42. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Vansickel AR, Eissenberg T. Electronic cigarettes: effective nicotine delivery after acute administration. Nicotine Tob Res, 2013; 15: 267–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Wagener TL, Meier E, Hale JJ, Oliver ER, Warner ML, Driskill LM, Gillaspy SR, Siegel MB, Foster S. Pilot investigation of changes in readiness and confidence to quit smoking after E-cigarette experimentation and 1 week of use. Nicotine Tob Res, 2014; 16: 108–14. [DOI] [PubMed] [Google Scholar]
  56. Welzl H, Alessandri B, Oettinger R, Batig K. The effects of long-term nicotine treatment on locomotion, exploration and memory in young and old rats. Psychopharmacology (Berl.), 1988; 96: 317–23. [DOI] [PubMed] [Google Scholar]
  57. Williams M, Villarreal A, Bozhilov K, Lin S, Talbot P. Metal and silicate particles including nanoparticles are present in electronic cigarette cartomizer fluid and aerosol. PLoS One, 2013; 8: e57987. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Zhou X, Zhuo X, Xie F, Kluetzman K, Shu YZ, Humphreys WG, Ding X. Role of CYP2A5 in the clearance of nicotine and cotinine: insights from studies on a Cyp2a5-null mouse model. J. Pharmacol. Exp. Ther, 2010; 332: 578–87. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES