Abstract
We have analyzed the contribution of different efflux components to glutathione-mediated abrogation of ciprofloxacin's activity in Escherichia coli and the underlying potential mechanism(s) behind this phenomenon. The results indicated that glutathione increased the total active efflux, thereby partially contributing to glutathione-mediated neutralization of ciprofloxacin's antibacterial action in E. coli. However, the role of glutathione-mediated increased efflux becomes evident in the absence of a functional TolC-AcrAB efflux pump.
TEXT
Antibiotic efflux is the energy-dependent extrusion of antimicrobial substances from a bacterial cell (1). It is one of the most important and clinically relevant mechanisms for mediating antibiotic resistance in Gram-negative bacteria such as Escherichia coli (2, 3). Approximately 5 to 10% of the genes present in the bacterial genome encode transport-related functions, and many of them are part of the antibiotic efflux machinery (4). Importantly, antibiotic efflux is one of the major health concerns due to the fact that a single antibiotic efflux pump can confer simultaneous resistance to a variety of antimicrobials (1, 2). Overall two different groups of components, namely, efflux transporters and outer membrane efflux proteins (OEPs), are part of the bacterial antibiotic efflux machinery. In E. coli, five major families of efflux transporters are reported (2, 5), viz., MFS (major facilitator superfamily), MATE (multidrug and toxic efflux), RND (resistance-nodulation-division), SMR (small multidrug resistance), and ABC (ATP binding cassette). These efflux transporters act in association with one of the four different OEPs in E. coli, namely tolC, yjcP (mdtP), yohG (mdtQ), and ylcB (cusC) (5). Out of the different OEPs mentioned above, TolC is the major channel for antibiotic efflux across the outer membrane of E. coli (5). Notably, TolC can function in conjunction with three different types of transport systems, including RND (viz., AcrAB), MFS (viz., EmrAB), and ABC (viz., MacAB) transporters (6).
Fluoroquinolones such as ciprofloxacin are synthetic antibiotics, having immense therapeutic value against a variety of bacterial pathogens due to their extended antibacterial spectrum and improved systemic distribution inside the host tissues (7). As per our earlier findings (8) and emerging scientific paradigm, many of the classical antibiotics, including ciprofloxacin, kill bacteria through induction of oxidative stress (9–11). We have previously established that the presence of antioxidants, viz., glutathione (GSH), gives protection against the antibacterial action of ciprofloxacin in E. coli (8, 12), which was attributed to GSH-mediated neutralization of oxidative stress caused by ciprofloxacin in E. coli. According to a subsequent independent report (13), GSH-mediated protection against another fluoroquinolone, namely, norfloxacin, is more efficient in strains having an intact efflux system encoded by tolC-acrAB, suggesting that the antibiotic efflux machinery could partially contribute to GSH-mediated protection against fluoroquinolones.
The aim of the present study was to analyze the contribution of efflux machinery components toward the GSH-mediated decrease in ciprofloxacin activity against E. coli using Keio collection single-gene deletion mutants (14). This comprehensive analysis involved 27 efflux transporters and 4 OEP mutants, including members of all known transporter families and OEPs. The growth kinetics of strains showing altered ciprofloxacin susceptibility in the presence or absence of GSH was determined. The effect of GSH on expression of efflux components that potentially have a role in the above-mentioned phenotype was subsequently monitored. We further analyzed the effect of GSH on the total efflux activity of wild-type E. coli and the corresponding efflux mutants having a plausible role in GSH-mediated decreased ciprofloxacin activity. In addition, the effect of GSH on ciprofloxacin accumulation in wild-type and mutant E. coli strains was investigated to determine whether uptake of ciprofloxacin in a bacterial cell is affected by the presence of GSH.
The impact of different drug transporters and OEPs on ciprofloxacin susceptibility and GSH-mediated decreased ciprofloxacin activity in E. coli was analyzed using Keio collection mutants. The ciprofloxacin susceptibilities of the wild-type parent strain BW25113 and 31 different efflux mutants were determined in both the presence and absence of 10 mM GSH. The preparation and handling of the glutathione solution, the microbial culture growth conditions, and the antibiotic susceptibility testing methods were essentially the same as those previously described by Goswami et al. (12). Mueller-Hinton (MH) agar was used for MIC determination as described previously (12) for assessing the antibiotic susceptibility of a given strain. Out of the 31 Keio collection mutants analyzed in the study, 10 strains carried a deletion in RND, 12 in MFS, 2 each in SMR and ABC, 1 in MATE, and 4 in the OEP components of the efflux machinery (for details, see Table S1 in the supplemental material). Out of the 31 efflux mutants tested, only 2 drug transporter mutants, namely, JW0451 and JW0452 (carrying acrB and acrA mutations, respectively) and 1 OEP mutant, JW5503 (with tolC mutation), exhibited 4-fold increased ciprofloxacin susceptibility in comparison to that for BW25113 (Table 1). The unaltered ciprofloxacin susceptibilities of the remaining efflux pump mutants confirmed the auxiliary functions of these efflux components toward ciprofloxacin resistance, substantiating several earlier reports (15, 16). The presence of GSH decreased the antibacterial activity of ciprofloxacin to a lesser extent in JW0452, JW0451, and JW5503 than in BW25113, both in terms of absolute values and fold changes. This signifies that antibiotic efflux not only determines the innate ciprofloxacin susceptibility of E. coli but also plays an important role in the GSH-mediated phenotype. Our conclusions extend the earlier findings of Dhamdhere et al. (17) that GSH-mediated protection against fluoroquinolones is more efficient but not completely dependent on efflux pumps. The addition of GSH had no effect on the antibacterial activity of ciprofloxacin for the remaining mutants (see Table S1 in the supplemental material), signifying that these efflux functions do not play a significant primary role in the phenotype observed.
TABLE 1.
Strain name | MIC of ciprofloxacin (ng/ml) |
GSH-mediated increase in MIC (ratio of without GSH/with GSH) (fold change) | |
---|---|---|---|
Without GSH | With GSH | ||
BW25113 | 16 | 256 | 16 |
JW5503 | 4 | 32 | 8 |
JW0452 | 4 | 32 | 8 |
JW0451 | 4 | 32 | 8 |
Only those strains where a change in MIC was observed in the presence or absence of GSH compared with that for the wild-type parent strain BW25113 are shown. The ciprofloxacin MICs for all remaining efflux pump mutants (mentioned in Table S1 in the supplemental material) were unaltered compared with that for BW25113, both in the presence and absence of 10 mM GSH.
The growth curves of BW25113, JW0452, JW0451, and JW5503 were generated by incubating 1% inocula (optical density at 600 nm [OD600] of ∼2.0) of these strains overnight under different growth conditions in Luria-Bertani (LB) medium. The bacterial growth was monitored by determining the OD600 in an automated optical density monitoring system (Bioscreen C; OY Growth Curves, Finland) at 37°C with a continuous medium shaking mode. The growth curve data revealed that strains JW0452, JW0451, and JW5503 were relatively slow growing (generation times of 52, 55, and 58 min, respectively) compared to the parent strain BW25113 (∼47 min) in LB medium (Fig. 1A). The presence of GSH decreased the growth rate of all four strains by increasing their generation times to 69, 71, 61, and 55 min, respectively (Fig. 1B). The growth profiles of BW25113, JW0452, JW0451, and JW5503 in LB medium substantiated our MIC data. They further indicated that irrespective of the growth medium, the acrA, acrB, and tolC mutants exhibited similar responses to GSH and/or ciprofloxacin. The above-mentioned efflux mutants failed to grow in the presence of 4 ng/ml ciprofloxacin, corroborating our MIC data (Fig. 1C). The presence of GSH allowed the parent strain BW25113 to grow successfully at 16 ng/ml ciprofloxacin (Fig. 1D), whereas the same strain failed to grow under analogous conditions in the absence of GSH (Fig. 1E). Additionally, GSH could not rescue the growth of JW0452, JW0451, and JW5503, unlike that of BW25113, when grown with 16 ng/ml ciprofloxacin (Fig. 1D).
Involvement of GSH in regulating the expression of acrA, acrB, and tolC, thereby affecting the extent of GSH-mediated protection against ciprofloxacin in mutant strains, was investigated next. The expression levels of acrA, acrB, and tolC were compared in the presence and absence of exogenously added GSH, using reverse transcription-quantitative PCR (RT-qPCR) in logarithmically growing BW25113 cells (OD600 of ∼0.5). RNA was extracted using NucleoSpin RNA II (Macherey-Nagel, Germany), according to the manufacturer's instructions. All downstream steps were the same as those described previously (18), and the resulting threshold cycle (CT) values were normalized using rpoD as a reference gene. The acrA, acrB, tolC, and rpoD primer details are given in Table S2 in the supplemental material. The results of the analysis showed that expression of acrA, acrB, and tolC is not affected by the exogenous GSH (see Fig. S1 in the supplemental material). The expression of acrA, acrB, and tolC is reported to be regulated essentially at the transcription level (5, 19) and not at the translational/posttranslational level to the best of our knowledge. It therefore rules out the possibility of exogenous GSH regulating the expression levels of acrA, acrB, and tolC in wild-type E. coli.
GSH might be playing a role in determining the total efflux activity of the E. coli as JW0452, JW0451, and JW5503 showed decreased extents of GSH-mediated protection against ciprofloxacin compared with that of BW25113. Accordingly, the total efflux activities of BW25113, JW0452, JW0451, and JW5503 were determined during mid-exponential growth phase (OD600 of ∼0.5) in the presence and absence of 10 mM GSH, employing flow cytometry as described previously (20). The assay was based on the principle of increased retention of the fluorescent dye Hoechst 33342 inside bacterial cells by virtue of its diminished efflux activity. The individual representative histograms show the extent of Hoechst 33342 dye retained in different bacterial strains (Fig. 2A). The fluorescence of the dye (arbitrary units [AU]) retained in the bacteria is shown in the bar charts (Fig. 2B). The functionality of the assay system was ascertained by significantly increased fluorescence of BW25133 cells treated by 70% isopropanol in comparison to that in the live bacterial cells (data not shown). Fluorescent dye retention levels within diverse E. coli strains were quantified and compared. A negligible amount of fluorescent dye was retained inside BW25113 (mean ± standard error of the mean [SEM] of 3.5 ± 0.1), signifying a healthy level of preexisting efflux activity in the wild-type parent E. coli strain. The presence of GSH did not further decrease the dye retention in BW25113 as it was nominal to start with. Therefore, it becomes tricky to recognize the effect of GSH on the efflux activity of wild-type E. coli BW25113. On the other hand, notable increases in the intracellular fluorescence levels of JW0452, JW0451, and JW5503 (between 52 and 61%) were seen in comparison to that in BW25113 (P < 0.01) due to build up of the fluorescent dye within these strains because of their compromised efflux activity. In particular, the tolC mutant JW5503 exhibited the highest dye retention (61.1 ± 0.8) compared to that in BW25113, corroborating several earlier reports demonstrating the importance of tolC (2, 5). JW0451 and JW0452 exhibited ∼56 and 48% increases in fluorescence, respectively, in comparison to that in the parent strain. Interestingly, the presence of GSH caused a significant decrease in fluorescence in all of the efflux pump mutants (P < 0.05). The presence of GSH diminished the fluorescence due to dye retention in JW0451 and JW0452 by ∼38 and 25%, respectively (P < 0.01). Similarly, JW5503 exhibited a decrease in the fluorescence of the dye by ∼11% in the presence of GSH (P < 0.05). GSH-mediated increases in the efflux activities of JW0452, JW0451, and JW5503 signify that GSH increases the total efflux to a significant extent in E. coli lacking a functional TolC-AcrAB pump. Our proposition is further supported by a previous report showing GSH-mediated increase in the activities of potassium efflux channels, namely, KefB and KefC, in E. coli (21). Moreover, partial restoration of the severely compromised efflux activity of E. coli in the absence of a functional TolC-AcrAB pump through activation of enduring machinery indicates the auxiliary role of efflux in the GSH-mediated phenotype.
As antibiotic efflux ostensibly contributed to GSH-mediated decreased ciprofloxacin activity, we further investigated the effect of exogenous GSH on ciprofloxacin accumulation in wild-type E. coli BW25113 and the corresponding TolC-AcrAB mutants (i.e., JW0452, JW0451, and JW5503). This analysis becomes relevant as GSH-mediated increased efflux should simultaneously be epitomized by reduced accumulation of ciprofloxacin inside affected bacteria. The effect of GSH supplementation on ciprofloxacin accumulation in BW25113, JW0452, JW0451, and JW5503 was evaluated by fluorimetry as described previously (22). The amount of ciprofloxacin accumulated was calculated by comparison with a standard curve for ciprofloxacin (20 to 2000 ng/ml) in 0.1 M glycine-hydrochloride (pH 3.0). The results are expressed as nanograms of ciprofloxacin incorporated per milligram dry weight of bacteria. The results of the ciprofloxacin accumulation assay are shown in Fig. 3. The presence of GSH decreased ciprofloxacin accumulation in the wild-type E. coli strain as BW25113 cells accumulated lower levels of ciprofloxacin with GSH after 5, 10, or 15 min of antibiotic exposure (Fig. 3A) in comparison to its complete absence at 0 min (data not shown). Similarly, the presence of GSH resulted in significantly reduced (P < 0.05) accumulation of ciprofloxacin in JW0451, JW0452, and JW5503 at one or more time points (Fig. 3B, C, and D, respectively). The GSH-mediated decline in the ciprofloxacin accumulation within BW25113, JW0452, JW0451, and JW5503 is another reflection of their increased efflux activity. Importantly, GSH cannot affect the ciprofloxacin uptake directly in E. coli as their routes of entry are separate. GSH enters through the yliABCD transporter (23), whereas ciprofloxacin gains entry through both porin- and nonporin-mediated pathways in E. coli (24). The present study therefore clearly establishes that the presence of GSH not only results in increased bacterial efflux activity but also causes decreased ciprofloxacin accumulation in E. coli. Nevertheless, it is important to remember that GSH-mediated reversal of ciprofloxacin's activity is not merely due to increased efflux, as GSH has already been reported to neutralize the antibiotic-induced oxidative stress to decrease the antibacterial action of ciprofloxacin (8). Therefore, on the basis of our previous (8) and present findings we propose that exogenous GSH reverses the effect of ciprofloxacin both by (i) neutralizing the oxidative stress involved in its antibacterial action (8) and (ii) increasing the ciprofloxacin efflux out of E. coli (present study). However, it is difficult to quantify their absolute contributions toward this phenotype due to an ongoing cross talk between multiple stress response pathways in E. coli (5, 25).
Taken together, the results of our present study reveal for the first time that GSH supplementation could promote ciprofloxacin resistance by increasing its efflux from E. coli. Since GSH can have a direct or indirect effect on the proton motive force in E. coli (26), it might also influence bacterial efflux activity; however, this proposition needs to be verified experimentally through a separate set of studies. Nevertheless, GSH is a prominent cellular antioxidant in both bacteria and animals, and its increased levels could have multifaceted implications on the clearance of bacterial infection by (i) differentially modulating the activity of antibiotic classes/subclasses (27, 28), (ii) decreasing the antibiotic-induced oxidative stress (8), (iii) promoting antibiotic efflux (present study), and (iv) delaying the natural clearance of bacterial pathogens (29). Consequently, further studies are needed in this direction to understand how alterations in intracellular thiol(s) or the overall cellular redox state can be used for better management of bacterial infection and antibiotic therapy.
Supplementary Material
ACKNOWLEDGMENTS
We are thankful to Irfan Shaikh for his technical assistance during the microbiological experiments.
We declare no conflicts of interest.
Funding Statement
This work was supported by internal departmental funds of the Department of Atomic Energy (DAE), Government of India. Part of the gene expression experiments was financed by the Knowledge Foundation, Sweden.
Footnotes
Supplemental material for this article may be found at http://dx.doi.org/10.1128/AAC.00414-16.
REFERENCES
- 1.Szmolka A, Nagy B. 2013. Multidrug resistant commensal Escherichia coli in animals and its impact for public health. Front Microbiol 4:258. doi: 10.3389/fmicb.2013.00258. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Li XZ, Nikaido H. 2004. Efflux-mediated drug resistance in bacteria. Drugs 64:159–204. doi: 10.2165/00003495-200464020-00004. [DOI] [PubMed] [Google Scholar]
- 3.Piddock LJ. 2006. Multidrug-resistance efflux pumps—not just for resistance. Nat Rev Microbiol 4:629–636. doi: 10.1038/nrmicro1464. [DOI] [PubMed] [Google Scholar]
- 4.Webber MA, Piddock LJ. 2003. The importance of efflux pumps in bacterial antibiotic resistance. J Antimicrob Chemother 51:9–11. doi: 10.1093/jac/dkg050. [DOI] [PubMed] [Google Scholar]
- 5.Zgurskaya HI, Krishnamoorthy G, Ntreh A, Lu S. 2011. Mechanism and function of the outer membrane channel TolC in multidrug resistance and physiology of enterobacteria. Front Microbiol 2:189. doi: 10.3389/fmicb.2011.00189. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Paulsen IT, Park JH, Choi PS, Saier MH Jr. 1997. A family of Gram-negative bacterial outer membrane factors that function in the export of proteins, carbohydrates, drugs and heavy metals from Gram-negative bacteria. FEMS Microbiol Lett 156:1–8. doi: 10.1016/S0378-1097(97)00379-0. [DOI] [PubMed] [Google Scholar]
- 7.Oliphant CM, Green GM. 2002. Quinolones: a comprehensive review. Am Fam Physician 65:455–464. [PubMed] [Google Scholar]
- 8.Goswami M, Mangoli SH, Jawali N. 2006. Involvement of reactive oxygen species in the action of ciprofloxacin against Escherichia coli. Antimicrob Agents Chemother 50:949–954. doi: 10.1128/AAC.50.3.949-954.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Becerra MC, Paez PL, Larovere LE, Albesa I. 2006. Lipids and DNA oxidation in Staphylococcus aureus as a consequence of oxidative stress generated by ciprofloxacin. Mol Cell Biochem 285:29–34. doi: 10.1007/s11010-005-9051-0. [DOI] [PubMed] [Google Scholar]
- 10.Kohanski MA, Dwyer DJ, Hayete B, Lawrence CA, Collins JJ. 2007. A common mechanism of cellular death induced by bactericidal antibiotics. Cell 130:797–810. doi: 10.1016/j.cell.2007.06.049. [DOI] [PubMed] [Google Scholar]
- 11.Dwyer DJ, Kohanski MA, Collins JJ. 2009. Role of reactive oxygen species in antibiotic action and resistance. Curr Opin Microbiol 12:482–489. doi: 10.1016/j.mib.2009.06.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Goswami M, Jawali N. 2010. N-Acetylcysteine-mediated modulation of bacterial antibiotic susceptibility. Antimicrob Agents Chemother 54:3529–3530. doi: 10.1128/AAC.00710-10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Dhamdhere G, Krishnamoorthy G, Zgurskaya HI. 2010. Interplay between drug efflux and antioxidants in Escherichia coli resistance to antibiotics. Antimicrob Agents Chemother 54:5366–5368. doi: 10.1128/AAC.00719-10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Baba T, Ara T, Hasegawa M, Takai Y, Okumura Y, Baba M, Datsenko KA, Tomita M, Wanner BL, Mori H. 2006. Construction of Escherichia coli K-12 in-frame, single-gene knockout mutants: the Keio collection. Mol Syst Biol 2:2006 0008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Tamae C, Liu A, Kim K, Sitz D, Hong J, Becket E, Bui A, Solaimani P, Tran KP, Yang H, Miller JH. 2008. Determination of antibiotic hypersensitivity among 4,000 single-gene-knockout mutants of Escherichia coli. J Bacteriol 190:5981–5988. doi: 10.1128/JB.01982-07. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Sulavik MC, Houseweart C, Cramer C, Jiwani N, Murgolo N, Greene J, DiDomenico B, Shaw KJ, Miller GH, Hare R, Shimer G. 2001. Antibiotic susceptibility profiles of Escherichia coli strains lacking multidrug efflux pump genes. Antimicrob Agents Chemother 45:1126–1136. doi: 10.1128/AAC.45.4.1126-1136.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Dhamdhere G, Zgurskaya HI. 2010. Metabolic shutdown in Escherichia coli cells lacking the outer membrane channel TolC. Mol Microbiol 77:743–754. doi: 10.1111/j.1365-2958.2010.07245.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Kumar R, Pradhan A, Khan FA, Lindstrom P, Ragnvaldsson D, Ivarsson P, Olsson PE, Jass J. 2015. Comparative analysis of stress induced gene expression in Caenorhabditis elegans following exposure to environmental and lab reconstituted complex metal mixture. PLoS One 10:e0132896. doi: 10.1371/journal.pone.0132896. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Nishino K, Nikaido E, Yamaguchi A. 2009. Regulation and physiological function of multidrug efflux pumps in Escherichia coli and Salmonella. Biochim Biophys Acta 1794:834–843. doi: 10.1016/j.bbapap.2009.02.002. [DOI] [PubMed] [Google Scholar]
- 20.Webber MA, Coldham NG. 2010. Measuring the activity of active efflux in Gram-negative bacteria. Methods Mol Biol 642:173–180. doi: 10.1007/978-1-60327-279-7_13. [DOI] [PubMed] [Google Scholar]
- 21.Ferguson GP, Munro AW, Douglas RM, McLaggan D, Booth IR. 1993. Activation of potassium channels during metabolite detoxification in Escherichia coli. Mol Microbiol 9:1297–1303. doi: 10.1111/j.1365-2958.1993.tb01259.x. [DOI] [PubMed] [Google Scholar]
- 22.Piddock LJ, Jin YF, Ricci V, Asuquo AE. 1999. Quinolone accumulation by Pseudomonas aeruginosa, Staphylococcus aureus and Escherichia coli. J Antimicrob Chemother 43:61–70. doi: 10.1093/jac/43.1.61. [DOI] [PubMed] [Google Scholar]
- 23.Suzuki H, Koyanagi T, Izuka S, Onishi A, Kumagai H. 2005. The yliA, -B, -C, and -D genes of Escherichia coli K-12 encode a novel glutathione importer with an ATP-binding cassette. J Bacteriol 187:5861–5867. doi: 10.1128/JB.187.17.5861-5867.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Diver JM. 1989. Quinolone uptake by bacteria and bacterial killing. Rev Infect Dis 11(Suppl 5):S941–S946. doi: 10.1093/clinids/11.Supplement_5.S941. [DOI] [PubMed] [Google Scholar]
- 25.Poole K. 2012. Bacterial stress responses as determinants of antimicrobial resistance. J Antimicrob Chemother 67:2069–2089. doi: 10.1093/jac/dks196. [DOI] [PubMed] [Google Scholar]
- 26.Riondet C, Cachon R, Wache Y, Alcaraz G, Divies C. 1999. Changes in the proton-motive force in Escherichia coli in response to external oxidoreduction potential. Eur J Biochem 262:595–599. doi: 10.1046/j.1432-1327.1999.00429.x. [DOI] [PubMed] [Google Scholar]
- 27.Goswami M, Mangoli S, Jawali N. 2011. Antibiotics and antioxidants: friends or foes during therapy? BARC Newsl 323:42–46. http://barc.gov.in/publications/nl/2011/2011111208.pdf. [Google Scholar]
- 28.Goswami M, Mangoli S, Jawali N. 2014. Importance of chemical modification at C-7 position of quinolones for glutathione-mediated reversal of antibacterial activity. Int J Antimicrob Agents 43:387−388. doi: 10.1016/j.ijantimicag.2014.01.004. [DOI] [PubMed] [Google Scholar]
- 29.Goswami M, Sharma D, Khan NM, Checker R, Sandur SK, Jawali N. 2014. Antioxidant supplementation enhances bacterial peritonitis in mice by inhibiting phagocytosis. J Med Microbiol 63:355–366. doi: 10.1099/jmm.0.067173-0. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.