Abstract
Infections caused by multidrug-resistant (MDR) bacteria present an emerging global health crisis, and the threat is intensified by the involvement of biofilms. Some biofilm infections involve more than one species; this can further challenge treatment using traditional antibiotics. Nanomaterials are being developed as alternative therapeutics to traditional antibiotics; here we report biodegradable polymer-stabilized oil-in-water nanosponges (BNS) and show their activity against dual-species bacterial biofilms. The described engineered nanosponges demonstrated broad-spectrum antimicrobial activity through prevention of dual-species biofilm formation as well as eradication of preformed biofilms. The BNS showed no toxicity against mammalian cells. Together, this data highlights the therapeutic potential of this platform.
Keywords: dual-species biofilms, multidrug-resistance, antimicrobials, nanosponges
Graphical Abstract
The escalating threat of multidrug-resistant (MDR) bacteria has resulted in untreatable infections that could kill millions of patients if appropriate measures are not taken.1,2 The ability of MDR bacteria to form biofilms further complicates treatment.3 , 4 , 5 Bacterial biofilm infections frequently occur on wounds,6,7 ,8 medical implants,9 , 10 and indwelling devices.11 These infections are difficult to treat due to the complexity of the involved physiologic state alongside the presence on the biofilm matrix that together protect bacteria against antimicrobial agents and host immune response; further, biofilms provide a site for selection of antibiotic resistance.12,13,14 Some clinically relevant biofilms, such as wound infections, typically involve two or more bacterial species,15,16 adding an additional therapeutic challenge compared to the monospecies biofilms typically involved with infected indwelling devices.17
Multispecies biofilm infections can be a therapeutic challenge because different species can exhibit distinct antibiotic susceptibilities.18 While the exact details of bacterial interactions in polymicrobial biofilms remain to be defined, inter-bacterial interactions, including metabolic connections and spatial organization, are involved.18 Synergistic interactions between some species, can occur, and there can be transfer of resistance genes and quorum sensing, which can lead to more resistant biofilms.19 The composition of the extracellular polymeric substances (EPS) matrix may be different in mono-compared to multi-species biofilms, and this can provide a stronger barrier against antibiotic penetration.18
Current strategies to combat multispecies biofilms focus on antibiotic cocktails or use of antibiotics with broad spectrum activity, able to provide activity against more than one species.17 These therapeutic strategies can be associated with increased incidence of antibiotic resistance development and off-target effects, such as hepatotoxicity or nephrotoxicity.17 These challenges are amplified by the decline in the number of new antibiotics entering the clinic, contributing to the urgency for developing novel antimicrobial therapies.20
Nanomaterials present a tool box for combating biofilm infections.21 The unique physicochemical properties of nanomaterials, such as shape, size and surface functionalities, allow them to access multiple antimicrobial modalities capable of overcoming the challenge of multi-species biofilm infections.22 As compared to small molecule antibiotics, nanomaterials have sizes and shapes similar to bacterial biomolecules, enabling multivalent interactions with bacteria.22 Further, their high surface-to-volume ratios allow high therapeutic loading, rendering them efficient delivery vehicles for antimicrobial agents. While antibiotics have specific bacterial targets and are prone to resistance development, nanomaterials exhibit several antimicrobial mechanisms novel to bacteria and hence less susceptible to resistance acquisition.23
Nanomaterials incorporating plant-derived essential oils have emerged as a promising resource to combat bacterial biofilms infections. Essential oils exhibit broad-spectrum activity against MDR bacteria.24 However, low aqueous solubility and stability limit their widespread application. Polymeric nanomaterials have shown efficient delivery of essential oils to bacteria enclosed within an EPS matrix.25,26,27 Previously, our group reported the use of biodegradable polymers to create nanosponges that act as nanocarriers of essential oils.28 , 29 The polymer scaffold was designed with disulfide and ester cross-linkers that impart aqueous stability while being biodegradable in the presence of endogenous biomolecules, including glutathione and esterase (Scheme 1b). The engineered nanosponges have a sponge-like composite morphology as shown by confocal microscopy images.28
Scheme 1.
Fabrication of biodegradable nanosponges. a) Schematic representation of the strategy used to fabricate carvacrol-loaded BNS. The resulting nanosponges demonstrate enhanced antimicrobial activity against dual-species biofilms. b) Chemical structures of PONI-GMT, DTDS (Disulfanediyldi(ethane-2,1-diyl) bis(11-sulfanylundecanoate)), and crosslinked structure of BNS, with degradation points highlighted: degradation of disulfide bond (cyan), degradation by glutathione (blue) and esterase (orange).
We report here the activity of oil-in-water polymer nanosponges against dual-species biofilm models of clinical isolates. Multiple biofilm combinations were tested, including methicillin-resistant Staphylococcus aureus (MRSA)-Pseudomonas aeruginosa and MRSA-Acinetobacter baumannii, common bacterial species combinations found in wound biofilm infections.30,31 Confocal laser scanning microscopy studies revealed that nanosponges effectively penetrated biofilms, interacting with both species of bacteria embedded within the EPS. The nanosponges eliminated mono- and dual-species bacterial biofilms, while showing minimal toxicity to mammalian fibroblast cells. Overall, we demonstrated the potent, broad-spectrum anti-biofilm activity of nanosponges against complex dual-species bacterial biofilms, offering a promising strategy for complex wound biofilm treatment.
Biodegradable nanosponges (BNS) were fabricated as previously described28 through emulsification of synthetically engineered poly(oxanorborneneimide) polymers and carvacrol, the major antimicrobial component of oregano oil.32 The PONI-GMT (poly(oxanorborneneimide)-backbone, guanidinium, maleimide, and tetraethylene glycol monomethyl ether moieties) scaffold possesses three structural features: 1) a cationic guanidinium moiety to enhance interaction with negatively charged bacterial membranes and EPS matrix;33 2) a tetraethylene glycol monomethyl ether moiety to enhance amphiphilicity, allowing polymers to self-assemble around the oil core; and 3) a maleimide moiety to conjugate with disulfide containing dithiol-disulfide (Disulfanediyldi(ethane-2,1-diyl) bis(11-sulfanylundecanoate)) (DTDS), loaded inside carvacrol, via maleimide-Michael addition reactions. The resulting cross-linked structure allows the formation of stable nanosponges and imparts degradability to the nanosponges at five points (Scheme 1). The nanosponge diameter was ~220 nm, as shown by dynamic light scattering (DLS) (Figure S1). The generation of cationic nanosponges, attributed to guanidine units, was validated by zeta (ζ) potential measurement, averaging at +24 mV (Figure S2).
We studied the anti-biofilm activity of BNS against multiple mono- and dual-species bacterial biofilms using clinical isolates of pathogenic Gram-positive and Gram-negative bacteria. We focused our antimicrobial studies on two dual-species biofilms: 1) MRSA (Gram-positive) and P. aeruginosa (Gram-negative); and 2) MRSA and A. baumannii (Gram-negative). MRSA and P. aeruginosa are frequent causes of wound infections, often together, as mixed-species biofilms.34 , 35 Dual-species biofilms formed by these two pathogens are more resistant to conventional antimicrobial therapy than monomicrobial biofilms, often leading to failure of treatment.36 Also, A. baumannii is associated with severe wound infections and often generate highly resistant mixed-species biofilms with other bacterial species.31 The ability of BNS to prevent formation of dual-species biofilms was first investigated. BNS effectively prevented formation of either mono- or dual-species biofilms (Figure 3c–d).
Figure 3.
Viability (a-b) and biomass (c-d) of 2-day old mono- and dual-species biofilms: MRSA-P. aeruginosa (a & c) and MRSA-A. baumannii (b&d) after 3 h treatment with nanosponges. The data shown are average of triplicates, and the error bars indicate the standard deviation. * p < 0.05, ** p < 0.01, and *** p < 0.001 compared to control (0 mM), as calculated using one-way ANOVA analysis
Effective biofilm eradication requires penetration and accumulation of therapeutics into biofilms,37 and hence we investigated the ability of BNS to penetrate EPS matrix of both mono- and dual-species biofilm using confocal microscopy. We treated dual-species biofilm of red fluorescent protein DsRed-expressing Escherichia coli and green fluorescent protein (GFP)-expressing MRSA with nanosponges loaded with a blue fluorescent dye (Pacific Blue) inside the oil core. We chose these two fluorescent protein-expressing bacterial strains due to their strong fluorescence signals. BNS could readily penetrate the biofilm matrix and colocalize within the bacteria for both mono-species (Figure S3–S4) and dual-species biofilms (Figure 1), highlighting their potential to be effective anti-biofilm agents.
Figure 1.
Representative 3D views of confocal image stacks of DsRed-expressing Escherichia coli and GFP-expressing MRSA dual-species biofilms, Pacific Blue-loaded BNS and their overlay after treating the biofilms for 1 h with Pacific Blue-loaded BNS at 240 mg/L of carvacrol oil in M9 media (untreated biofilm served as the negative control were prepared similarly without BNS treatment).
Having established good biofilm penetration, the efficacy of BNS against different mono- and dual-species bacterial biofilms was evaluated next. BNS demonstrated minimum biofilm bactericidal concentration (MBBC) values ranging from 120 to 240 mg/L, indicating their broad-spectrum activity (Figure 2a). Furthermore, we compared the MBBC values of BNS between mono- and dual-species biofilms of MRSA IDRL-6169 and P. aeruginosa CD-1006, while also assessing gentamicin, a broad-spectrum aminoglycoside antibiotic. As shown in Figure 2b, there was no change in MBBC observed for BNS (240 mg/L) when treating dual-species compared to mono-species biofilms, demonstrating effective treatment of the dual-species biofilms. In contrast, a 4-fold increase in MBBC was detected for gentamicin when treating the dual-species biofilms (1024 mg/L) as compared to P. aeruginosa only (256 mg/L), and a 128-fold increase when compared to MRSA only (8 mg/L). The increased tolerance to gentamicin underlines the challenge for traditional antibiotics posed by multispecies biofilms resulting from cooperative interactions between species within biofilms.19 In contrast, the inherent broad-spectrum nature of the BNS made this platform effective against dual-species biofilms, providing a promising strategy to combat real-world infections.
Figure 2.
a) MBBC values of nanosponges against multiple mono- and dual-species bacterial biofilms. b) MBBC values of nanosponges and gentamicin against MRSA and Pseudomonas aeruginosa mono- and dual-species biofilms.
Next, the activity of our nanosponges against preformed 2-day-old dual-species biofilms was evaluated. Established biofilms of MRSA-P. aeruginosa and MRSA-A. baumannii were treated with different concentrations of BNS for 3 h. Subsequently, biofilm viability was determined using Alamar Blue assay. As shown in Figure 3, BNS eradicated bacteria within the biofilms in a dose-dependent manner. Notably, BNS maintained activity in either mono- or dual-species settings.
Finally, we investigated BNS biocompatibility with mammalian NIH 3T3 fibroblasts as such cells would be expected to be present at the site of infected wounds. Fibroblasts play a critical role in wound healing by creating new extracellular matrix (ECM).38 Fibroblasts were treated with BNS for 3 h using the same concentrations active against biofilms. As shown in Figure 4, BNS demonstrated low toxicity towards fibroblast cells at therapeutically relevant concentrations. Given that BNS effectively eliminate dual-species pathogenic biofilms while having minimum effect on mammalian fibroblasts, these results suggest that BNS should be further evaluated as a potential approach to treat wound biofilm infections.
Figure 4.
Viability of 3T3 fibroblast cells after 3 h treatment with nanosponges. The data shown are average of triplicates, and the error bars indicate the standard deviation.
In summary, we have investigated the antimicrobial activity of biodegradable polymer-stabilized oil-in-water nanosponges against dual-species bacterial biofilms. The nanosponges demonstrated broad-spectrum antimicrobial activity against pathogenic mono- and dual-species biofilms, either through prevention of biofilm formation or eradication of preformed biofilms, with no observed toxicity to mammalian cells. In contrast to antibiotic treatment where dual-species biofilms showed increased tolerance against therapeutics, our nanosponges showed no diminished activity in dual-species biofilm settings. Taken together, this nanosponge platform provides a promising strategy to combat difficult-to-treat mixed-species bacterial biofilms with the potential to revolutionize the treatment of bacterial infections.
Supplementary Material
ACKNOWLEDGMENT
Clinical samples obtained from the Cooley Dickinson Hospital Microbiology Laboratory (Northampton, MA) were kindly provided by Dr. Margaret Riley. The microscopy data was gathered in the Light Microscopy Facility and Nikon Center of Excellence at the Institute for Applied Life Sciences, UMass Amherst with support from the Massachusetts Life Sciences Center.
Funding Sources
This research was funded by the NIH (R01 AI134770).
Footnotes
The authors declare no competing financial interest.
Supporting Information
Experimental procedures, size and zeta potential of BNS, mono-species biofilm penetration profile of BNS.
Experimental Section
Detailed experimental processes are provided in the Supporting Information.
REFERENCES
- 1.Nikaido H Multidrug Resistance in Bacteria. Annu. Rev. Biochem 2009. 10.1146/annurev.biochem.78.082907.145923. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.O’Neill J Antimicrobial Resistance : Tackling a Crisis for the Health and Wealth of Nations. Rev. Antimicrob. Resist 2016. [Google Scholar]
- 3.Flemming H; Wingender J The Biofilm Matrix. Nat. Publ. Gr 2010, 8 (9), 623–633. 10.1038/nrmicro2415. [DOI] [PubMed] [Google Scholar]
- 4.Hall-Stoodley L; Costerton JW; Stoodley P Bacterial Biofilms: From the Natural Environment to Infectious Diseases. Nat. Rev. Microbiol 2004, 2 (2), 95–108. 10.1038/nrmicro821. [DOI] [PubMed] [Google Scholar]
- 5.lemming H-C; Wingender J; Szewzyk U; Steinberg P; Rice SA; Kjelleberg S Biofilms: An Emergent Form of Bacterial Life. Nat. Rev. Microbiol 2016, 14 (9), 563–575. 10.1038/nrmicro.2016.94. [DOI] [PubMed] [Google Scholar]
- 6.Zhao G; Usui ML; Lippman SI; James GA; Stewart PS; Fleckman P; Olerud JE Biofilms and Inflammation in Chronic Wounds. Adv. wound care 2013, 2 (7), 389–399. 10.1089/wound.2012.0381. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Wolcott RD; Rhoads DD; Bennett ME; Wolcott BM; Gogokhia L; Costerton JW; Dowd SE Chronic Wounds and the Medical Biofilm Paradigm. J. Wound Care 2010. 10.12968/jowc.2010.19.2.46966. [DOI] [PubMed] [Google Scholar]
- 8.Wu YK; Cheng NC; Cheng CM Biofilms in Chronic Wounds: Pathogenesis and Diagnosis. Trends Biotechnol. 2018, 1–13. 10.1016/j.tibtech.2018.10.011. [DOI] [PubMed] [Google Scholar]
- 9.Veerachamy S; Yarlagadda T; Manivasagam G; Yarlagadda PK Bacterial Adherence and Biofilm Formation on Medical Implants: A Review. Proceedings of the Institution of Mechanical Engineers, Part H: Journal of Engineering in Medicine. 2014. 10.1177/0954411914556137. [DOI] [PubMed] [Google Scholar]
- 10.Arciola CR; Campoccia D Implant Infections: Adhesion, Biofilm Formation and Immune Evasion. Nat. Rev. Microbiol 2018, 16 (July). 10.1038/s41579-018-0019-y. [DOI] [PubMed] [Google Scholar]
- 11.Bryers JD Medial Biofilms. Biotechnol Bioeng. 2008 May 1; 100(1) 1–18. doi10.1002/bit.21838 2009, 100 (1), 1–18. 10.1002/bit.21838.Medical. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Van Acker H; Van Dijck P; Coenye T Molecular Mechanisms of Antimicrobial Tolerance and Resistance in Bacterial and Fungal Biofilms. Trends in Microbiology. 2014. 10.1016/j.tim.2014.02.001. [DOI] [PubMed] [Google Scholar]
- 13.Nadell CD; Drescher K; Wingreen NS; Bassler BL Extracellular Matrix Structure Governs Invasion Resistance in Bacterial Biofilms. ISME J. 2015. 10.1038/ismej.2014.246. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Donlan RM; Costerton JW Biofilms: Survival Mechanisms of Clinically Relevant Microorganisms. Clinical Microbiology Reviews. 2002. 10.1128/CMR.15.2.167-193.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Wolcott R; Costerton JW; Raoult D; Cutler SJ The Polymicrobial Nature of Biofilm Infection. Clinical Microbiology and Infection. 2013. 10.1111/j.1469-0691.2012.04001.x. [DOI] [PubMed] [Google Scholar]
- 16.Yang L; Liu Y; Wu H; Høiby N; Molin S; Song ZJ Current Understanding of Multispecies Biofilms. In International Journal of Oral Science; 2011. 10.4248/IJOS11027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Burmølle M; Webb JS; Rao D; Hansen LH; Sørensen SJ; Kjelleberg S Enhanced Biofilm Formation and Increased Resistance to Antimicrobial Agents and Bacterial Invasion Are Caused by Synergistic Interactions in Multispecies Biofilms. Appl. Environ. Microbiol 2006. 10.1128/AEM.03022-05. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Rao Y; Shang W; Yang Y; Zhou R; Rao X Fighting Mixed-Species Microbial Biofilms With Cold Atmospheric Plasma. Frontiers in Microbiology. 2020. 10.3389/fmicb.2020.01000. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Burmølle M; Ren D; Bjarnsholt T; Sørensen SJ Interactions in Multispecies Biofilms: Do They Actually Matter? Trends in Microbiology. 2014. 10.1016/j.tim.2013.12.004. [DOI] [PubMed] [Google Scholar]
- 20.Jagusztyn-Krynicka EK; Wyszyńska A The Decline of Antibiotic Era - New Approaches for Antibacterial Drug Discovery. Polish Journal of Microbiology. 2008. [PubMed] [Google Scholar]
- 21.Gupta A; Mumtaz S; Li CH; Hussain I; Rotello VM Combatting Antibiotic-Resistant Bacteria Using Nanomaterials. Chem. Soc. Rev 2019, 48 (2), 415–427. 10.1039/c7cs00748e. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Makabenta JMV; Nabawy A; Li C-H; Schmidt-Malan S; Patel R; Rotello VM Nanomaterial-Based Therapeutics for Antibiotic-Resistant Bacterial Infections. Nat. Rev. Microbiol 2020. 10.1038/s41579-020-0420-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Pelgrift RY; Friedman AJ Nanotechnology as a Therapeutic Tool to Combat Microbial Resistance. Adv. Drug Deliv. Rev 2013, 65 (13–14), 1803–1815. 10.1016/j.addr.2013.07.011. [DOI] [PubMed] [Google Scholar]
- 24.Kalemba D; Kunicka A Antibacterial and Antifungal Properties of Essential Oils. Curr. Med. Chem 2005. 10.2174/0929867033457719. [DOI] [PubMed] [Google Scholar]
- 25.Duncan B; Li X; Landis RF; Kim ST; Gupta A; Wang LS; Ramanathan R; Tang R; Boerth JA; Rotello VM Nanoparticle-Stabilized Capsules for the Treatment of Bacterial Biofilms. ACS Nano 2015. 10.1021/acsnano.5b01696. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Zhu DY; Landis RF; Li C-H; Gupta A; Wang L-S; Geng Y; Gopalakrishnan S; Guo JW; Rotello VM Dynamically Crosslinked Polymer Nanocomposites to Treat Multidrug-Resistant Bacterial Biofilms. Nanoscale 2018, 10 (39), 18651–18656. 10.1039/C8NR06348F. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Landis RF; Gupta A; Lee Y; Wang L; Golba B; Couillaud B; Ridolfo R; Das R; Rotello VM Cross-Linked Polymer-Stabilized Nanocomposites for the Treatment of Bacterial Bio Fi Lms. 2017. 10.1021/acsnano.6b07537. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Landis RF; Li CH; Gupta A; Lee YW; Yazdani M; Ngernyuang N; Altinbasak I; Mansoor S; Khichi MAS; Sanyal A; Rotello VM Biodegradable Nanocomposite Antimicrobials for the Eradication of Multidrug-Resistant Bacterial Biofilms without Accumulated Resistance. J. Am. Chem. Soc 2018. 10.1021/jacs.8b03575. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Li C; Chen X; Landis RF; Geng Y; Makabenta JM; Lemnios W; Gupta A; Rotello VM Phytochemical-Based Nanocomposites for the Treatment of Bacterial Biofilms. 2019. 10.1021/acsinfecdis.9b00134. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Pastar I; Nusbaum AG; Gil J; Patel SB; Chen J; Valdes J; Stojadinovic O; Plano LR; Tomic-Canic M; Davis SC Interactions of Methicillin Resistant Staphylococcus aureus USA300 and Pseudomonas aeruginosa in Polymicrobial Wound Infection. PLoS One 2013. 10.1371/journal.pone.0056846. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Liao YT; Kuo SC; Lee YT; Chen CP; Lin SW; Shen LJ; Fung CP; Cho WL; Chen TL Sheltering Effect and Indirect Pathogenesis of Carbapenem-Resistant Acinetobacter baumannii in Polymicrobial Infection. Antimicrob. Agents Chemother 2014. 10.1128/AAC.02636-13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Lambert RJW; Skandamis PN; Coote PJ; Nychas GJE A Study of the Minimum Inhibitory Concentration and Mode of Action of Oregano Essential Oil, Thymol and Carvacrol. J. Appl. Microbiol 2001. 10.1046/j.13652672.2001.01428.x. [DOI] [PubMed] [Google Scholar]
- 33.Li X; Yeh YC; Giri K; Mout R; Landis RF; Prakash YS; Rotello VM Control of Nanoparticle Penetration into Biofilms through Surface Design. Chem. Commun 2015. 10.1039/c4cc07737g. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Serra R; Grande R; Butrico L; Rossi A; Settimio UF; Caroleo B; Amato B; Gallelli L; De Franciscis S Chronic Wound Infections: The Role of Pseudomonas aeruginosa and Staphylococcus aureus. Expert Review of Anti-Infective Therapy. 2015. 10.1586/14787210.2015.1023291. [DOI] [PubMed] [Google Scholar]
- 35.DeLeon S; Clinton A; Fowler H; Everett J; Horswill AR; Rumbaugh KP Synergistic Interactions of Pseudomonas aeruginosa and Staphylococcus aureus in an In Vitro Wound Model. Infect. Immun 2014. 10.1128/IAI.02198-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Beaudoin T; Yau YCW; Stapleton PJ; Gong Y; Wang PW; Guttman DS; Waters V Staphylococcus aureus Interaction with Pseudomonas aeruginosa Biofilm Enhances Tobramycin Resistance. npj Biofilms Microbiomes 2017. 10.1038/s41522-017-0035-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Wu H; Moser C; Wang H-Z; Høiby N; Song Z-J Strategies for Combating Bacterial Biofilm Infections. Int. J. Oral Sci 2015, 7 (1), 1–7. 10.1038/ijos.2014.65. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Tracy LE; Minasian RA; Caterson EJ Extracellular Matrix and Dermal Fibroblast Function in the Healing Wound. Advances in Wound Care. 2016. 10.1089/wound.2014.0561. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.