Abstract
Background
Prior studies of pediatric musculoskeletal infection have suggested that methicillin-resistant Staphylococcus aureus (MRSA) infections result in worse outcomes compared with infections with methicillin-susceptible S aureus (MSSA) strains. Based on these results, clinical prediction algorithms have been developed to differentiate between MRSA and MSSA early in a patient’s clinical course. This study compares hospital outcomes for pediatric patients with MRSA and MSSA musculoskeletal infection presenting to the emergency department at a large tertiary care children’s hospital.
Methods
A retrospective study identified pediatric patients with S aureus musculoskeletal infection over a 5-year period (2008–2013) by sequential review of all pediatric orthopedic consults. Relevant demographic information, laboratory values, and clinical outcomes were obtained from the electronic medical record.
Results
Of the 91 identified cases of S aureus pediatric musculoskeletal infection, there were 49 cases of MRSA infection (53%) and 42 cases of MSSA infection (47%). There were no significant differences between MRSA and MSSA infections in median hospital length of stay (4.8 vs 5.7 days, P = .50), febrile days (0.0 vs 1.5 days, P = .10), and antibiotic duration (28 vs 34 days, P = .18). Methicillin-resistant S aureus infections were more likely to require operative intervention than MSSA infection (85% vs 62%, P = .15). A logistic regression model based on C-reactive protein, temperature, white blood cell count, pulse, and respiratory rate at presentation demonstrated poor ability to differentiate between MRSA and MSSA infection.
Conclusions
The results demonstrated no significant differences between MSSA and MRSA musculoskeletal infections for most hospital outcomes measured. However, MRSA infections required more operative interventions than MSSA infections. In addition, a predictive model based on severity markers obtained at presentation was unable to effectively differentiate between MRSA and MSSA infection. The clinical utility and capacity for early differentiation of MRSA and MSSA depends on virulence patterns that may vary temporally and geographically.
Keywords: methicillin-resistant Staphyloccus aureus (MRSA), osteomyelitis, pediatric, septic arthritis, Staphylococcus aureus
Staphylococcus aureus is the most common cause of pediatric musculoskeletal infection (MSKI), accounting for 50%–70% of culture-confirmed infections [1–4]. Although methicillin-susceptible S aureus (MSSA) has historically been the predominant cause of community-acquired S aureus infections, there was a significant increase in the prevalence of methicillin-resistant S aureus (MRSA) between 2000 and 2010 [2, 5, 6]. Recent reports have shown MRSA to be the causative pathogen in 30%–50% of cases of pediatric osteomyelitis, septic arthritis, and pyomyositis [1, 2, 7].
The relative severity of infections caused by MRSA versus MSSA in children remains controversial. Several studies on pediatric MSKI have reported increased markers of inflammation, more surgical interventions, and prolonged hospital lengths of stay for MRSA compared with MSSA [1, 5, 8, 9]. However, in the context of S aureus bacteremia and necrotizing pneumonia, other studies have reported no association between methicillin resistance and infection severity or subsequent complications [10, 11]. Regardless, the evidence of increased virulence of MRSA compared with MSSA has prompted the development of algorithms aimed at early identification of methicillin-resistant strains [12, 13], which can have significant implications on management and therapy.
Regional variability in S aureus patterns of methicillin resistance and relative virulence are well documented in both North America and Europe [14, 15]. Moreover, there is increasing evidence that the relative proportion of MSSA strains is rising for the first time in several decades [16]. The aim of this study was to determine the prevalence and severity of MRSA and MSSA pediatric MSKI at a large tertiary care children’s hospital in the United States. Based on current clinical experience, the authors hypothesized that there would be no significant differences in in the severity of infections caused by MRSA and MSSA.
MATERIALS AND METHODS
An institutional review board-approved retrospective review was conducted to identify patients aged 0–18 who presented to the pediatric emergency room at a tertiary care children’s hospital with concern for MSKI over a 5-year period (2008–2013). Two hundred seventy-three patients were initially identified through review of the pediatric orthopedic consult list from both the emergency department and the inpatient ward. Patients with a positive S aureus blood or tissue culture were included in the study. The cases were confirmed as MSKI with either intraoperative culture or magnetic resonance imaging. Patients with incidental community-acquired S aureus MSKIs were included in the study. Patients with a diagnosis of posttraumatic infection, postoperative infection, chronic osteomyelitis, or cellulitis were excluded to reduce confounding variables in the analysis.
Demographic information, laboratory values, and relevant clinical information and outcomes were obtained from the electronic medical record. Laboratory values recorded included C-reactive protein (CRP), white blood cell (WBC) count, temperature, and erythrocyte sedimentation rate (ESR). Clinical outcomes included hospital length of stay (LOS), number of operative interventions, duration of antibiotics, and intensive care unit LOS. Furthermore, infection severity and degree of dissemination were determined for each infection as defined by the operational definitions in Supplementary Table 1 [17].
Statistical Methods
Data analysis was performed using the statistical analysis tool GraphPad Prism 6 (GraphPad Software Inc., La Jolla, CA). Comparison analysis was performed using Fisher’s exact test for dichotomous variables and Mann-Whitney U test for continuous variables to account for non-parametric data distribution. P values less than or equal to .05 were considered statistically significant. Logistic regression and Harrell’s C statistic calculations for predictive model differentiating MRSA and MSSA were performed using STATA14 (StataCorp, College Station, TX).
RESULTS
Demographic and presentation data for the patients included in the study are displayed in Table 1. Overall, there were 91 cases of MSKI included in the study: 49 were caused by MRSA strains and 42 were caused by MSSA strains. There were no significant differences between patients with MRSA and MSSA MSKI with regard to sex, age, or weight bearing status at presentation (Table 1). The clinical syndromes (eg, pyomyositis, osteomyelitis, etc) caused by MRSA and MSSA are displayed in Table 2. The anatomic location of MSKIs caused by MRSA and MSSA are displayed in Table 3. There were no statistically significant differences in distribution of clinical syndromes or anatomic location of infection between MRSA and MSSA (P > .05).
Table 1.
Comparisons of Musculoskeletal Infections Due to MRSA vs MSSAa
| Clinical Information | MRSA (n = 49) | MSSA (n = 42) | P Value |
|---|---|---|---|
| Epidemiologic Characteristics | |||
| M/F | 32:17 | 29:13 | .82 |
| Age (years) | 6.7 (3.3–11.1) | 5.7 (1.3–9.0) | .13 |
| Weight bearing at presentation (Y/N) | 21:28 | 12:30 | .19 |
| Local/disseminated infection | 19:30 | 11:31 | .26 |
| Markers of Severity | |||
| ED CRP (mg/L) | 96.2 (35.3–183.2) | 68.1 (40.5–145.0) | .28 |
| ED WBC (103/µL) | 12.4 (9.3–15.5) | 10.7 (8.5–13.2) | .08 |
| ED Temp (°F) | 99.7F (98.4–101.4) | 99.3F (98.5–100.6) | .59 |
| ED ESR (mm/hr) | 34.5 (16.5–58.3) | 37.0 (18.0–55.5) | .95 |
| Hospital Outcomes and Laboratory Values | |||
| Hospital LOS (days) | 4.8 (2.8–9.7) | 5.7 (4.0–9.0) | .50 |
| No. of days with fever | 0.0 (0.0–3.0) | 1.5 (0.0–3.0) | .10 |
| Duration of antibiotics (days) | 28 (10–49) | 34 (28–42) | .18 |
| Requiring operative interventions, n (%) | 41 (84%) | 26 (62%) | .015* |
| Thromboembolic disease, n (%) | 5 (10.2%) | 4 (9.5%) | >.99 |
Abbreviations: CRP, C-reactive protein; ED, emergency department; ESR, erythrocyte sedimentation rate; F, female; LOS, length of stay; M, male; MRSA, methicillin-resistant Staphylococcus aureus; MSSA, methicillin-susceptible S aureus; N, no; WBC, white blood cell count; Y, yes.
Median values reported with 25th and 75th percentiles in parentheses. P values determined by Mann-Whitney U test or Fisher’s exact test. P value significance defined as P < .05 (denoted by*).
Table 2.
Clinical Syndromes Caused by MRSA and MSSA
| Diagnosis | N (%) |
|---|---|
| MRSA (n = 49) | |
| Superficial abscess | 12 (24.5%) |
| Deep abscess | 4 (8.2%) |
| Osteomyelitis | 3 (6.1%) |
| Pyomyositis | 2 (4.1%) |
| Septic joint | 2 (4.1%) |
| Complexa | 26 (53.1%) |
| Total | 49 (100%) |
| MSSA (n = 42) | |
| Superficial abscess | 7 (16.7%) |
| Pyomyositis | 5 (11.9%) |
| Osteomyelitis | 3 (7.1%) |
| Septic joint | 3 (7.1%) |
| Deep abscess | 1 (2.4%) |
| Complexa | 23 (54.8%) |
| Total | 42 (100%) |
Abbreviations: MRSA, methicillin-resistant Staphylococcus aureus; MSSA, methicillin-susceptible S aureus.
Complex defined as involving a combination of muscle, joint, and bone tissue.
Table 3.
Anatomic Locations of Infections Caused by MRSA and MSSA
| MRSA (n = 49) | N (%) |
|---|---|
| Knee | 15 (30.6%) |
| Hip/pelvisa | 9 (18.4%) |
| Multifocal | 8 (16.3%) |
| Ankle | 3 (6.1%) |
| Thigh | 2 (4.1%) |
| Leg | 2 (4.1%) |
| Hand | 2 (4.1%) |
| Shoulder | 2 (4.1%) |
| Wrist | 1 (2.0%) |
| Forearm | 1 (2.0%) |
| Elbow | 1 (2.0%) |
| Spine | 1 (2.0%) |
| Arm | 1 (2.0%) |
| Foot | 1 (2.0%) |
| Total | 49 (100%) |
| MSSA (n = 42) | N (%) |
|---|---|
| Hip/pelvis | 17 (40.5%) |
| Multifocal | 8 (19.0%) |
| Knee | 5 (11.9%) |
| Leg | 4 (9.5%) |
| Thigh | 3 (7.1%) |
| Foot | 3 (7.1%) |
| Arm | 1 (2.4%) |
| Shoulder | 1 (2.4%) |
| Total | 42 (100%) |
Abbreviations: MRSA, methicillin-resistant Staphylococcus aureus; MSSA, methicillin-susceptible S aureus.
Definitions: Hip and pelvis defined as sacroiliac joint, pelvis, hip joint, proximal 1/3rd femur, thigh–middle 1/3rd femur, knee–distal 1/3rd femur, knee joint, proximal 1/3rd tibia, leg–middle 1/3rd tibia, ankle–distal 1/3rd tibia, ankle joint, hindfoot, foot–mid and forefoot, shoulder–shoulder girdle, shoulder joint, proximal 1/3rd humerus, arm–middle 1/3rd humerus, elbow–distal 1/3rd humerus, elbow joint, proximal 1/3rd radius/ulna, forearm–middle 1/3rd of radius/ulna, wrist–distal 1/3rd radius/ulna, wrist joint, carpus, hand–metacarpals and phalanges. Anatomic sites include surrounding muscle and soft tissue.
Inflammatory markers at presentation of MRSA and MSSA MSKI are compared in Table 1. There were no significant differences in CRP, ESR, WBC, or temperature at presentation between MRSA and MSSA MSKI. There were also no significant differences in hospital LOS, febrile days, or duration of antibiotics between MRSA and MSSA. However, a higher proportion of MRSA infections had operative interventions compared with MSSA infections. Subgroup analyses for MRSA and MSSA severity for local and disseminated infections (defined in Supplementary Table 1) yielded similar results, with no differences in severity for local or disseminated infections except for increased operative interventions for disseminated infections (odds ratio [OR], 3.15; 95% confidence interval [CI], 1.18–8.4).
To determine whether a predictive algorithm may be used to differentiate between MRSA and MSSA infections, a logistic regression was performed with relevant variables collected in the emergency department. The variables included in the logistic regression included CRP, temperature, WBC count, pulse, and respiratory rate at presentation. These variables were chosen based on anticipated clinical importance for differentiating between MRSA and MSSA [12, 13]. The ORs for CRP, temperature, pulse, and respiration were all nonsignificant (P > .05). The WBC count had a statistically significant OR of 1.15 favoring MRSA and a 95% CI of 1.03–1.29. A receiver operator characteristic (ROC) curve is displayed in Figure 1. The area under the ROC curve was 0.676, which suggests that the model has poor ability to differentiate between MRSA and MSSA disease [18].
Figure 1.
Logistic regression to predict methicillin-resistant Staphylococcus aureus (MRSA) versus methicillin-susceptible S aureus (MSSA) infection. A predictive model using C-reactive protein, temperature, white blood cells, pulse, and respiratory rate at presentation has a poor ability to differentiate between MRSA versus MSSA infection. The area under the receiver operator characteristic (ROC) was 0.6755, indicating poor predictive capacity.
DISCUSSION
This study compared all cases of pediatric S aureus MSKI at a large tertiary care children’s hospital over a 5-year period. The authors found no significant differences in inflammatory markers and LOS between MRSA and MSSA pediatric MSKIs. However, there were increased operative interventions for MRSA compared with MSSA.
Prior studies of children with MSKIs have demonstrated increased severity of disease caused by MRSA compared with MSSA. Hawkshead et al [8] reviewed 97 pediatric patients with S aureus osteomyelitis and reported that MRSA infections correlated with increased CRP, WBC count, hospital LOS, and number of interventions compared with MSSA. Likewise, Martínez-Aguilar [9] et al found that MSKI caused by MRSA resulted in more febrile days and longer hospital stays compared with MSSA in a retrospective review of 59 patients. Our study, involving an equivalent number of patients, did not reveal similar differences in hospital LOS between MRSA and MSSA infections, despite more operative interventions for MRSA infections. Because operative interventions were the only factors that differed between MRSA and MSSA infections, this may suggest a lower threshold by providers for drainage procedures after MRSA is identified by joint aspirate or bone biopsy. The lack of an overall severity difference between MRSA and MSSA disease may suggest a change in the relative virulence of MRSA and MSSA over time (ie, acquisition of increased virulence by community-acquired MSSA strains), because the previous studies were conducted 5–10 years ago or more. This may also be related to regional variability in the relative virulence of MRSA and MSSA, as well as host factors such as sites of infection or baseline comorbidities.
Staphylococcus aureus virulence is highly dependent on the presence of numerous factors including cytotoxins, adhesion molecules, and resistance mechanisms [19]. Leukocidins such as Panton-Valentine leukocidin, LukAB (also known as LukGH), and alpha hemolysin (Hla) are well characterized mediators of S aureus leukocyte destruction and infection pathogenesis [20–23]. Expression of these virulence factors varies in different strains of S aureus [24]. In addition, regulation of virulence gene expression depends on many factors including the accessory gene regulator (agr), which has a crucial role in quorum sensing and infection pathogenesis [25, 26]. Variable expression of these virulence factors likely contributes to the differences observed in S aureus infection severity, regardless of the presence of methicillin resistance.
Algorithms using clinical presentation and initial laboratory values have been developed to differentiate between MRSA and MSSA pediatric MSKIs. Ju et al [12] used temperature >38°C, hematocrit <34%, WBC >12 000 cell/µL, and CRP >13 mg/L as predictors of MRSA osteomyelitis in a logistic regression model with predictive probability of 92% for MRSA when all 4 predictors were satisfied. A separate model developed by Dietrich et al [13] that used temperature, absolute neutrophil count, and CRP as predictive variables demonstrated 87% accuracy in predicting cases of MRSA and MSSA. However, when these variables were used to generate a predictive model from our study cohort, the calculated areas under the ROCs were only 0.70 and 0.61, respectively. The difference in efficacy of the algorithms again may be due to a shift in relative virulence over time, regional differences in relative virulence, or differences in host factors between cohorts.
The present study was limited as a retrospective review because treatment protocols for individual patients varied based on the physicians involved in patient care. There also may have been bias in ascertainment, because patients were only included in the study if they received an orthopedic consult. This may have resulted in biased inclusion of more severe MSKIs. In addition, earlier identification of MSKI in the emergency department coupled with empiric antibiotic treatment effective against both MRSA and MSSA may have lessened the difference in clinical outcomes between strains compared with past studies. In addition, although the analysis includes 49 patients with MRSA and 42 with MSSA, it is possible that the sample size is underpowered to determine a true difference in subtle hospital outcomes between MRSA and MSSA. Furthermore, the authors recognize that the outcomes of hospital LOS and number of operative interventions are surrogate markers of MSKI severity and may be influenced by other factors besides infection severity.
CONCLUSIONS
In the absence of effective clinical prediction models for differentiation between MRSA and MSSA, empiric treatment covering both MRSA and MSSA should remain the standard of care. In the future, rapid molecular diagnostic tests that identify MRSA and MSSA may eliminate the need for (potentially inaccurate) clinical prediction algorithms and improve antibiotic stewardship.
Supplementary Data
Supplementary materials are available at Open Forum Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.
Supplementary Material
Acknowledgments
Disclaimer. The findings and conclusions in this report are those of the authors and do not necessarily represent the views of the National Institutes of Health or Pediatric Orthopedic Society of North America (POSNA).
Financial support. This work was funded in part by the National Institutes of Health (Grant K23AI113150; to I. P. T.) and the POSNA (to J. S.).
Potential conflicts of interest. All authors: No reported conflicts.
All authors have submitted the ICMJE Form for Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.
References
- 1. Gafur OA, Copley LA, Hollmig ST, et al. The impact of the current epidemiology of pediatric musculoskeletal infection on evaluation and treatment guidelines. J Pediatr Orthop 2008; 28:777–85. [DOI] [PubMed] [Google Scholar]
- 2. Sarkissian EJ, Gans I, Gunderson MA, et al. Community-acquired methicillin-resistant Staphylococcus aureus musculoskeletal infections: emerging trends over the past decade. J Pediatr Orthop 2015; 00:1–5. [DOI] [PubMed] [Google Scholar]
- 3. Saavedra-Lozano J, Mejías A, Ahmad N, et al. Changing trends in acute osteomyelitis in children: impact of methicillin-resistant Staphylococcus aureus infections. J Pediatr Orthop 2015; 28:569–575. [DOI] [PubMed] [Google Scholar]
- 4. Williams DJ, Deis JN, Tardy J, Creech CB. Culture-negative osteoarticular infections in the era of community-associated methicillin-resistant Staphylococcus aureus. Pediatr Infect Dis J 2011; 30:523–5. [DOI] [PubMed] [Google Scholar]
- 5. Mera RM, Suaya JA, Amrine-Madsen H, et al. Increasing role of Staphylococcus aureus and community-acquired methicillin-resistant Staphylococcus aureus infections in the United States: a 10-year trend of replacement and expansion. Microb Drug Resist 2011; 17:321–8. [DOI] [PubMed] [Google Scholar]
- 6. Klein EY, Sun L, Smith DL, Laxminarayan R. The changing epidemiology of methicillin-resistant Staphylococcus aureus in the United States: a national observational study. Am J Epidemiol 2013; 177:666–74. [DOI] [PubMed] [Google Scholar]
- 7. Arnold SR, Elias D, Buckingham SC, et al. Changing patterns of acute hematogenous osteomyelitis and septic arthritis: emergence of community-associated methicillin-resistant Staphylococcus aureus. J Pediatr Orthop 2006; 26:703–8. [DOI] [PubMed] [Google Scholar]
- 8. Hawkshead JJ, 3rd, Patel NB, Steele RW, Heinrich SD. Comparative severity of pediatric osteomyelitis attributable to methicillin-resistant versus methicillin-sensitive Staphylococcus aureus. J Pediatr Orthop 2009; 29:85–90. [DOI] [PubMed] [Google Scholar]
- 9. Martínez-Aguilar G, Avalos-Mishaan A, Hulten K, et al. Community-acquired, methicillin-resistant and methicillin-susceptible Staphylococcus aureus musculoskeletal infections in children. Pediatr Infect Dis J 2004; 23:701–6. [DOI] [PubMed] [Google Scholar]
- 10. Sicot N, Khanafer N, Meyssonnier V, et al. Methicillin resistance is not a predictor of severity in community-acquired Staphylococcus aureus necrotizing pneumonia–results of a prospective observational study. Clin Microbiol Infect 2013; 19:E142–8. [DOI] [PubMed] [Google Scholar]
- 11. Yaw LK, Robinson JO, Ho KM. A comparison of long-term outcomes after meticillin-resistant and meticillin-sensitive Staphylococcus aureus bacteraemia: an observational cohort study. Lancet Infect Dis 2014; 14:967–75. [DOI] [PubMed] [Google Scholar]
- 12. Ju KL, Zurakowski D, Kocher MS. Differentiating between methicillin-resistant and methicillin-sensitive Staphylococcus aureus osteomyelitis in children: an evidence-based clinical prediction algorithm. J Bone Joint Surg Am 2011; 93:1693–701. [DOI] [PubMed] [Google Scholar]
- 13. Dietrich LN, Reid D, Doo D, et al. Predicting MSSA in acute hematogenous osteomyelitis in a setting with MRSA prevalence. J Pediatr Orthop 2014; 35:1. [DOI] [PubMed] [Google Scholar]
- 14. Diekema DJ, Pfaller M, Schmitz FJ, et al. Survey of infections due to Staphylococcus species: frequency of occurrence and antimicrobial susceptibility of isolates collected in the United States, Canada, Latin America, Europe, and the Western Pacific region for the SENTRY Antimicrobial Surveillanc. Clin Infect Dis 2001; 32Suppl 2:S114–32. [DOI] [PubMed] [Google Scholar]
- 15. Voss A, Milatovic D, Wallrauch-Schwarz C, et al. Methicillin-resistant Staphylococcus aureus in Europe. Eur J Clin Microbiol Infect Dis 1994; 13:50–5. [DOI] [PubMed] [Google Scholar]
- 16. Sutter DE, Milburn E, Chukwuma U, et al. Changing Susceptibility of Staphylococcus aureus in a US pediatric population. Pediatrics 2016; 137:2015–3099. [DOI] [PubMed] [Google Scholar]
- 17. Mignemi ME, Benvenuti MA, An TJ, et al. A novel classification system based on dissemination of musculoskeletal infection is predictive of hospital outcomes. J Pediatr Orthop 2016. doi:10.1097/BPO.0000000000000811. [DOI] [PubMed] [Google Scholar]
- 18. Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation 2007; 115:928–35. [DOI] [PubMed] [Google Scholar]
- 19. Vandenesch F, Lina G, Henry T. Staphylococcus aureus hemolysins, bi-component leukocidins, and cytolytic peptides: a redundant arsenal of membrane-damaging virulence factors? Front Cell Infect Microbiol 2012; 2:12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. DuMont AL, Nygaard T, Watkins R, et al. Characterization of a new cytotoxin that contributes to Staphylococcus aureus pathogenesis. Mol Microbiol 2009; 130:9492–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Boyle-Vavra S, Daum RS. Community-acquired methicillin-resistant Staphylococcus aureus: the role of Panton-Valentine leukocidin. Lab Invest 2007; 87:3–9. [DOI] [PubMed] [Google Scholar]
- 22. Bubeck Wardenburg J, Patel RJ, Schneewind O. Surface proteins and exotoxins are required for the pathogenesis of Staphylococcus aureus pneumonia. Infect Immun 2007; 75:1040–4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. DuMont AL, Yoong P, Surewaard BG, et al. Staphylococcus aureus elaborates leukocidin AB to mediate escape from within human neutrophils. Infect Immun 2013; 81:1830–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Ziebandt AK, Kusch H, Degner M, et al. Proteomics uncovers extreme heterogeneity in the Staphylococcus aureus exoproteome due to genomic plasticity and variant gene regulation. Proteomics 2010; 10:1634–44. [DOI] [PubMed] [Google Scholar]
- 25. Yarwood JM, Schlievert PM. Quorum sensing in Staphylococcus infections. J Clin Invest 2003; 112:1620–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Recsei P, Kreiswirth B, O’Reilly M, et al. Regulation of exoprotein gene expression in Staphylococcus aureus by agar. Mol Gen Genet 1986; 202:58–61. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.