Abstract
Objective
To assess urology residency program modifications in the context of COVID-19, and perceptions of the impact on urology trainees.
Methods
A cross-sectional survey of program leadership and residents at accredited US urology residencies was administered between April 28, 2020 to March 11, 2020. Total cohort responses are reported, and subanalyses were preformed comparing responses between those in in high vs low COVID-19 geographic regions, and between program leaders vs residents.
Results
Program leaders from 43% of programs and residents from 18% of programs responded. Respondents reported decreased surgical volume (83%-100% varying by subspecialty), increased use of telehealth (99%), a transition to virtual educational platforms (95%) and decreased size of inpatient resident teams (90%). Most residents are participating in care of COVID-19 patients (83%) and 20% endorsed that urology residents have been re-deployed. Seventy nine percent of respondents perceive a negative impact of recent events on urology surgery training and anxiety regarding competency upon completion of residency training was more pronounced among respondents in high COVID-19 regions.
Conclusion
Major modifications to urology training programs were implemented in response to COVID-19. Attention must be paid to the downstream effects of the training disruption on urology residents.
The viral pandemic disease, coronavirus disease 2019 (COVID-19), has abruptly disrupted urology practice and urology training. The immediate and long-term impacts remain unknown and have yet to be systematically studied. The impact of COVID-19 on the United States (US) healthcare workforce is multi-faceted and continues to evolve. In geographic regions that have been inundated with COVID-19 patients requiring hospital care, resources such as personnel, hospital beds, ventilators and personal protective equipment are in high demand and short supply. In such settings, hospitals are further constrained by providers contracting COVID-19, necessitating self-isolation as recommended by the Centers for Disease Control. Re-deployment has been implemented in these hospitals in an effort to meet patient care needs. In contrast, many hospitals in regions with lower COVID-19 prevalence have limited their routine patient care to preserve resources, resulting in decreased clinical volume.
As a surgical subspecialist, the urologist typically performs a high volume of scheduled surgeries in ambulatory surgery centers and hospital based operating rooms, in addition to clinic procedures and ambulatory clinic visits. This traditional healthcare delivery pattern has been severely disrupted by COVID-19. The downstream impact on the urologic trainee with respect to surgical and ambulatory volume, educational opportunities, and workforce restructuring has yet to be described. Furthermore, amidst the uncertainty regarding the future of healthcare, there are likely to be varying impacts on the trainee's psyche ranging from moral injury, burnout, and fatigue to a renewed sense of purpose in medicine, improved morale and pride in one's work. In this study, we aim to assess the impact of healthcare disruption due to COVID-19 on urology residency programs and trainees via survey of US program directors (PDs) and residents. We hypothesized a priori that a reduced case volume would lead to a decreased perception of surgical preparedness, but an improved sense of morale and purpose.
MATERIALS AND METHODS
Study Population
An anonymous online survey was administered to residents and program leadership (PDs and associate PDs [APDs]) in the United States. The American Urological Association (AUA) website was used to identify the 142 accredited urology residency programs. PD email addresses were collected from the AUA residency listing page, the AUA member directory, or individual program websites. Of the accredited urology residency programs, 15 programs did not have available contact information for program leadership. The survey was sent to 127 PDs with request to complete and forward communication to their APD (if applicable), and their resident cohort for completion. The survey was distributed on April 28, 2020 and closed on May 11, 2020. No responses were excluded. This study was deemed exempt from review by the Institutional Review Board at the University of California, Los Angeles.
Survey Design
The 27-question survey was designed and administered via our institutional license of the web-based Qualtrics platform (Qualtrics, Provo, UT). Prior to distribution, the survey questions were developed in an iterative fashion with input from the authors and additional faculty and residents at our institution. We collected demographic data including respondent type (PD, APD, resident), program location, gender, training year (if applicable). Additional domains assessed included (1) clinical modifications (surgical and ambulatory), (2) educational modifications, (3) workforce restructuring, and (4) perceptions of the impact of COVID-19 on training programs and trainees. This final domain of the survey was an attempt to address perceptions of the intangible impacts of the viral pandemic on urology training programs and trainees. Questions in this domain were phrased in a narrative fashion and responses collected on a 5-point Likert scale. The complete survey can be found in supplemental material.
Statistical Analysis
Responses were exported from Qualtrics to Microsoft Excel and all statistical analyses were performed with Stata statistical software version 16.1 (StataCorp, College Station, Texas). All survey question responses were coded as binary or categorical variables. Descriptive statistics of the overall cohort were performed. We defined a geographic variable to identify respondents in high-COVID-19 regions. Respondents from the 10 US states or districts with the highest number of per-capita COVID-19 infections at the time of survey closure (May 11, 2020) were considered to be in ‘high COVID-19’ regions (New York, New Jersey, Massachusetts, Rhode Island, Connecticut, Washington D.C, Delaware, Louisiana, Illinois, and Maryland). We performed sub-group analysis with data stratified by high COVID-19 vs low COVID-19 geographic status comparing outcomes with Pearson's chi-square test. For additional sub-group analysis, PD and APD responses were combined and defined as ‘program leaders’. Using Pearson's chi-square test we then compared program leader vs resident responses to investigate any differences in perception of residency impact related to role. P values <.05 were considered significant.
RESULTS
Demographics
We received survey responses from 64 program leaders representing 55 programs (55/127 = 43%) and 106 residents representing 23 programs (23/127 = 18%). The geographic distribution of respondents is represented in Figure 1 ; 32% were located in high COVID-19 regions. Most respondents were male (66%). Of the resident respondents, the majority were junior residents (67% PGYs 1-3 vs 33% PGYs 4-6).
Clinical and Educational Modifications
There were consistent reports of decreased surgical and ambulatory volume. We queried changes in specific subspecialties’ surgical case volume and saw reported a decrease in volume across all subspecialties, including urologic emergency case volume (Fig. 2 ). Residents continued to assist in surgical cases that were ongoing (93%). Almost all (99%) respondents reported implementation of telehealth for ambulatory visits; fewer reported resident participation in telehealth encounters (65%) and continued participation in in-person clinic encounters (51%). We assessed changes to standard educational conferences including grand rounds, didactics, journal club, morbidity & mortality, and indications conferences. Nearly all respondents (99%) reported discontinuation of in-person conferences with most reporting a transition to virtual platforms (95%). Fifty four percent of respondents reported an increase in number of educational activities.
Workforce Restructuring
Overall, 90% of respondents reported a decrease in the number of residents per team managing inpatients (admitted patients and consultations). A majority (83%) reported resident participation in the care of COVID-19 infected patients, or persons under investigation, during their duties as a urology resident. More than half of respondents reported a resident had to stay-at-home for some period of time due to exposure, personal illness or family member illness (57%). We also queried how programs were managing trainees who fall into at-risk populations due to pregnancy or immunocompromise. The majority responded that some modification has been made: 42% (28/66) of those who responded to this question reported pregnant residents are not providing care to COVID-19 positive patients; 36% (24/66) of those who responded reported pregnant residents are not providing direct patient care at all. Similar modifications were reported regarding immunocompromised residents: 40% of respondents reported these residents are not providing care to COVID-19 positive patients and 40% reported they are not providing direct patient care. Urology trainees were re-deployed (20%) in a diverse number of settings (61% intensive care unit, 39% wards, 15% emergency room, 6% nonurology telehealth, 6% general surgery team, 3% invasive-procedures team; more than 1 choice allowed). Education in preparation for possible redeployment was provided in the form of in-person didactics (8.4%), in-person procedural instruction (13%), virtual didactic or self-directed learning (48%) (more than 1 format choice allowed). Additional support services that were provided to urology trainees by either institutions or programs included childcare (60%), temporary overnight accommodations (75%) and meals (53%) (more than 1 response allowed).
Impact on Trainees
We queried perceptions of the downstream effects of modifications to urology training programs (Table 1 ). Most (80%) respondents reported agreement (“agree” or “strongly agree”) with a statement indicating that program changes have negatively impacted surgical training. Half (51%) reported agreement with a statement endorsing increased anxiety about competency upon residency completion. However, only a small proportion (9%) agreed with a statement supporting increased likelihood of pursuit of fellowship training. A majority of respondents agreed with statements regarding recent changes resulting in more time for self-directed learning (90%) and more time for research (77%). A minority of respondents agreed with statements endorsing improved morale (23%) and more pride in their work (29%) in the wake of COVID-19 related program changes. Approximately half (54%) agreed with a statement endorsing home-life disruption and 39% agreed with a statement endorsing increased financial concerns.
Table 1.
Changes in Urology Services due to COVID-19 Have: | Full Cohort | High COVID-19 Region |
||
---|---|---|---|---|
No | Yes | P value | ||
N = 170 | N = 116 | N = 54 | ||
Had a negative impact on surgical training: | .40 | |||
Disagree | 8% | 9% | 4% | |
Neutral | 12% | 12% | 11% | |
Agree | 79% | 77% | 83% | |
Unanswered | 2% | 2% | 2% | |
Increased anxiety about competency on residency completion: | .02 | |||
Disagree | 29% | 35% | 15% | |
Neutral | 19% | 18% | 20% | |
Agree | 51% | 45% | 63% | |
Unanswered | 2% | 2% | 2% | |
Allowed more time for self-directed learning: | .35 | |||
Disagree | 5% | 4% | 7% | |
Neutral | 5% | 3% | 7% | |
Agree | 88% | 91% | 83% | |
Unanswered | 2% | 2% | 2% | |
Allowed more time for research: | .07 | |||
Disagree | 7% | 4% | 13% | |
Neutral | 15% | 14% | 19% | |
Agree | 76% | 80% | 67% | |
Unanswered | 2% | 2% | 2% | |
Made me feel more pride in my work: | .58 | |||
Disagree | 24% | 22% | 28% | |
Neutral | 45% | 45% | 46% | |
Agree | 29% | 31% | 24% | |
Unanswered | 2% | 2% | 2% | |
Improved morale | .14 | |||
Disagree | 45% | 40% | 56% | |
Neutral | 31% | 34% | 26% | |
Agree | 22% | 25% | 17% | |
Unanswered | 2% | 2% | 2% | |
Increased the likelihood of postresidency fellowship training: | .14 | |||
Disagree | 34% | 38% | 24% | |
Neutral | 55% | 53% | 59% | |
Agree | 9% | 7% | 13% | |
Unanswered | 2% | 2% | 4% | |
Disrupted home life: | .51 | |||
Disagree | 24% | 27% | 19% | |
Neutral | 21% | 20% | 22% | |
Agree | 54% | 52% | 57% | |
Unanswered | 2% | 2% | 2% | |
Increased financial concerns: | .99 | |||
Disagree | 32% | 33% | 31% | |
Neutral | 28% | 28% | 28% | |
Agree | 38% | 38% | 39% | |
Unanswered | 2% | 2% | 2% |
Comparison of High and Low COVID-19 Regions
In subgroup analysis we compared responses between high COVID-19 vs low COVID-19 regions, as previously defined (Supplemental Table 1). Significantly more respondents from high COVID-19 regions reported decreased emergency urologic surgical volume (76% vs 22%, P = .01) and cancelled educational activities (11% vs 1%, P <.01). A significantly larger proportion of respondents in high COVID-19 regions reported urology trainees caring for COVID-19 patients (91% vs 7%, P = .03), being re-deployed (37% vs 11%, P <.01), and being required to stay at home due to sickness or exposure (78% vs 20%, P <.01). More respondents in high COVID-19 regions reported that pregnant residents continue to work without modification (11% vs 7%, P <.01, Supplemental Table 1). There was a significantly higher proportion of respondents in high COVID-19 regions who endorsed increased anxiety about competency upon completion of residency (63% vs 45%, P = .02) (Table 1). There was no significant difference in the proportion of respondents who agreed with statements regarding increased pride in work or improved morale between cohorts.
Comparison of Program Leader and Resident Responses
In additional sub-group analysis, we compared responses between program leaders and residents. There was concordance in nearly every domain, with no statistically significant differences between groups in responses regarding clinical modifications, educational modifications or workforce restructuring. However, responses to 2 questions regarding perception of impacts differed significantly. Compared to program leaders, residents were more likely to disagree with a statement that “changes in urology services due to COVID-19 have disrupted home life” (31% vs 12%, P = .013). Compared to program leaders, residents were less likely to agree with a statement that ‘changes in urology services due to COVID-19 have increased my worries about my family's finances’ (27% vs 56% P <.01).
DISCUSSION
Here, we report the first US national survey of urology residency program leadership and trainees assessing COVID-19-related program modifications and their impact on trainees. Our results confirm the hypothesis that significant modifications have been made across all aspects of surgical training. In accordance with Urologic and other surgical society guidelines recommending postponement of non-emergent surgeries, nearly all respondents reported a decrease in surgeries performed across all subspecialties.1, 2, 3 Interestingly, surgical volume decrease was also noted for emergency services, particularly in high COVID-19 regions, which may reflect an aversion of the general public to seek any care due to fear related to COVID-19.4 The surgical volume decrease has important implications regarding experience lost to trainees. In light of this, it is not surprising that most respondents agreed with a statement that COVID-19 related changes have negatively impacted surgical training. The Accreditation Council for Graduate Medical Education and the American Board of Urology maintain procedural and surgical case-based standards for accreditation and resident advancement.5 , 6 Because this survey was administered at a single time point, we cannot comment on duration of decreased case volume, however attention should be paid to resident-reported Accreditation Council for Graduate Medical Education case logs during this time-period, and into the coming months. Program leaders and governing bodies should consider modifications of, or exceptions to, volume-based standards (ie, replace or supplement with competency-based standards). For most residents in 5- or 6-year programs, a short duration of decrease in high volume subspecialties (ie, oncology, endourology) will likely be offset by adequate volume during non-peak-pandemic times. However, many trainees have more limited exposure to subspecialties including pediatrics and reconstructive urology. Residents on such a rotation during the peak-pandemic scale-back may have missed out entirely on their required case volumes and critical exposure to a potential field of interest. Program leaders must consider options to compensate for such missed opportunities, including flexibility in future rotation scheduling or off-rotation experiences. Worth noting is the significant proportion of respondents who endorsed increased time for self-directed learning and research efforts. The notion that value lost with decreased surgical volume may be offset by enhanced time for other educational endeavors is intriguing, given the complex cognitive machinations needed to successfully treat patients with surgical diseases. Future study is needed into the relative effects of these changes.
In the ambulatory setting, a similar scale-back was seen with nearly all respondents reporting decreased in-person clinic visits and implementation of telehealth encounters. While 65% of respondents report that residents are participating in telehealth encounters, there is certainly room for improvement in engaging residents in a meaningful way in this aspect of patient care. There has been much discussion in the literature regarding implementation of telehealth as an opportunity for clinical education; we must be sure to maximize educational value as telehealth will possibly become a fixed component of urologic patient care.7, 8, 9
In the context of Centers for Disease Control recommendations to employ social distancing and avoid large groups, it is not surprising that our respondents endorsed a nearly universal transition to virtual platforms for conferences and didactics serving educational purposes.10 While not surprising, it is important to note that respondents in high COVID-19 regions reported a high proportion of educational sessions being cancelled outright. However, a high proportion of respondents in both high and low-COVID-19 regions (83% and 91%) reported that program changes due to COVID-19 have allowed for more time for self-directed learning. How to best capitalize on this time, and attempt to compensate for decreased operative experience, is of utmost importance. For years, medical educators have been investigating novel strategies to educate adult trainees and the COVID-19 pandemic has forced adoption of some of these strategies. Some of these novel strategies include employing a flipped classroom (learners first pre-study with a didactic lecture followed by expert-led, interactive session), and generating libraries of virtual lectures and surgical videos from thought-leaders in the field.11 , 12 Rapid implementation of virtual didactic series have enabled learners across the US (and internationally) access to free, high-quality, educational material.13 , 14
We found significant workforce restructuring occurred among urology trainees. Changes included decreased size of inpatient teams, education in preparation for redeployment, and, particularly in high COVID-19 regions, re-deployment. These findings are in line with modifications recently reported in otolaryngology, general surgery and neurosurgery.15, 16, 17 Programs also had to manage resident absence due to personal illness or exposure, and protect medically vulnerable residents.18 Although there is a dearth of information regarding the impacts of COVID-19 infection on pregnant women, their fetuses and their newborns, various expert medical societies have recommended avoidance of contact with COVID-19 positive individuals, and at least one national society recommended pregnant health-care workers in their third trimester abstain from all patient contact.19 , 20 Although the absolute number of pregnant urology residents in the US is likely small, many of our respondents reported some clinical modification for pregnant trainees. However, high COVID-19 region respondents reported a higher proportion of pregnant residents working without modification, likely representing a more critical need for medical providers in those regions. Program leaders should consider scheduling modifications (ie, participation in telehealth) as feasible.
Respondent perceptions of the impact of program changes on urology trainees were surprising. Most agreed with a statement that program changes have negatively impacted surgical training, many agreed with a statement endorsing increased anxiety about competency upon residency completion. Subgroup analysis highlighted a higher proportion of respondents in the high COVID-19 region cohort endorsed increased anxiety about competency upon completion of residency. This aligns predictably with the difference between high and low COVID-19 region cohorts with respect to resident redeployment and residents required to stay-at-home, proxies for more severe disruption in Urologic training. An awareness of this finding is important for program leaders across the country, and particularly in high COVID-19 areas. Interestingly, there was no difference between the high and low COVID-19 area cohorts regarding morale or pride in one's work; our a priori hypothesis that the pandemic experience in high COVID-19 regions would elicit a galvanizing or demoralizing response was not supported. This may reflect overall resident resiliency or that psychological effects may present in a delayed fashion. Half of respondents endorsed agreement with a statement regarding disrupted home life, despite widely available supportive services including childcare, temporary accommodations and supplemental meals. This aligns with the general surgery literature where residents surveyed were primarily concerned about the impact of COVID-19 on their family.21 Compared to program leaders, residents appear relatively protected from anxieties surrounding financial toxicity of the COVID-19 pandemic, likely because of the contractual and fixed nature of their salary.
Our study has several limitations. As a survey study it is subject to response bias related to phrasing and question order. Additionally, respondents in areas impacted by COVID-19 may be over-represented due to interest or perceived applicability. With respect to our decision to define high COVID-19 regions on the state level, we are aware that we fail to capture the variability of COVID-19 infection density down to the city level. However, most residency programs are affiliated with large medical centers typically located in high population-density areas of a given state, a fact that helps overcome this limitation. Finally, while overall program representation was strong (which is good for relatively objective measures such as impact on surgical and clinical volumes, education strategies and workforce restructuring) we may lose some power to detect more individualized (ie, psychosocial) impacts of COVID-19 on trainees by our relatively low raw resident response rate.
CONCLUSIONS
In the months after the emergence of COVID-19, US urology residency programs underwent significant modifications including decreased surgical and ambulatory volume, increased use of telehealth, increased educational activities via virtual platforms and significant workforce restructuring. In the context of these changes, program leaders and trainees perceive an overall negative impact on surgical training and increased anxiety about competency. It will be important for program leaders and trainees to collaborate in developing solutions to the unique challenges faced by trainees during this time period and moving forward.
Footnotes
Supplementary material associated with this article can be found in the online version at https://doi.org/10.1016/j.urology.2020.05.051.
Appendix. SUPPLEMENTARY MATERIALS
References
- 1.Ribal MJ, Cornford P, Briganti A, et al. European association of urology guidelines office rapid reaction group: an organisation-wide collaborative effort to adapt the european association of urology guidelines recommendations to the coronavirus disease 2019 era. Eur Urol. 2020:1–8. doi: 10.1016/j.eururo.2020.04.056. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Diaz A, Sarac BA, Schoenbrunner AR, Janis JE, Pawlik TM. Elective surgery in the time of COVID-19. Am J Surg. 2020;(xxxx):4–6. doi: 10.1016/j.amjsurg.2020.04.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.COVID-19 . American College of Surgeons; 2020. Elective Case Triage Guidelines for Surgical Care.https://www.facs.org/covid-19/clinical-guidance/elective-case Available at: Published Accessed May 17, 2020. [Google Scholar]
- 4.Garcia S, Albaghdadi MS, Meraj PM, et al. Reduction in ST-Segment elevation cardiac catheterization laboratory activations in the United States during COVID-19 pandemic. J Am Coll Cardiol. 2020 doi: 10.1016/j.jacc.2020.04.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Accreditation Council for Graduate Medical Education: Urology. Available at: https://www.acgme.org/Specialties/Overview/pfcatid/26/Urology. Accessed May 17, 2020.
- 6.American Board of Urology: Residency Requirements. Available at:https://www.abu.org/residency-requirements/. Accessed May 17, 2020.
- 7.Chick RC, Clifton GT, Peace KM, et al. Using technology to maintain the education of residents during the COVID-19 pandemic. J Surg Educ. 2020 doi: 10.1016/j.jsurg.2020.03.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Gadzinski AJ, Andino JJ, Odisho AY, Watts KL, Gore JL, Ellimoottil C. Telemedicine and eConsults for hospitalized patients during COVID-19. Urology. 2020:1–3. doi: 10.1016/j.urology.2020.04.061. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Gadzinski AJ, Gore JL, Ellimoottil C, Odisho AY, Watts KL. Implementing telemedicine in response to the COVID-19 pandemic. J Urol. 2020:1–3. doi: 10.1097/JU.0000000000001033. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Centers for Disease Control. Coronavirus Disease 2019 (COVID-19): Communities, Schools, Workplaces, and Events. Available at:https://www.cdc.gov/coronavirus/2019-ncov/community/index.html. Accessed May 17, 2020.
- 11.Coe TM, Jogerst KM, Sell NM, et al. Practical techniques to adapt surgical resident education to the COVID-19 era. Ann Surg. 2020;53:1. doi: 10.1097/SLA.0000000000003993. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Kwon YS, Tabakin AL, Patel HV, et al. Adapting urology residency training in the COVID-19 era. Urology. 2020;(January) doi: 10.1016/j.urology.2020.04.065. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Urology Collaborative Online Video Didactics (COViD). Available at:https://urologycovid.ucsf.edu/. Accessed May 17, 2020.
- 14.EMPIRE: Urology Lecture Series: New York Section, AUA. Available at: https://nyaua.com/empire/. Accessed May 17, 2020.
- 15.Nassar AH, Zern NK, Mcintyre LK, et al. Emergency restructuring of a general surgery residency program during the coronavirus disease 2019 pandemic: the University of Washington Experience. JAMA Surg. 2020:3–6. doi: 10.1001/jamasurg.2020.1219. [DOI] [PubMed] [Google Scholar]
- 16.Rasmussen SA, Smulian JC, Lednicky JA, Wen TS, Jamieson DJ. Impact of the COVID-19 pandemic on neurosurgical residency training in new orleans. World Neurosurgery. 2020 doi: 10.1016/j.wneu.2020.04.208. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Cai Y, Jiam NT, Wai KC, Shuman EA, Roland LT, Chang JL. Otolaryngology resident practices and perceptions in the initial phase of the U.S. COVID-19 pandemic. Laryngoscope. 2020 doi: 10.1002/lary.28733. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Vargo E, Ali M, Henry F, et al. Cleveland clinic akron general urology residency program's COVID-19 experience. Urology. 2020;(January) doi: 10.1016/j.urology.2020.04.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.RCOG - Royal College of Obstetricians and Gynaecologists. Occupational Health Advice for Employers and Pregnant Women during the COVID-19 Pandemic. Available at: https://www.rcog.org.uk/globalassets/documents/guidelines/2020-04-27-occupational–health–advice–for–employers-and–pregnant-women.pdf. Accessed May 17, 2020.
- 20.Rimmer A. COVID-19: doctors in final trimester of pregnancy should avoid direct patient contact. BMJ. 2020;368:m1173. doi: 10.1136/bmj.m1173. [DOI] [PubMed] [Google Scholar]
- 21.He K, Stolarski A, Whang E, Kristo G. Addressing general surgery residents’ concerns in the early phase of the COVID-19 pandemic. J Surg Educ. 2020 doi: 10.1016/j.jsurg.2020.04.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.