Abstract
Background
Efforts to reduce nosocomial spread of COVID-19 have resulted in unprecedented disruptions in clinical workflows and numerous unexpected stressors for imaging departments across the country. Our purpose was to more precisely evaluate these impacts on radiologists through a nationwide survey.
Methods
A 43-item anonymous questionnaire was adapted from the AO Spine Foundation's survey and distributed to 1521 unique email addresses using REDCap™ (Research Electronic Data Capture). Additional invitations were sent out to American Society of Emergency Radiology (ASER) and Association of University Radiologists (AUR) members. Responses were collected over a period of 8 days. Descriptive analyses and multivariate modeling were performed using SAS v9.4 software.
Results
A total of 689 responses from radiologists across 44 different states met the criteria for inclusion in the analysis. As many as 61% of respondents rated their level of anxiety with regard to COVID-19 to be a 7 out of 10 or greater, and higher scores were positively correlated the standardized number of COVID-19 cases in a respondent's state (RR = 1.11, 95% CI: 1.02–1.21, p = 0.01). Citing the stressor of “personal health” was a strong predictor of higher anxiety scores (RR 1.23; 95% CI: 1.13–1.34, p < 0.01). By contrast, participants who reported needing no coping methods were more likely to self-report lower anxiety scores (RR 0.4; 95% CI: 0.3–0.53, p < 0.01).
Conclusion
COVID-19 has had a significant impact on radiologists across the nation. As these unique stressors continue to evolve, further attention must be paid to the ways in which we may continue to support radiologists working in drastically altered practice environments and in remote settings.
Abbreviations: ASER, American Society of Emergency Radiology; AUR, Association of University Radiologists; REDCap, Research Electronic Data Capture; COVID-19, Coronavirus Disease 2019; US, United States; CT, computed tomography; PUI, persons under investigation; PPE, personal protective equipment; CXR, chest X-ray
Keywords: Radiology, Anxiety, COVID-19, Psychological impact, Chest X-ray, Pneumonia, Coronavirus, SARS-CoV-2, Pandemic, Unemployment, Stress, Economic distress
Highlights
-
•
Citing the stressor of “personal health” was predictive of higher self-reported anxiety scores.
-
•
Individuals who responded with “no coping needed” were less likely to report high levels of anxiety.
-
•
Childcare assistance for healthcare workers remains an area of opportunity for institutional support.
1. Introduction
Rapidly escalating reports of Coronavirus Disease 2019 (COVID-19) worldwide eventually led the World Health Organization (WHO) to officially declare a state of global pandemic on March 11, 2020 [1]. Since then, the United States (US) has experienced an exponential growth in viral spread, with close to 1,800,000 cases and over 100,000 deaths at the time of this writing [2]. Although radiologists are not routinely considered among frontline workers, chest imaging nevertheless remains one of the pillars of the workup and risk stratification of COVID-19 cases [[3], [4], [5], [6],44]. Yet, shifting perspectives on the utility of chest computed tomography (CT) in the diagnosis and screening of COVID-19 have resulted in conflicting guidelines from leading societies [[7], [8], [9], [10]]. Moreover, heightened infection control parameters led to an inability to meet demand for urgent imaging in many medical centers around the country, whereby cleaning protocols for persons under investigation (PUI) may require up to an hour of downtime in between scans [11,12,44]. Especially during the early days of the pandemic, these factors and others contributed significantly to strain and unease among imaging centers nationwide.
Major challenges related to the preparedness of radiology departments in the wake of the outbreak included, but were not limited to, personal protective equipment (PPE) shortages, infection control protocols related to imaging of PUI for SARS-CoV-2 infection, and providing for adequate teleradiology infrastructure for those radiologists who were able to work remotely [13,14]. In particular, rethinking how clinical work can be delivered in a timely manner while minimizing the risk of infection may have lasting impacts on the practice of radiology long after the pandemic subsides [15,16]. The level of impact that the outbreak has had on personal wellness is also not to be underestimated. As described in previous literature, healthcare workers experienced significant adverse psychological effects during the Severe Acute Respiratory Syndrome (SARS) global outbreak of 2003 [[17], [18], [19]]. Emerging studies on healthcare workers during the current COVID-19 pandemic are showing expectedly similar sequelae, with reports of increased anxiety, depression, insomnia, and psychological distress [[20], [21], [22], [23]]. Yet, for the many radiologists who are uniquely able to work in semi-secluded reading stations or remotely, data on the extent of these psychosocial impacts is lacking [13,14].
A more precise understanding of these challenges faced by imaging centers nationwide will better inform medical center guidelines and wellness initiatives to help support radiologists during these unprecedented times. To this end, we adapted the AO Spine Foundation's survey investigating the impacts of COVID-19 on spine surgeons to create our survey studying the impacts of the pandemic on radiologists [24]. As a secondary objective, we assessed the factors associated with high levels of anxiety in relation to the COVID-19 pandemic [[17], [18], [19], [20], [21], [22], [23]].
2. Material and methods
This study was designed as an anonymous online survey which aimed to capture the perceived impact of COVID-19 on radiologists. Our survey was adapted from the AO Spine Foundation's survey studying the impact of COVID-19 on spine surgeons [24]. This study was exempt from review as determined by our Institutional Review Board (application number HS-20-00291).
2.1. The survey
The core study group, which included 3 radiologists, drafted and pilot tested the survey. The survey consisted of 43 questions with an estimated time to completion of approximately 10 min. Data collected included participants' general demographics and perceived impact of the COVID-19 pandemic on the following domains: broad impact; clinical duties; institutional preparedness; research and education; finances; and psychological impact. The response options were designed in the following formats: binary (yes/no), with options to specify for “I don't know” and/or “not applicable”; selection of all applicable items from a list with options to specify alternatives via text boxes; and rating the participant's agreement with statements using a Likert scale of five or ten ordered response levels (e.g., strongly agree, agree, neither agree nor disagree, disagree, and strongly disagree). The survey was administered through REDCap™ (Research Electronic Data Capture) tools hosted by our institution. Owing to its secure methods of data capture and streamlined exporting schema, REDCap™ is a web-based software platform designed specifically for use in research studies [25,26].
2.2. Study design
A list of radiology departmental contacts consisting of 388 unique email addresses from 144 institutions was compiled from all 155 American Medical College Application Services (AMCAS) participating and non-participating American medical schools. The 17 Canadian medical institutions were also initially included [27,28]. Potential departmental contacts included departmental chairs, program directors, associate program directors, and assistant program directors as identified from the faculties' web pages. When those contacts were unavailable, staff or faculty radiologists in the department were contacted instead. Departmental contacts for 28 institutions were not available despite an extensive internet search. Email addresses of 69 academic radiology administrative contacts maintained by our institution and contacts from a database compiled for a previous study were also used [29], yielding a final distribution list of 1521 unique email addresses. Invitations were additionally sent by the American Society of Emergency Radiologists (ASER) and the Association of University Radiologists (AUR) to their respective member listservs. Given the urgency of the data collection, we hoped that supplementing our initial contact list with societal listservs would allow us to reach the widest distribution of radiologists throughout the country as quickly as possible. As participants were encouraged to invite their peers and trainees to expand the outreach, we are unable to report a total number of potential respondents.
All invitations were sent with an explanation of the study and a generic link created by REDCap™ system. In an effort to protect the participant anonymity, survey results were maintained without personal identifiers on internal servers and the principal investigator's password-protected computer. As we made no efforts to track participation, including IP address tracking, monitoring email inbox activities, acquiring read receipts, and monitoring browser activities, we were similarly unable to screen for duplicate submissions from the same individual. Participants were encouraged to take the survey in a location and time of their choosing, and to maintain a private and discrete environment during administration. Results were collected over a period of 8 days starting on April 3, 2020.
2.3. Data analysis
Descriptive analysis included mean (SD) for age, percent distribution of response options, independent t-tests for continuous measurements, and Chi-squared tests for categorical data. A hierarchical multivariate Poisson regression model with Generalized Estimating Equation (GEE) was used to identify the factors associated with anxiety and account for geographic clustering from overrepresented states. Considering the large sample size of the study, variables with a p-value of less than 0.05 in the univariate analysis were selected for subsequent inclusion in the multivariate model. Analyses were controlled for the standardized count of COVID-19 cases per unit population in a respondent's state of practice on the day of survey submission [30]. Median split was used to dichotomize the anxiety output. To minimize any risk of bias due to skewed geographic distributions and varying population densities between states, these factors were adjusted for as potential confounders in the hierarchical multivariate analysis. Statistical analyses and hierarchical modeling were conducted using SAS software version 9.4 (SAS Institute Inc., Cary, NC, USA).
3. Results
3.1. Characteristics of the participants
A total of 689 respondents (mean age = 45 ± 11 years, 53% men) from 44 states were included in the analysis. Respondents who did not complete any questions beyond the demographics portion of the survey were excluded. Although respondents from Canada were initially considered for inclusion, these responses were ultimately excluded from the analysis due to a relatively small number of participants. The demographic and professional characteristics of the respondents included for analysis are outlined in Table 1 , and the distribution of respondents across the US is depicted in Fig. 1 . Briefly, the majority of respondents were attending radiologists (80%), worked in academic hospitals (73%), and most frequently reported ‘general radiology’ as their field of practice (38%). Over 85% of the respondents reported spending more than half of their work-related time performing clinical duties, though direct patient contact was reportedly minimal. The question asking about the participant's proportion of work related to research had the lowest response rate (84%). Response rates of other survey questions ranged between 86 and 100%. Below we present a subset of our survey results; full results are available in the supplemental tables accompanying this manuscript (Suppl 1–5).
Table 1.
Characteristics | n = 689 |
---|---|
Age (years)⁎ | 45 ± 11 |
Level of training | |
In training (resident) | 110/689 (16%) |
In training (fellow) | 25/689 (4%) |
Radiologist/attending radiologist | 551/689 (80%) |
Other | 3/689 (0.4%) |
Gender | |
Male | 365/688 (53%) |
Female | 317/688 (47%) |
Nonbinary | 1/688 (0.2%) |
Prefer not to say | 5/688 (0.7%) |
Main areas of practice⁎⁎ | |
General radiology | 263/689 (38%) |
Abdominal radiology | 157/689 (23%) |
Cardiothoracic radiology | 63/689 (9%) |
Vascular and interventional radiology | 89/689 (13%) |
Musculoskeletal radiology | 92/689 (13%) |
Emergency radiology | 137/689 (20%) |
Neuroradiology | 100/689 (15%) |
Nuclear medicine | 43/689 (6%) |
Women's imaging | 113/689 (16%) |
Pediatric radiology | 53/689 (8%) |
Other | 19/689 (3%) |
Main types of practice environment⁎⁎ | |
Academic hospital | 505/689 (73%) |
Private | 175/689 (25%) |
Public hospital (including veteran affairs and county) | 100/689 (15%) |
Other | 17/689 (3%) |
Main settings of practice⁎⁎ | |
Outpatient | 499/689 (72%) |
Inpatient | 519/689 (75%) |
Emergency | 384/689 (56%) |
Teleradiology | 82/689 (12%) |
Other | 5/689 (0.7%) |
Distribution of time spent on professional activities | |
Clinical duties | |
0–25% | 42/681 (6%) |
26–50% | 58/681 (9%) |
51–75% | 170/681 (25%) |
>75% | 411/681 (60%) |
Direct patient contact | |
0–25% | 415/603 (69%) |
26–50% | 99/603 (16%) |
51–75% | 33/603 (5%) |
>75% | 56/603 (9%) |
Education (teaching) | |
0–25% | 467/618 (76%) |
26–50% | 120/618 (19%) |
51–75% | 23/618 (4%) |
>75% | 8/618 (1%) |
Research | |
0–25% | 529/579 (91%) |
26–50% | 44/579 (8%) |
51–75% | 4/579 (1%) |
>75% | 2/579 (0.4%) |
Note: all data are presented as numerators and denominators with percentages in parentheses unless otherwise specified.
Reported as mean ± standard deviation.
These questions gave respondents the option to ‘select all that apply’.
3.2. Broad impact of COVID-19
Almost 43% of respondents answered either “no” or “I don't know” when asked if they had easy access to COVID-19 testing. Half (50%) of respondents felt the media portrayal of the outbreak was accurate; however, as many as 32% of respondents felt the media response was exaggerated/excessive. The vast majority of respondents (94%) were under a government-ordered shelter-at-home mandate, with 90% and 91% reporting shut down of nonessential business and educational establishments, respectively.
3.3. Preparedness for COVID-19
On an institutional level, 76% of respondents reported that their medical center was adequately prepared for surge potential, but only 40% reported that their institution had a mass casualty imaging plan in place. Nearly a quarter of respondents (23%) reported that their medical center did not have adequate strategies in place to control the viral spread, and only 60% reported an increase in utilization of portable imaging; however, only 65% of respondents indicated that they had adequate teleradiology support, with even fewer reporting that their medical center had adequate PPE for staff (51%) and patients (46%).
In keeping with national trends [31], less than half of respondents had ready access to N95 respirator masks (43%) or face shields/eye protection (46%), though surgical masks (78%) and gloves (71%) were in good supply. The majority of institutions had put into place social mitigation measures for staff, including restrictions on domestic travel (60%), discontinuation of in-person academic activities (57%), provisions for isolated and/or single-station reading rooms (60%), and mandated self-quarantine after travel abroad (60%). Even more respondents (80%) reported that their institution was holding all meetings and conferences over video and telephone, with 73% reporting that non-essential staff were allowed to work remotely. For essential radiology staff required to remain onsite, 60% of respondents reported that their medical facility had provided for isolated/single station reading rooms.
3.4. Clinical duties and workload
The most common primary diagnostic approaches to PUI for COVID-19 were chest X-ray (CXR) (46%) and RT-PCR (43%), though 16% of respondents were not aware of their medical center's primary screening modality. On a personal level, the majority of respondents (86%) felt they had adequate knowledge for interpreting COVID-19 related images. Nearly all (99%) participants reported a decrease in workload, with 60% noting greater than 50% reduction. As many as 59% of respondents reported that their medical center was no longer performing non-emergent imaging or procedures. Reasons for cancelations of non-emergent imaging and procedures varied widely, with the most frequently cited reasons being medical center guidelines (68%) and cancelation by patients (61%). However, despite limited availability of PPE, 78% of participants reported that they were being asked to or already were performing duties outside of their usual scope, such as supplementing physicians in the intensive care unit or emergency department, “handing out PPE to employees”, and being “pulled to chest/ER radiology”.
3.5. Leadership
Most respondents (84%) felt that the response from their medical center leadership was acceptable/appropriate; however 15% felt the response from their leadership was inadequate. Yet, the majority (72%) felt that their medical center leadership was providing them with adequately frequent updates with respect to the pandemic.
3.6. Research and education
Of the respondents who were currently engaged in research (55%), 64% reported either a decrease or outright halt of their research activities. Among those who were engaged in teaching radiology trainees (60%), 92% felt that the effects of the COVID-19 pandemic are negatively affecting educational experiences. These sentiments were in agreement with the 87% of current fellows and residents who similarly reported that they felt their training was being negatively impacted.
3.7. Financial impact
Most participants anticipated or had already experienced a decrease in overall income (56%), though 60% anticipated less than 25% change. A small proportion (11%) reported experiencing partial or complete layoff due to the outbreak. One third of respondents (33%) anticipated that it would take more than two months to return to normal capacity after transitioning back to usual business routines.
3.8. Psychological impact
When asked to rate their level of anxiety regarding the COVID-19 outbreak on a scale of 1 to 10, participants self-reported an average score of 6.71 (SD ± 2.03). Notably, over half (61%) respondents rated their anxiety levels to be a 7 or higher. The most commonly cited stressors were family health (71%), personal health (47%), and financial concerns (33%). Family time (59%) and exercising (57%) were the most frequently cited coping strategies, followed by watching television (44%), reading (34%), and talking to friends (33%). When asked to select the top 3 ways in which they were currently spending most their time, 63% of respondents reported being with family, however 59% and 40% selected “clinical work” and “keeping up with COVID-19 news”, respectively. Notably, 9% of respondents denied needing any coping methods.
From our univariate analysis of factors influencing anxiety scores, we found that 29% of variables reached statistical significance at the 0.05 level. These variables were subsequently selected for inclusion in the multivariate model. Using a median anxiety score of 7 as the split value, anxiety scores were dichotomized prior to modeling. Our multivariate analysis of the factors associated with radiologists' level of anxiety in relation to COVID-19 pandemic demonstrated that, after controlling for potential risk factors, the stressor of “personal health” was strongly correlated with higher anxiety scores (RR 1.23; 95% CI: 1.13–1.34, p < 0.01). Other statistically significant contributors to anxiety included the stressor “ability of my hospital/department to manage the outbreak” (RR 1.1; 95% CI 1.03–1.18, p < 0.01) and the standardized count of COVID-19 positive people in respondents' state of practice (RR 1.11, 95% CI: 1.02–1.21, p = 0.01). Conversely, participants who answered “no coping needed” (RR 0.4; 95% CI: 0.3–0.53, p < 0.01), as well as those who endorsed the stressor “suspension of nonessential activity” (RR 0.82; 95% CI: 0.72–0.94, p < 0.01) were less likely to report higher levels of anxiety, suggestive of a strong protective effect (Table 2 ).
Table 2.
Univariate1 |
Univariate (unadjusted)2 |
Multivariate (adjusted)3 |
|||||
---|---|---|---|---|---|---|---|
Sig. | RR Ratio | Confidence Interval | Sig. | RR Ratio | Confidence Interval | Sig. | |
Methods of coping with stress and anxiety related to the outbreak (selected vs not selected) | |||||||
No coping needed | <0.01 | 0.35 | 0.28–0.44 | <0.01 | 0.4 | 0.3–0.53 | <0.01⁎ |
Talking with friends | <0.01 | 1.19 | 1.1–1.29 | <0.01 | 1.02 | 0.93–1.12 | 0.62 |
Spending time with family | <0.01 | 1.25 | 1.13–1.39 | <0.01 | 1.04 | 0.92–1.18 | 0.51 |
Television | <0.01 | 1.14 | 1.01–1.28 | 0.03 | 0.98 | 0.89–1.08 | 0.69 |
Anticipated effect on clinical practice 1 year from now (selected vs not selected) | |||||||
No change | <0.01 | 0.83 | 0.71–0.97 | 0.02 | 0.96 | 0.84–1.1 | 0.58 |
Increased usage of PPE | 0.01 | 1.13 | 1.01–1.26 | 0.03 | 1.04 | 0.96–1.12 | 0.35 |
Have patients reschedule if they feel sick | 0.04 | 1.13 | 1.02–1.24 | 0.02 | 1.09 | 1–1.18 | 0.06 |
Perception of your medical center's ability to handle future public health concerns | |||||||
Negatively impacted vs no impact | <0.01 | 1.27 | 1.17–1.38 | <0.01 | 1.06 | 0.96–1.17 | 0.26 |
Positively impacted vs no impact | 1.02 | 0.92–1.13 | 0.68 | 1.06 | 0.97–1.15 | 0.19 | |
Gender | |||||||
Female vs male | <0.01 | 1.23 | 1.08–1.4 | <0.01 | 1.11 | 1.01–1.23 | 0.04 |
Likert scale: I feel adequately equipped with knowledge for interpreting COVID-19 imaging (1 = strongly disagree, 5 = strongly agree) | |||||||
2 vs 1 | <0.01 | 1.14 | 0.85–1.53 | 0.4 | 1.3 | 1.05–1.62 | 0.02 |
3 vs 1 | 1.01 | 0.78–1.32 | 0.91 | 1.22 | 0.96–1.53 | 0.1 | |
4 vs 1 | 0.97 | 0.7–1.36 | 0.88 | 1.26 | 0.96–1.66 | 0.09 | |
5 vs 1 | 0.84 | 0.63–1.11 | 0.21 | 1.11 | 0.86–1.44 | 0.41 | |
My medical center has adequate personal protective equipment for patients (yes, no, I don't know) | |||||||
I don't know vs no | 0.01 | 0.88 | 0.79–0.97 | 0.01 | 0.96 | 0.85–1.09 | 0.52 |
Yes vs no | 0.8 | 0.73–0.89 | <0.01 | 0.9 | 0.81–0.99 | 0.04 | |
My medical center has adequate personal protective equipment for staff (Yes, No, I don't know) | |||||||
I don't know vs no | 0.02 | 0.82 | 0.71–0.96 | 0.01 | 0.86 | 0.7–1.06 | 0.16 |
Yes vs no | 0.83 | 0.77–0.9 | <0.01 | 1.01 | 0.9–1.14 | 0.83 | |
Main setting of practice (selected vs not selected) | |||||||
Teleradiology | <0.01 | 0.76 | 0.62–0.94 | <0.01 | 0.87 | 0.73–1.04 | 0.13 |
Likert scale: My medical center has adequate strategies in place to control the spread of COVID-19 (1 = strongly disagree, 5 = strongly agree) | |||||||
2 vs 1 | <0.01 | 1.02 | 0.85–1.21 | 0.87 | 1.04 | 0.91–1.18 | 0.57 |
3 vs 1 | 0.91 | 0.79–1.06 | 0.22 | 1.01 | 0.85–1.21 | 0.89 | |
4 vs 1 | 0.82 | 0.68–0.98 | 0.03 | 0.95 | 0.79–1.15 | 0.6 | |
5 vs 1 | 0.71 | 0.59–0.85 | <0.01 | 0.95 | 0.77–1.18 | 0.65 | |
Top 3 stressors in relation to the outbreak (selected vs not selected) | |||||||
Personal health | <0.01 | 0.48 | 0.38–0.59 | <0.01 | 1.23 | 1.13–1.34 | <0.01⁎ |
Family health | <0.01 | 0.59 | 0.49–0.72 | <0.01 | 1.06 | 0.96–1.17 | 0.27 |
Ability of my hospital/department to manage the outbreak | <0.01 | 1.19 | 1.1–1.29 | <0.01 | 1.1 | 1.03–1.18 | <0.01⁎ |
Suspension of non-essential activities | <0.01 | 0.73 | 0.62–0.85 | <0.01 | 0.82 | 0.72–0.94 | <0.01⁎ |
Clinical work directly related to COVID-19 | 0.03 | 1.14 | 1.04–1.24 | <0.01 | 1.02 | 0.95–1.09 | 0.59 |
Likert scale: My medical center is prepared for surge potential and increased imaging demand (1 = strongly disagree, 5 = strongly agree) | |||||||
2 vs 1 | <0.01 | 0.9 | 0.75–1.1 | 0.3 | 1.01 | 0.87–1.16 | 0.91 |
3 vs 1 | 0.91 | 0.79–1.06 | 0.22 | 1.05 | 0.92–1.2 | 0.44 | |
4 vs 1 | 0.82 | 0.69–0.96 | 0.02 | 1.04 | 0.87–1.24 | 0.67 | |
5 vs 1 | 0.69 | 0.59–0.8 | <0.01 | 0.96 | 0.77–1.19 | 0.69 | |
Likert scale: My medical center has successfully implemented teleradiology (1 = strongly disagree, 5 = strongly agree) | |||||||
2 vs 1 | <0.01 | 1.09 | 0.94–1.28 | 0.25 | 1.11 | 0.99–1.25 | 0.08 |
3 vs 1 | 0.93 | 0.8–1.09 | 0.37 | 1.02 | 0.92–1.13 | 0.66 | |
4 vs 1 | 1.02 | 0.9–1.17 | 0.72 | 1.08 | 0.97–1.21 | 0.16 | |
5 vs 1 | 0.85 | 0.73–0.99 | 0.03 | 1.06 | 0.95–1.18 | 0.33 | |
Do you have easy access to COVID-19 testing? | |||||||
I don't know vs no | 0.01 | 1.02 | 0.91–1.13 | 0.76 | 1.07 | 0.99–1.15 | 0.1 |
Yes vs no | 0.89 | 0.8–1 | 0.06 | 0.95 | 0.86–1.04 | 0.24 | |
COVID-19 cumulative cases by State | |||||||
Standardized COVID-19 count/100 | 1.15 | 1.07–1.23 | <0.01 | 1.11 | 1.02–1.21 | 0.01 |
These covariates were significant using a cut-off p-value of <0.01.
P value from descriptive statistics without considering the hierarchical data structure of radiologists clustered by state.
Unadjusted: from univariate hierarchical model.
Adjusted: from multivariate hierarchical model.
When we recreated the hierarchical modeling for subset analysis using only those respondents who selected “private” for their main setting of practice (n = 139), we observed a similar pattern of beta coefficients as seen in the model constructed from the full dataset. When we examined the pattern of the β coefficients between those respondents who selected “resident” (n = 89) or “fellow” (n = 22) as compared to those who selected “radiologist/attending radiologist” for their level of training, only the following 3 questionnaire items demonstrated large differences between groups: “my medical center has successfully implemented teleradiology”; “perception of your medical center's ability to handle future public health concerns”; and a response of “no coping needed” when asked to identify “methods of coping with stress and anxiety related to the outbreak”. For residents and fellows, successful implementation of teleradiology offered a much stronger protective effect (RR 0.53; 95% CI: 0.33–0.85, p < 0.01) than for attending radiologists (RR 0.93; 95% CI: 0.82–1.06, p = 0.27), with an interaction effect of p = 0.02. Direct comparison of relative risks suggest that altered perception of one's medical center's ability to handle future outbreaks – both positively or negatively – generated more anxiety for residents and fellows (positively impacted – RR 1.25; 95% CI 1.02–1.53, p = 0.03; negatively impacted – RR 1.32; 95% CI 1.15–1.51, p < 0.01) as compared to attending radiologists (positively impacted – RR 0.95; 95% CI 0.87–1.04, p = 0.28; negatively impacted – RR 1.1; 95% CI 1–1.22, p = 0.05). However, neither response generated a significant interaction effect (p = 0.10). Conversely, our results suggest that a response of “no coping needed” was potentially less protective for residents and fellows (RR 0.53; 95% CI 0.39–0.72, p < 0.01) as compared to attending radiologists (RR 0.32; 95% CI 0.23–0.44, p < 0.01), although the interaction effect was again non-significant (p = 0.10).
3.9. Perceived future impact
Over 77% of respondents felt that their perception of their medical center's ability to handle future public health concerns was either unchanged or positively impacted in light of recent responses to COVID-19. The most frequently cited anticipated effects on future clinical practice included heightened awareness of infection control (62%), increased remote work options (52%), and increased use of telecommunication (49%).
4. Discussion
Our findings from this nationwide survey adapted from the AO Spine Foundation's survey suggest that the COVID-19 pandemic has profoundly affected radiologists across the US [24]. While facing many challenges at home due to government mandated restrictions to control the spread of virus, radiologists also face a daunting number of challenges related to their clinical work, research, medical education, and administrative duties. We found that the majority of respondents across many practice settings reported significant changes to the nature and volume of their workload, as well as heightened levels of anxiety related to the pandemic. Although our sample was composed mostly of academic radiologists, the fact that we mostly observed similar patterns of β-change in our private practice and residents/fellows subpopulations suggests that our results do hold some degree of generalizability across practice settings and levels of training. Only the question regarding successful implementation of teleradiology demonstrated a significant interaction effect between residents/fellows and attending physicians. While perhaps somewhat related to life station and career stage, the exact reasoning behind this difference remains under investigation.
In keeping with similar findings in related healthcare fields, our data support that radiologists are also experiencing high levels of anxiety and adverse psychological effects as a result of the COVID-19 pandemic. Though the average self-reported anxiety score in our sample population was only 6.71, the fact that 61% of respondents reported scores of 7 out of 10 or greater suggests that a significant subset is suffering disproportionately from severe levels of anxiety. While it is expected that frontline workers may experience adverse psychological effects given the level of risk to which they are exposed [22,23], recent literature also suggests that non-frontline healthcare workers are similarly at an increased risk of anxiety-related sequelae secondary to so-called “vicarious traumatizations” [20,21]. This idea is highlighted by one of our respondents who noted, “radiology … isn't one of the ‘front line’ fields so there is a lot of mixed feelings in terms of potential exposure at work … guilt as well as relief”, and further supported by our findings that anxiety scores were positively correlated with the standardized count of COVID-19 cases in a respondent's state of practice. Moreover, citing the stressor of “personal health” was a strong predictor of high anxiety levels, which is consistent with related research suggesting that non-frontline workers are more likely to be worried about becoming infected than their frontline counterparts [20]. Our results additionally support existing literature which suggest that coherent institutional leadership and protocols serve to reduce psychological stress [23]. Yet, for unclear reasons, citing the stressor “my family's health” was not a statistically significant predictor of anxiety in the multivariate model. This is at odds with other published data suggesting that concerns for family health do, in fact, drive increased anxiety scores in related healthcare professions [20,23,24].
To cope with anxiety and stress, radiologists most frequently reported spending time with family (63%) and exercising (57%). However, several also left comments noting that they were “too busy with work” or that the “number of Zoom meetings has exploded” such that they did not have time to engage in self-care. Curiously, reports of “no coping needed” with regard to pandemic-associated stress were protective against high levels of anxiety. While it is possible that these respondents may be influenced by positive factors not otherwise captured by our questionnaire, it is also plausible that individuals who feel compelled to deny needing any coping strategies may concordantly be more likely to deny feeling any anxiety at all. Further study is needed to determine if this truly represents a spectrum of individuals with more developed “wellness toolkits”, or whether these responses are in fact indicative of more maladaptive coping strategies [32]. While related literature has found that factors such as seeking support from friends and family and adequate hospital supply of PPE may potentially have protective psychosocial effects [23], these factors notably did not fall out as statistically significant in our model.
Ostensibly, though 99% the survey respondents reported decreased caseloads, several noted in the open-ended comments that they had experienced increases in other types of work, including administrative duties, developing online curricula for residents, and childcare responsibilities. These findings, though unanticipated in the original design of our survey, nevertheless highlight some of the many important areas in which additional support is needed. Of particular note, managing unexpected childcare obligations among healthcare workers in the wake of school closures remains especially challenging [33,34]. As many as 29% of US healthcare workers require childcare assistance for children aged 3–12 years. Even when accounting for support from older children and non-working adults, this number remains as high as 15%, representing one of the highest childcare obligation burdens of all US workforce sectors [33]. Thus, although 65% of respondents agreed that their medical center had successfully implemented teleradiology, it is likely that a significant portion of these respondents are unable to fully perform their roles remotely as a result of attending to increased childcare needs [33,34]. Though comprehensive institutional databases and medical student volunteers have provided temporary stopgaps [35,36], there remain significant gaps in the amount of support available for many clinicians.
Due to the evolving role of imaging in the triage and management of COVID-19 patients, radiologists have played an integral role in the healthcare response to the viral outbreak. Earlier in the course of the pandemic, the use of chest CT as a first-line screening and triage modality showed promise [37,38,44]; however, concerns regarding poor specificity of CT findings for COVID-19-associated pneumonia have prompted several professional societies and expert panels to issue recommendations to the contrary [13,14,39,40]. Though limited endorsement for first-line use of chest CT remains primarily restricted to medical triage settings, CXR is generally thought to be less revealing in early disease owing to its poor ability to resolve characteristic ground glass opacities [3,4,8,40,44]. Yet surprisingly, almost half of survey respondents reported that CXRs were part of their medical center's primary diagnostic approach to PUI for COVID-19. It is possible these protocols may be driven by the need for rapid triage decision-making in the emergency care setting, as testing capacities and wait times for laboratory confirmation were inconsistent in many centers across the US [41,42]. The small yet notable number of survey respondents who did not feel adequately prepared to interpret COVID-19 imaging as well as the percentage of those who were unaware of their center's diagnostic approach to a PUI cases are likely also related to this ongoing dilemma, whereby many institutions have been forced to construct their own internal protocols based on local need [8,[43], [44], [45]]. Although interim clinical guidelines for COVID-19 are currently in place in many medical centers, regional variations in practices and uncertainty regarding the role of imaging in COVID-19 will continue to be an issue that radiologists will face as we wait for more conclusive evidence to become available.
Limitations to this study include the sample population being largely composed of radiologists from academic institutions given the difficulty in establishing contact with smaller private and community practices. Other limitations include the cross-sectional nature of the survey, as responses will likely vary significantly at different points during the pandemic. Though we attempted to control for the number of COVID-19 cases per unit population for each day of survey administration, these measures are at best imprecise and subject to regional testing capacities and reporting. Furthermore, while there were several additional areas of interest related to the impact of COVID-19, our survey was designed to be brief and may lack relevant details. For example, while our questionnaire asked specifically after methods of coping in response to the pandemic, we acknowledge that many individuals may have had in place pre-existing coping mechanisms for anxiety not directly related to the pandemic. These granular details are thus not fully resolved in our dataset. Finally, we do recognize a possibility of sampling bias such that the voluntary nature of our design runs the risk of self-selecting for those respondents with particularly strong inclinations. Future directions include follow-up studies to assess for changes over time as the virus continues to spread preferentially among certain geographic “hot spots”. We also hope to include additional demographic factors as possible correlates in future analyses, such as age, race, and political affiliation.
5. Conclusion
Though this study offers a snapshot in time, the very nature of the pandemic is incredibly dynamic. The challenges faced by US radiologists will undoubtedly continue to evolve as the numbers of COVID-19 cases continue to rise. Notable areas of future research include investigations into institutional policies surrounding the use of imaging in COVID-19, economic impacts as practices continue to adjust to decreased revenues, and provisions to support radiologists working remotely in times of crisis and demand.
Grants received
None.
Disclosures
None.
Summary statement
Radiologists nationwide are faced with extraordinary stressors and unprecedented shifts in clinical workflows in the wake of the COVID-19 pandemic.
CRediT authorship contribution statement
Natalie L. Demirjian: Investigation, Writing - original draft, Writing - review & editing, Data curation. Brandon K.K. Fields: Investigation, Writing - original draft, Writing - review & editing, Data curation. Catherine Song: Writing - original draft, Writing - review & editing, Data curation. Sravanthi Reddy: Data curation, Writing - review & editing, Investigation. Bhushan Desai: Investigation, Data curation, Writing - review & editing. Steven Y. Cen: Formal analysis, Writing - review & editing. Sana Salehi: Data curation, Writing - review & editing. Ali Gholamrezanezhad: Data curation, Writing - original draft, Writing - review & editing, Conceptualization, Investigation.
Declaration of competing interest
The authors declare that they have no conflicts of interest.
Acknowledgments
We thank AO Spine for sharing their survey with us. AO Spine is a clinical division of the AO Foundation, which is an independent, medically-guided not-for-profit organization. We also thank Xiaomeng Lei, MPH for her assistance with the statistical analysis and datasheet restructuring.
Footnotes
Supplementary data to this article can be found online at https://doi.org/10.1016/j.clinimag.2020.08.027.
Appendix A. Supplementary data
References
- 1.WHO . World Health Organization website; 2020. Director-General's opening remarks at the media briefing on COVID-19 — 11 March 2020.https://www.who.int/dg/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19---11-march-2020 Published March 11. Accessed March 29, 2020. [Google Scholar]
- 2.Cases in the U.S. Centers for Disease Control and Prevention website. https://www.cdc.gov/coronavirus/2019-ncov/cases-updates/cases-in-us.html. Updated June 1, 2020. Accessed June 1, 2020.
- 3.Salehi S., Abedi A., Balakrishnan S., Gholamrezanezhad A. Coronavirus Disease 2019 (COVID-19): a systematic review of imaging findings in 919 patients. AJR Am J Roentgenol. 2020;215(1):87–93. doi: 10.2214/AJR.20.23034. [DOI] [PubMed] [Google Scholar]
- 4.Hosseiny M., Kooraki S., Gholamrezanezhad A., Reddy S., Myers L. Radiology perspective of Coronavirus Disease 2019 (COVID-19): lessons from Severe Acute Respiratory Syndrome and Middle East Respiratory Syndrome. AJR Am J Roentgenol. 2020;214(5):1078–1082. doi: 10.2214/AJR.20.22969. [DOI] [PubMed] [Google Scholar]
- 5.Kooraki S., Hosseiny M., Myers L., Gholamrezanezhad A. Coronavirus (COVID-19) outbreak: what the Department of Radiology should know. J Am Coll Radiol. 2020;17(4):447–451. doi: 10.1016/j.jacr.2020.02.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Demirjian N.L., Fields B.K.K., Gholamrezanezhad A. Role of chest CT in resource-driven healthcare systems. AJR Am J Roentgenol. 2020:W1. doi: 10.2214/ajr.20.23498. [DOI] [PubMed] [Google Scholar]
- 7.ACR recommendations for the use of chest radiography and computed tomography (CT) for suspected COVID-19 infection. American College of Radiology website; 2020. https://www.acr.org/Advocacy-and-Economics/ACR-Position-Statements/Recommendations-for-Chest-Radiography-and-CT-for-Suspected-COVID19-Infection Published March 11. Updated March 22, 2020. Accessed April 8, 2020. [Google Scholar]
- 8.Rubin G.D., Ryerson C.J., Haramati L.B., Sverzellati N., Kanne J.P., Raoof S. The role of chest imaging in patient management during the COVID-19 pandemic: a multinational consensus statement from the Fleischner Society. Radiology. 2020;296(1):172–180. doi: 10.1148/radiol.2020201365. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Revel M.P., Parkar A.P., Prosch H., Silva M., Sverzellati N., Gleeson F. COVID-19 patients and the radiology department — advice from the European Society of Radiology (ESR) and the European Society of Thoracic Imaging (ESTI) Eur Radiol. 2020:1–7. doi: 10.1007/s00330-020-06865-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Simpson S., Kay F.U., Abbara S., Bhalla S., Chung J.H., Chung M. Radiological Society of North America expert consensus statement on reporting chest CT findings related to COVID-19. Endorsed by the Society of Thoracic Radiology, the American College of Radiology, and RSNA. J Thorac Imaging. 2020;2(2):e200152. doi: 10.1097/RTI.0000000000000524. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Nakajima K., Kato H., Yamashiro T., Izumi T., Takeuchi I., Nakajima H. COVID-19 pneumonia: infection control protocol inside computed tomography suites. Jpn J Radiol. 2020 doi: 10.1007/s11604-020-00948-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Azam S.A., Myers L., Fields B.K.K., Demirjian N.L., Patel D., Roberge E. Coronavirus disease 2019 (COVID-19) pandemic: review of guidelines for resuming non-urgent imaging and procedures in radiology during Phase II. Clin Imaging. 2020;67:30–36. doi: 10.1016/j.clinimag.2020.05.032. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Mossa-Basha M., Meltzer C.C., Kim D.C., Tuite M.J., Kolli K.P., Tan B.S. Radiology department preparedness for COVID-19: radiology scientific expert panel. Radiology. 2020:200988. doi: 10.1148/radiol.2020200988. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Mossa-Basha M., Medverd J., Linnau K., Lynch J.B., Wener M.H., Kicska G. Policies and guidelines for COVID-19 preparedness: experiences from the University of Washington. Radiology. 2020 doi: 10.1148/radiol.2020201326. [DOI] [PubMed] [Google Scholar]
- 15.Stempniak M. Radiology Business website; 2020. Radiologist predicts “sea change” in specialty after COVID-19, with many docs reading from home.https://www.radiologybusiness.com/topics/care-delivery/teleradiology-coronavirus-covid-19-pandemic-work-home Published March 31. [Google Scholar]
- 16.Stempniak M. Coronavirus may provide “wake-up call” to let more radiologists work from home. Radiology business. Radiology Business Website. https://www.radiologybusiness.com/topics/policy/coronavirus-radiologists-work-home-teleradiology. Published March 16, 202. Accessed April 8, 2020.
- 17.Lee A.M., Wong J.G., McAlonan G.M., Cheung V., Cheung C., Sham P.C. Stress and psychological distress among SARS survivors 1 year after the outbreak. Can J Psychiatry. 2007;52(4):233–240. doi: 10.1177/070674370705200405. [DOI] [PubMed] [Google Scholar]
- 18.Maunder R., Hunter J., Vincent L., Bennett J., Peladeau N., Leszcz M. The immediate psychological and occupational impact of the 2003 SARS outbreak in a teaching hospital. CMAJ. 2003;168(10):1245–1251. [PMC free article] [PubMed] [Google Scholar]
- 19.Wu P., Fang Y., Guan Z., Fan B., Kong J., Yao Z. The psychological impact of the SARS epidemic on hospital employees in China: exposure, risk perception, and altruistic acceptance of risk. Can J Psychiatry. 2009;54(5):302–311. doi: 10.1177/070674370905400504. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Wu Y., Wang J., Luo C., Hu S., Lin X., Anderson A.E. A comparison of burnout frequency among oncology physicians and nurses working on the frontline and usual wards during the COVID-19 epidemic in Wuhan, China. J Pain Symptom Manage. 2020;60(1):e60–e65. doi: 10.1016/j.jpainsymman.2020.04.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Li Z., Ge J., Yang M., Feng J., Qiao M., Jiang R. Vicarious traumatization in the general public, members, and non-members of medical teams aiding in COVID-19 control. Brain Behav Immun. 2020:1–4. doi: 10.1016/j.bbi.2020.03.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Lai J., Ma S., Wang Y., Cai Z., Hu J., Wei N. Factors associated with mental health outcomes among health care workers exposed to Coronavirus Disease 2019. JAMA Netw Open. 2020;3(3) doi: 10.1001/jamanetworkopen.2020.3976. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Cai H., Tu B., Ma J., Chen L., Fu L., Jiang Y. Psychological impact and coping strategies of frontline medical staff in Hunan between January and March 2020 during the outbreak of Coronavirus Disease 2019 (COVID19) in Hubei, China. Med Sci Monit. 2020;26:e924171. doi: 10.12659/MSM.924171. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Louie P.K., Harada G.K., McCarthy M.H., Germscheid N., Cheung J.P.Y., Neva M.H. The impact of COVID-19 pandemic on spine surgeons worldwide. Global Spine J. 2020;10(5):534–552. doi: 10.1177/2192568220925783. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Harris P.A., Taylor R., Minor B.L., Elliott V., Fernandez M., O’Neal L. The REDCap consortium: building an international community of software platform partners. J Biomed Inform. 2019;95:103208. doi: 10.1016/j.jbi.2019.103208. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Harris P.A., Taylor R., Thielke R., Payne J., Gonzalez N., Conde J.G. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–381. doi: 10.1016/j.jbi.2008.08.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Participating medical schools and deadlines. Association of American Medical Colleges website. https://students-residents.aamc.org/applying-medical-school/article/participating-medical-schools-deadlines/. Accessed April 8, 2020.
- 28.Netrakanti V. Medical schools in Canada: Comprehensive list of medical schools in Canada. BeMo website. https://bemoacademicconsulting.com/blog/list-of-medical-schools-in-canada. Accessed April 8, 2020.
- 29.Tofighi S., Abedi A., Salehi S., Myers L., Reddy S., Gholamrezanezhad A. Reason for exam imaging reporting and data system: consensus reached on quality assessment of radiology requisitions. J Patient Saf. 2020:1–7. doi: 10.1097/PTS.0000000000000653. [DOI] [PubMed] [Google Scholar]
- 30.Novel coronavirus (COVID-19) cases, provided by JHU CSSE. Johns Hopkins University Center for Systems Science and Engineering; 2020. https://github.com/CSSEGISandData/COVID-19 Updated April 13, 2020. Accessed April 13, 2020. [Google Scholar]
- 31.Vankin D. Los Angeles Times website; 2020. From Julia Louis-Dreyfus’ house to N95-like masks: Architects join the COVID-19 fight.https://www.latimes.com/entertainment-arts/story/2020-04-07/3d-printers-masks-ppe-for-medical-workers-usc-architects Published April 7. Accessed May 5, 2020. [Google Scholar]
- 32.Mojtabai R., Olfson M., Sampson N.A., Jin R., Druss B., Wang P.S. Barriers to mental health treatment: results from the National Comorbidity Survey Replication. Psychol Med. 2011;41(8):1751–1761. doi: 10.1017/S0033291710002291. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Bayham J., Fenichel E.P. Impact of school closures for COVID-19 on the US health-care workforce and net mortality: a modelling study. Lancet Public Health. 2020;5(5):e271–e278. doi: 10.1016/S2468-2667(20)30082-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Dewey C., Hingle S., Goelz E., Linzer M. Supporting clinicians during the COVID-19 pandemic. Ann Intern Med. 2020;172(11):752–753. doi: 10.7326/M20-1033. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Vagal A., Reeder S.B., Sodickson D.K., Goh V., Bhujwalla Z.M., Krupinski E.A. The impact of the COVID-19 pandemic on the radiology research enterprise: radiology scientific expert panel. Radiology. 2020:201393. doi: 10.1148/radiol.2020201393. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.5 ways medical students can assist during the COVID-19 pandemic. American Medical Association website; 2020. https://www.ama-assn.org/delivering-care/public-health/5-ways-medical-students-can-assist-during-covid-19-pandemic Published March 25. Accessed May 4, 2020. [Google Scholar]
- 37.Fang Y., Zhang H., Xie J., Lin M., Ying L., Pang P. Sensitivity of chest CT for COVID-19: comparison to RT-PCR. Radiology. 2020;296(2):E115–E117. doi: 10.1148/radiol.2020200432. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Ai T., Yang Z., Hou H., Zhan C., Chen C., Lv W. Correlation of chest CT and RT-PCR testing for Coronavirus Disease 2019 (COVID-19) in China: a report of 1014 cases. Radiology. 2020;296(2):E32–E40. doi: 10.1148/radiol.2020200642. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.ACR recommendations for the use of chest radiography and computed tomography (CT) for suspected COVID-19 infection. American College of Radiology website; 2020. https://www.acr.org/Advocacy-and-Economics/ACR-Position-Statements/Recommendations-for-Chest-Radiography-and-CT-for-Suspected-COVID19-Infection Published March 11, 2020. Updated March 22. Accessed April 4, 2020. [Google Scholar]
- 40.Simpson S., Kay F.U., Abbara S., Bhalla S., Chung J.H., Chung M. Radiological Society of North America expert consensus statement on reporting chest CT findings related to COVID-19. Endorsed by the Society of Thoracic Radiology, the American College of Radiology, and RSNA-secondary publication. J Thorac Imaging. 2020;35(4):219–227. doi: 10.1097/RTI.0000000000000524. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Testing FAQ: These FAQs provide answers to some of the most frequently asked questions on COVID-19. Johns Hopkins University & Medicine website. https://coronavirus.jhu.edu/testing/testing-faq. Accessed May 3, 2020.
- 42.Coronavirus Disease (COVID-19): Getting tested for COVID-19. California Government of Northwest Territories website; 2020. https://www.hss.gov.nt.ca/en/services/coronavirus-disease-covid-19/getting-tested-covid-19#how-long-does-it-take-to-get-the-results-of-a-covid-19-test- Updated April 24. Accessed May 3, 2020. [Google Scholar]
- 43.Revel M.P., Parkar A.P., Prosch H., Silva M., Sverzellati N., Gleeson F. COVID-19 patients and the radiology department — advice from the European Society of Radiology (ESR) and the European Society of Thoracic Imaging (ESTI) Eur Radiol. 2020 doi: 10.1007/s00330-020-06865-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Fields B.K.K., Demirjian N.L., Gholamrezanezhad A. Coronavirus Disease 2019 (COVID-19) diagnostic technologies: A country-based retrospective analysis of screening and containment procedures during the first wave of the pandemic. Clin Imaging. 2020;67:219–225. doi: 10.1016/j.clinimag.2020.08.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Shin H., Abdelhalim A., Chau S., Shah S., Desai B., Gholamrezanezhad A. Responding to coronavirus disease 2019: LA County hospital experience. Emerg Radiol. 2020 doi: 10.1007/s10140-020-01818-w. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.