Abstract
Purpose
The Federal Aviation Administration quantifies hazardous attitudes (HAs) among pilots using a scale. HAs have been linked to aviation risk. We assessed the influence of HAs and other factors in treatment decision making in radiation oncology (RO).
Methods and Materials
An anonymous survey was sent to 809 radiation oncologists in US cities housing the top 25 cancer centers. The survey included an HA scale adapted for RO and presented 9 cases assessing risk-tolerant radiation therapy prescribing habits and compliance with the American Society for Radiation Oncology's Choosing Wisely recommendations. Demographic and treatment decision data were dichotomized to identify factors associated with prescribing habits using univariable and multivariable (MVA) logistic regression analyses.
Results
A total of 139 responses (17.1%) were received, and 103 were eligible for analysis. Among respondents, 40% were female, ages were evenly distributed, and 83% were in academics. Median scores for all attitudes (macho, anti-authority, worry, resignation, and impulsivity) were below the aviation thresholds for hazard and data from surgical specialties. On MVA, responders >50 years old with >5 years’ experience were 4.45 times more likely to recommend risk-tolerant radiation (P = .016). Macho attitude was negatively associated with Choosing Wisely compliant treatments (odds ratio [OR], 0.12; P = .001). Physicians who reported having previously retreated the supraclavicular fossa without complication were more likely to recommend retreatment in medically unfit patients if they felt the complication was avoided owing to careful planning (OR, 5.2; P = .008).
Conclusions
To our knowledge, this represents the first study analyzing physician attitudes in RO and their effect on self-reported treatment decisions. This work suggests that attitude may be among the factors that influence risk-tolerant prescribing practices and compliance with Choosing Wisely recommendations.
Introduction
Although radiation oncology (RO) is data driven and evidence based, there exists considerable variability in physician decision-making. In fact, a leading RO journal developed a regular feature showcasing nuance and variability in practice, titled “The Gray Zone.”1
Physician decision-making remains overall poorly understood, although it is likely influenced by many factors. The airline industry has found attitude to be a significant factor in decision making.2 The Federal Aviation Administration tests all pilots for hazardous attitudes (HAs)—those contributing to poor decision-making and risky behavior.3 Five HAs are officially recognized—anti-authority, impulsivity, invulnerability, macho, and resignation (Table 1)4 —and they have been implicated in plane crashes.5 In the modified short HA scale, the single attitude of invulnerability is broken down into lack of worry and excess of self-confidence.
Table 1.
Definitions and antidotes for hazardous attitudes officially recognized by the Federal Aviation Administration4⁎
| Hazardous attitude | Definition | Antidote |
|---|---|---|
| Macho | “I can do it.” | “Taking chances is foolish.” |
| Antiauthority | “Don't tell me.” | “Follow the rules. They are usually right.” |
| Resignation | “What's the use?” | “I'm not helpless. I can make a difference.” |
| Impulsivity | “Do it quickly.” | “Not so fast. Think first.” |
| Invulnerability | “It won't happen to me.” | “It could happen to me.” |
Worry and self-confidence are attitudes not officially recognized by the Federal Aviation Administration as hazardous, although they are routinely measured.
In recent years, the concept of HAs has been applied to medicine. Among orthopedic surgeons, HA expression has been linked to rates of reoperation and readmission.6, 7, 8 This work suggests that the HA scale used to determine safe practices in pilots may have merit in medical decision-making as well. Because radiation oncologists routinely make high-impact decisions affecting the lives of patients with cancer, a better understanding of the factors associated with good judgment in radiation oncology would be valuable. The baseline expression of HAs and their influence on physician decision-making in RO is unknown. In this study, we conducted a survey analysis of radiation oncologists to assess the presence and influence of HAs in treatment decision-making.
Methods and Materials
Study design and participants
We developed an anonymous survey including an adapted version of the HA Scale. The institutional Human Investigations Committee deemed this work exempt from review (Yale). The online survey was distributed via email during a 2-month period, ending December 2017, to 809 radiation oncologists registered with the American Society of Radiation Oncology (ASTRO) in the US cities housing the top 25 cancer centers (US News & World Report rankings9). Practicing radiation oncologists were eligible. Consent was obtained and confidentiality was maintained.
Survey development
We pilot-tested our survey with practicing radiation oncologists to assess questions for clarity and meaning. The survey included 70 closed-ended, multiple-choice questions. It consisted of 4 major components: (1) an attitudes survey using the aviation HA Scale adapted for RO, (2) 4 clinical scenarios to assess willingness to prescribe risk-tolerant treatments (toward organs at risk [OARs]), (3) practice scenarios based on 5 ASTRO Choosing Wisely10, 11, 12 recommendations, and (4) demographic data.
Originally an ipsative measurement scale, the instrument assessing HAs was modified into a Likert-type scale and validated.13, 14, 15, 16, 17 We used an abbreviated 30-item version of the scale consisting of declarative statements adapted for RO.15 This short scale was selected to be consistent with literature examining HAs in medicine that measure self-confidence and worry rather than invulnerability. Statements such as “I like to practice unusual aircraft attitudes” were converted to “I like to practice unusual treatment paradigms in radiation oncology.” Like in the aviation scale, 5 unique questions to assess each attitude (macho, antiauthority, resignation, impulsivity, and invulnerability broken down into its seperate components of self-confidence and worry) were included. The threshold for concerning levels of individual HA expression (a score >20) was consistent with aviation survey grading.15
The survey then assessed 2 sets of clinical scenarios: prescribing habits and adherence to society guidelines. Four cases aimed to understand nuances in prescribing habits, including risk tolerance. These 4 cases included (1) radiation therapy (RT) allocation for an elderly woman with stage I breast cancer eligible for RT omission, (2) prescribing habits for an ultracentral lung tumor, (3) retreatment dose-fractionation of the supraclavicular fossa for a patient with symptomatic breast cancer who received prior RT, and (4) balancing risk of RT pneumonitis with adequate planning target volume coverage in lung cancer. Risk tolerance for the purposes of this study was relative to the organs at risk, such that whole-breast RT prescription was considered more risk tolerant than partial breast RT, and stereotactic body RT prescription was considered more risk tolerant than conventionally fractionated RT for an ultracentral lung cancer. In the first 3 cases, variations of patient health were presented. For the case involving retreatment of the supraclavicular fossa, an additional question (Why do you feel you did not observe a complication with retreatment?) was presented if respondents reported prior experience in retreatment of this area without complications. Respondents were asked if they thought the absence of complications was owed to chance, careful planning and delivery, or short patient survival.
The ASTRO Choosing Wisely guidelines are well known and felt to represent information that general practitioners would be aware of, even outside of their disease site specializations. Respondents were asked about their preferred treatment in patient scenarios representing targets of ASTRO's prior Choosing Wisely campaigns: management of bone metastases, low-risk prostate cancer, low-risk endometrial cancer, brain metastases, and the use of hypofractionated whole-breast RT.11, 12 Answer choices were classified as Choosing Wisely compliant or noncompliant. Nonidentifiable physician demographics and practice characteristics were also queried.
Data analysis
Descriptive statistics were used to characterize the cohort and HA expression. Demographic data, practice patterns information, and HA responses were dichotomized for univariable (UVA) and multivariable (MVA) logistic regression analyses. The attitude of self-confidence was not analyzed for association, because radiation oncologists—unlike surgeons and pilots—do not primarily rely on physical abilities, but rather performance of team members (dosimetry, therapy, physics) for job execution. For MVA, an interaction term including age and years since residency was created. We used UVA to identify demographic factors and HAs associated with compliance with Choosing Wisely recommendations and risk-tolerant (to OARs) prescriptions. Both UVA and MVA were used to identify demographic factors, practice characteristics, and HAs associated with risk-tolerant prescriptions. Additional UVA was performed to determine whether prior experience with a particular treatment scenario was associated with increased or decreased likelihood of risk-tolerant prescribing. Statistical significance was defined as P ≤ .05. Stata SE software, version 13.1 (Stata, College Station, Texas), was used.
Results
Respondent demographics
Of the 809 surveys sent, 139 were started, yielding a response rate of 17.1%. Additionally, 12 surveys were accessed by anonymous link. Excluding responses submitted as blank (n = 2), with significant missing data (n = 14), or without consent (n = 2), 103 surveys were eligible for analysis. There was a male predominance in responses, and respondents’ ages were evenly distributed (Table 2). Most respondents practiced in academic centers or university settings, completed residency within the prior 10 years, reported an average of 11 to 30 patients on treatment during a typical week, and worked regularly with more than 6 RO colleagues.
Table 2.
Demographic and practice characteristics of respondents eligible for analysis
| Variable | Respondents, No. (%) (n = 103) |
|---|---|
| Age, y | |
| <35 | 23 (22.33) |
| 35-40 | 25 (24.27) |
| 41-50 | 26 (25.24) |
| 51-60 | 18 (17.48) |
| >60 | 11 (10.68) |
| Sex | |
| Male | 61 (59.22) |
| Female | 41 (39.81) |
| Not reported | 1 (0.97) |
| Time since residency, y | |
| Still in training | 3 (2.91) |
| 0-5 | 38 (36.89) |
| 6-10 | 21 (20.39) |
| 11-15 | 8 (7.77) |
| 16-20 | 9 (8.74) |
| >20 | 24 (23.30) |
| Practice setting | |
| Academic or university | 85 (82.52) |
| Private practice, freestanding | 5 (4.85) |
| Private practice, hospital based | 8 (7.77) |
| Military or government | 1 (0.97) |
| Other | 4 (3.88) |
| Patients on treatment, average, No. | |
| 0-10 | 20 (19.42) |
| 11-20 | 60 (58.25) |
| 20-30 | 21 (20.39) |
| >30 | 2 (1.94) |
| Radiation oncology colleagues, No. | |
| 0 | 2 (1.94) |
| 1-5 | 27 (26.21) |
| 6-10 | 28 (27.18) |
| >10 | 46 (44.66) |
Hazardous attitudes in radiation oncology
Median scores for all HAs were below aviation thresholds for hazard (Table 3A). Hazardous levels of macho were expressed in 15.6% of respondents and hazardous levels of worry in 12.5% (Table 3B). Gender was not significantly associated with hazardous levels of any attitude (data not shown).
Table 3A.
Hazardous-attitude levels among respondents
| Hazardous attitude | Median (range) level of attitude reported | SD18 |
|---|---|---|
| Macho | 15 (7-24) | 3.48 |
| Antiauthority | 9 (5-20) | 3.03 |
| Worry | 15 (6-23) | 3.80 |
| Resignation | 12 (6-19) | 2.94 |
| Impulsivity | 15 (8-21) | 2.53 |
Abbreviation: SD = standard deviation.
Table 3B.
Rates of hazardous levels of hazardous attitudes among radiation oncology, neurosurgery,18 and orthopedic surgery (Hazardous Attitude Score >20)6
| Rate, % | |||
|---|---|---|---|
| Hazardous attitude | Radiation oncology (%) | Neurosurgery (%) | Orthopedic surgery (%) |
| Macho | 7.8 | 0.0 | 28 |
| Antiauthority | 0.0 | 1.5 | 3 |
| Worry | 9.7 | 3.7 | 6 |
| Resignation | 0.0 | 7.7 | 0.3 |
| Impulsivity | 1.9 | 0.4 | 1 |
Factors associated with risk-tolerant prescription
On UVA, age of >50 years (odds ratio [OR], 3.65; 95% confidence interval [CI], 1.39-9.56; P = .008) and experience of >5 years (OR, 3.18; 95% CI, 1.08-9.37; P = .036) were significantly associated with risk-tolerant prescribing. On MVA, respondents >50 years old with >5 years of experience were 4.45 (95% CI, 1.32-15.0) times more likely to recommend risk-tolerant prescriptions (P = .016; Table 4).
Table 4.
Univariable and multivariable logistic regression analysis for variables potentially associated with willingness to recommend risk-tolerant prescriptions (radiation doses that pose greater risk to the organs at risk) in surveyed scenarios*
| Variable | Univariable OR (95% CI) | P | Multivariable OR (95% CI) | P |
|---|---|---|---|---|
| Age, y | ||||
| ≤50 | 1 [Reference] | N/A | - | - |
| >50 | 3.65 (1.39-9.56) | .008 | - | - |
| Sex | ||||
| Female | 1 [Reference] | N/A | - | - |
| Male | 0.86 (0.34-.2.21) | .76 | - | - |
| Time since residency, y | ||||
| ≤5 | 1 [Reference] | N/A | - | - |
| >5 | 3.18 (1.08-9.37) | .036 | - | - |
| Age, y, and time since residency, y | ||||
| ≤50 & ≤5 | - | - | 1 [Reference] | N/A |
| >50 & ≤5 | - | – | N/A | N/A |
| ≤50 & >5 | - | - | 1.65 (0.46-5.93) | .44 |
| >50 & >5 | - | - | 4.45 (1.32-15.0) | .016 |
| Practice setting | ||||
| Academic | 1 [Reference] | N/A | 1 [Reference] | N/A |
| Other | 0.38 (0.08-1.67) | .19 | 0.31 (0.06-1.64) | .17 |
| Patients on treatment, No. | ||||
| <21 | 1 [Reference] | N/A | 1 [Reference] | N/A |
| ≥21 | 2.13 (0.77-5.9) | .14 | 2.36 (0.77-7.19) | .13 |
| Radiation oncology colleagues, No. | ||||
| 0-10 | 1 [Reference] | N/A | - | - |
| >10 | 1.32 (0.53-3.3) | .55 | - | - |
| Comfortable with treating lung | ||||
| No | 1 [Reference] | N/A | - | - |
| Yes | 0.63 (0.26-1.62) | .35 | - | - |
| Comfortable with treating breast | ||||
| No | 1 [Reference] | N/A | - | - |
| Yes | 0.91 (0.35-2.35) | .85 | - | - |
| Macho | ||||
| Bottom 3 quartiles | 1 [Reference] | N/A | - | - |
| Top quartile | 2.09 (0.72-6.04) | .17 | - | - |
| Antiauthority | ||||
| Bottom 3 quartiles | 1 [Reference] | N/A | - | - |
| Top quartile | 0.63 (0.13-3.08) | .57 | - | - |
| Worry | ||||
| Bottom 3 quartiles | 1 [Reference] | N/A | - | - |
| Top quartile | 0.63 (0.19-2.08) | .45 | - | - |
| Resignation | ||||
| Bottom 3 quartiles | 1 [Reference] | N/A | - | - |
| Top quartile | 1.01 (0.30-3.46) | .98 | - | - |
| Impulsivity | ||||
| Bottom 3 quartiles | 1 [Reference] | N/A | - | - |
| Top quartile | 0.96 (0.31-2.95) | .94 | - | - |
Abbreviations: CI = confidence interval; N/A = not applicable; OR = odds ratio.
Variables with empty spaces in the multivariate column were not included in the final multivariable analysis model.
Previous experience and perceptions surrounding complication avoidance were also associated with prescribing preferences. Physicians who reported prior experience irradiating the supraclavicular fossa without complication were significantly more likely to recommend retreatment in patients with poor health if they felt the complication was avoided owing to careful planning and delivery rather than to chance or patient death (OR, 5.2; 95% CI, 1.55-17.61; P = .008).
Compliance with the Choosing Wisely campaign
The macho attitude was negatively associated with compliance with Choosing Wisely recommendations (OR, 0.12; 95% CI, 0.03-0.40; P = .001). There was no association between compliance and any other HA or demographic or practice characteristic (Table 5).
Table 5.
Univariable logistic regression analysis for variables potentially associated with compliance with Choosing Wisely scenarios among respondents
| Variable | OR (95% CI) | P |
|---|---|---|
| Age, y | ||
| ≤50 | 1 [Reference] | N/A |
| >50 | 0.66 (0.20-2.18) | .50 |
| Sex | ||
| Female | 1 [Reference] | N/A |
| Male | 1.60 (0.46-5.59) | .46 |
| Time since residency, y | ||
| ≤5 | 1 [Reference] | N/A |
| >5 | 0.37 (0.10-1.40) | .14 |
| Practice setting* | ||
| Academic | - | - |
| Other | - | - |
| Patients on treatment, No. | ||
| <21 | 1 [Reference] | N/A |
| ≥21 | 1.06 (0.27-4.18) | .93 |
| Radiation oncology colleagues, No. | ||
| 0-10 | 1 [Reference] | N/A |
| >10 | 0.56 (0.18-1.75) | .32 |
| Macho | ||
| Bottom 3 quartiles | 1 [Reference] | N/A |
| Top quartile | 0.12 (0.03-0.40) | .001 |
| Antiauthority | ||
| Bottom 3 quartiles | 1 [Reference] | N/A |
| Top quartile | 1.83 (0.22-15.4) | .58 |
| Worry | ||
| Bottom 3 quartiles | 1 [Reference] | N/A |
| Top quartile | 0.68 (0.19-2.41) | .55 |
| Resignation | ||
| Bottom 3 quartiles | 1 [Reference] | N/A |
| Top quartile | 0.43 (0.17-1.57) | .20 |
| Impulsivity | ||
| Bottom 3 quartiles | 1 [Reference] | N/A |
| Top quartile | 0.63 (0.18-2.26) | .48 |
Abbreviations: CI = confidence interval; N/A = not applicable; OR = odds ratio.
Too few observations for analysis.
Discussion
The role of attitude in decision-making has been well established in the aviation industry. Given the recent application of this concept to surgical specialties,6, 7, 8 we sought to determine the influence of HAs and other factors on physician decision-making among radiation oncologists. Our analysis demonstrated low levels of HAs among radiation oncologists. The macho attitude was associated with nonadherence to Choosing Wisely recommendations, whereas older age and more experience were associated with a propensity to recommend risk-tolerant prescriptions. We also found that retreatment was almost 5 times more likely to be recommended when the physician felt that careful planning was responsible for avoidance of a complication. To our knowledge, this represents the first study examining hazardous attitudes in RO.
It is not clear why older age and more experience are significantly associated with an increased likelihood of recommending risk-tolerant (to OARs) prescriptions. This difference could reflect the influence of the 3-dimensional era, in which more dose-volume metrics are available, shifting focus to preventing complications. Alternatively, the threshold of 5 years of practice to observe this effect could indicate physicians having practiced long enough to see recurrences and shifting their priorities from avoiding complications to avoiding recurrence.
Although our study is thought-provoking regarding the role of attitude and other factors in decision-making in RO, there are limitations. First, there is potential nonresponse bias. The mean response rate for email surveys is approximately 36%, with recent data exhibiting lower response rates.19,20 Among oncology surveys, RO surveys may have response rates as low as 5%.21 Our respondents also included a large number of academic physicians and might not be representative of the total population of radiation oncologists. The influence of training site, disease-site specialization, local practice culture, peer influence, and era of training cannot be reliably assessed by these data but is likely to contribute to prescribing preferences. In addition, given that prescribing too much or too little RT can both be risky, it would be worthwhile to determine whether HAs are correlated with prescriptions that are risk tolerant for recurrence.
Conclusions
Our study demonstrates low levels of HAs in radiation oncologists compared with pilots and orthopedic surgeons. However, the macho attitude was associated with nonadherence to Choosing Wisely recommendations. Other factors, such as increasing age and experience, were associated with a propensity to recommend risk-tolerant (to OARs) prescriptions. To our knowledge, this is the first study examining physician attitudes in the field of RO. This work lays the foundation for further efforts to identify factors associated with physician decision-making in this field.
Acknowledgments
We acknowledge aviation enthusiast John Caserta, MSFS, ChFC, for his contributions to this work, as well as the Yale Department of Therapeutic Radiology for donating raffle prizes for participation in the survey.
Footnotes
Sources of support: This research and the resulting manuscript did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. The Yale Department of Therapeutic Radiology donated 2 iWatches as raffle prizes for participation in the survey.
Disclosures: Dr Decker reports receiving research grants from Merck & Co, Inc; receiving consulting fees from AstraZeneca and Regeneron; serving on a speaker's bureau for AstraZeneca; receiving honoraria from Physician Education Resource and PeerView; receiving personal fees from AstraZeneca, Regeneron, Physician Education Resource, and PeerView; and advisory board participation with Merck & Co, Inc, AstraZeneca, Regeneron, and Novocure. Dr Ford reports being an unpaid member of the Board of Directors of the American Society for Radiation Oncology. Dr Brown reports receiving honoraria from Varian Medical Systems for speaking engagements. Dr Dosoretz reports being an owner/manager of Advocate Radiation Oncology, LLC. Dr Evans reports receiving payments or honoraria from the Clarity Patient Safety Organization for ongoing for work in the radiation oncology health advisory committee. All other authors have no disclosures to share.
Research data are not available at this time. As part of the institutional review board approval, the data obtained from this survey are kept until publication, then destroyed.
References
- 1.Palma DA, Yom SS, Bennett KE, Corry J, Zietman AL. Introducing: The Red Journal Gray Zone. Int J Radiat Oncol Biol Phys. 2017;97:1. doi: 10.1016/j.ijrobp.2016.11.052. [DOI] [PubMed] [Google Scholar]
- 2.Federal Aviation Administration. Glossary. Available at:https://www.faa.gov/regulations_policies/handbooks_manuals/aviation/aviation_instructors_handbook/media/aviation_instructors_handbook.pdf. Accessed December 8, 2020.
- 3.Federal Aviation Administration. Aeronotical decision-making. In: Pilot's Handbook of Aeronautic Knowledge. 2001. Available at:https://www.faa.gov/sites/faa.gov/files/regulations_policies/handbooks_manuals/aviation/phak/04_phak_ch2.pdf. Accessed September 16, 2022.
- 4.Federal Aviation Administration. Risk management and single-pilot resource management.Available at:https://www.faa.gov/regulations_policies/handbooks_manuals/aviation/aviation_instructors_handbook/media/aviation_instructors_handbook.pdf. Accessed December 8, 2020.
- 5.Nuñez B, López C, Velazquez J, Mora OA, Román K. 20th International Symposium on Aviation Psychology. Core Scholar; 2019. Hazardous attitudes in us part 121 airline accidents. Paper presented at. [Google Scholar]
- 6.Bruinsma WE, Becker SJ, Guitton TG, Kadzielski J, Ring D. How prevalent are hazardous attitudes among orthopedic surgeons? Clin Orthop Relat Res. 2015;473:1582–1589. doi: 10.1007/s11999-014-3966-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Meunier A, Posadzy K, Tinghög G, Aspenberg P. Risk preferences and attitudes to surgery in decision making. Acta Orthop. 2017;88:466–471. doi: 10.1080/17453674.2017.1298353. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Kadzielski J, McCormick F, Herndon JH, Rubash H, Ring D. Surgeons’ attitudes are associated with reoperation and readmission rates. Clin Orthop Relat Res. 2015;473:1544–1551. doi: 10.1007/s11999-014-3687-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.US News & World Report. Best Hospitals for Cancer. Available at: https://health.usnews.com/best-hospitals/rankings/cancer. Accessed September 1, 2017.
- 10.Hahn C, Kavanagh B, Bhatnagar A, et al. Choosing wisely: The American Society for Radiation Oncology's top 5 list. Pract Radiat Oncol. 2014;4:349–355. doi: 10.1016/j.prro.2014.06.003. [DOI] [PubMed] [Google Scholar]
- 11.American Society for Radiation Oncology. 2013 choosing wisely list. Available at: https://www.astro.org/Patient-Care-and-Research/Patient-Education/2013-Choosing-Wisely-List. Accessed December 8, 2020.
- 12.American Society for Radiation Oncology. 2014 choosing wisely list. Available at:https://www.astro.org/Patient-Care-and-Research/Patient-Education/2014-Choosing-Wisely-List. Accessed December 8, 2020.
- 13.Holt RW, Boehm-Davis DA, Fitzgerald KA, Matyuf MM, Baughman WA, Littman DC. Behavioral validation of a hazardous thought pattern instrument. Proceedings of the Human Factors Society Annual Meeting. 1991;35:77–81. [Google Scholar]
- 14.Federal Aviation Administration. Airman research questionnaire: Methodology and overall results. Available at: https://www.faa.gov/data_research/research/med_humanfacs/oamtechreports/1990s/media/am95-27.pdf. Accessed December 8, 2020.
- 15.Aviation Human Factors. New hazardous attitudes scale. Available at:http://www.avhf.com/html/Evaluation/GMasonHazAttitudeScale/GM2.asp. Accessed December 8, 2020.
- 16.Hunter DR. Measurement of hazardous attitudes among pilots. Int J Aviat Psychol. 2005;15:23–43. [Google Scholar]
- 17.Aviation Human Factors. Hazardous Attitudes Scale—New version. Available at: http://www.avhf.com/html/Researcher/Haz_Attitude_New.asp. Accessed December 8, 2020.
- 18.Muskens IS, van der Burgt SME, Senders JT, Lamba N, Peerdeman SM, Broekman ML. Behavior and attitudes among European neurosurgeons—An international survey. J Clin Neurosci. 2018;55:5–9. doi: 10.1016/j.jocn.2018.07.019. [DOI] [PubMed] [Google Scholar]
- 19.Sheehan KB. E-mail survey response rates: A review. J Comput-Mediat Comm. 2001;6(2) [Google Scholar]
- 20.Phillips AW, Reddy S, Durning SJ. Improving response rates and evaluating nonresponse bias in surveys: AMEE Guide No. 102. Med Teach. 2016;38:217–228. doi: 10.3109/0142159X.2015.1105945. [DOI] [PubMed] [Google Scholar]
- 21.Parekh AD, Bates JE, Amdur RJ. Response rate and nonresponse bias in oncology survey studies. Am J Clin Oncol. 2020;43:229–230. doi: 10.1097/COC.0000000000000665. [DOI] [PubMed] [Google Scholar]