Skip to main content
NCBI home page
Lippincott Open Access logoLink to Lippincott Open Access
. 2020 Sep 15;275(1):e75–e81. doi: 10.1097/SLA.0000000000004078

Global Overview of Response Rates in Patient and Health Care Professional Surveys in Surgery

A Systematic Review

Vincent Maurice Meyer ∗, Stan Benjamens †, Mostafa El Moumni †, Johan F M Lange †, Robert A Pol †
PMCID: PMC8683255  PMID: 32649458

Abstract

Objective:

Identify key demographic factors and modes of follow-up in surgical survey response.

Summary Background Data:

Surveys are widely used in surgery to assess patient and procedural outcomes, but response rates vary widely which compromises study quality. Currently there is no consensus as to what the average response rate is and which factors are associated with higher response rates.

Methods:

The National Library of Medicine (MEDLINE/PubMed) was systematically searched from Januray 1, 2007 until February 1, 2020 using the following strategy: (((questionnaire) OR survey) AND “response rate”) AND (surgery OR surgical). Original survey studies from surgical(-related) fields reporting on response rate were included. Through one-way analysis of variance we present mean response rate per survey mode over time, number of additional contacts, country of origin, and type of interviewee.

Results:

The average response is 70% over 811 studies in patients and 53% over 1746 doctor surveys. In-person surveys yield an average 76% response rate, followed by postal (65%) and online (46% web-based vs 51% email) surveys. Patients respond significantly more often than doctors to surveys by mail (P < 0.001), email (P = 0.003), web-based surveys (P < 0.001) and mixed mode surveys (P = 0.006). Additional contacts significantly improve response rate in email (P = 0.26) and web-based (P = 0.041) surveys in doctors. A wide variation in response rates was identified between countries.

Conclusions:

Every survey is unique, but the main commonality between studies is response rate. Response rates appear to be highly dependent on type of survey, follow-up, geography, and interviewee type.

Keywords: email, postal, questionnaire, response rate, survey, telephone

Surveys are often conducted in the field of surgery, where they represent a valuable means of gaining insight into a topic of interest (operative technique, quality of life, complications, expert opinion) from a wide-ranging selection of people (surgeons, patients, residents, students). This robust sampling method provides useful information when the sample selected is representative of the population and its design reliable, unbiased, and discriminatory.1–3

The quality of a survey is mostly threatened by a lack of response (nonresponse bias, incomplete questionnaires) or an undesired response (social desirability bias, poor test-retest reliability, satisficing). Significant research has been done on the latter by Krosnick, who introduced the theory of “satisficing" in survey methodology.4 Krosnick states that it involves a significant amount of cognitive work to select the optimal answer to a question and (some) respondents would want to minimize that burden. Weak or strong satisficing, a portmanteau of satisfy and suffice, then reflects the act of shortcutting cognitive processes to alleviate the burden of choosing. The respondent answers the questions at hand sufficiently, but with the least effort. This will manifest in selecting “don’t know" options, random answers, and socially desirable answer options. The degree of satisficing depends on the motivation of the respondent and task difficulty.5

In the lack of response, the items themselves are hugely important; shorter questions and surveys, engagement to the subject, personalization of the questionnaire, and yes/no questions will attribute to a higher response rate.6–9 Survey mode, number and type of follow up, type of interviewee, and geographic variance also significantly impact response rate.10–13 These measurable aspects of response rate comprise a considerable, but only a part, of the puzzle. A low participation rate will introduce nonresponder selection bias (random sampling variability), which impairs validity of the researchers‘ results and as such is often noted as a study weakness by peer reviewers.3,14

A tremendous effort is therefore made toward increasing response rates to surveys. A 2009 Cochrane systematic review examined 121 different strategies to improve response rate in 481 postal and 32 electronic surveys showing that a monetary incentive, personalization, and shortening of the survey improves response rate.15,16 However, it does not state what a “good” or “acceptable” response rate is. Although often critiqued and with >500 studies reporting on interventions to enhance response rates, we still lack a consensus as to what an ideal or even average response rate is.1,17–19

Through a global systematic review of the literature we aim to provide objective data on response rates in survey studies in the field of surgery. We will present the average response rate per type of survey and follow-up, country, and type of interviewee thereby providing researchers with a tool for individual study design.

MATERIALS AND METHODS

Search Strategy

Data collection and analysis were performed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.20 The National Library of Medicine (MEDLINE/PubMed) was systematically searched from January 1, 2017 until February 1, 2020 as follows: (((questionnaire) OR survey) AND “response rate”) AND (surgery OR surgical). The review process was discussed in detail with all authors beforehand. Studies were independently screened by 2 authors (V.M. and S.B.). Studies were marked if one of the authors doubted suitability and were subsequently checked by the first author to ensure uniform reporting. In case of disagreement, consensus was reached through discussion with all authors.

Studies reporting in English on response rates to questionnaires in surgical and surgery-related fields of medicine were included. When studies reported response rates on multiple types of interviewees or modes of survey, these sub results were included separately. Studies reporting multiple surveys over time were excluded due to possible bias. Reviews, conference abstracts, case reports, and studies reporting solely from nonsurgical (or related) fields of medicine, paramedicine, or nursing were also excluded. Primary end point was mean response rate per type of survey. Secondary outcomes were response rate over time and per type of follow up, country of origin, and type of subject. Subjects were either patients or health care professionals (doctors). All identified articles were extracted to an Excel sheet in a predefined format containing Pubmed ID, title, authors, country, field of surgery, no. of interviewees that responded, response rate, no of interventions, type of interventions, mandatory nature, and responder reward. Surveys were divided in person (face to face or telephone), postal, email, or web-based surveys in case of an online questionnaire. The miscellaneous group entails mixed-mode surveys.

Follow-up was recorded as none, once, twice or >3. Follow-up could consist of a different mode of survey, that is, a telephone call after a letter was sent. Data were analyzed using IBM Statistics software SPSS 19 (2010).21 Descriptive statistics were obtained. Student t test was used to compare between health care professionals and patients. One-way analysis of variance analysis was performed for response rate over time and per follow-up contact. Countries with <10 survey studies were grouped under continent.

RESULTS

Literature Search

The initial search resulted in 5693 potential studies. After screening of the abstracts 1435 articles were excluded, leaving 4258 articles for full-text assessment. After a detailed examination, 1679 articles were excluded for various reasons (see online supplement PRISMA Flow Chart). The final selection yielded 2579 surveys matching the inclusion criteria.

Response Rates Relative to Type of Survey

The average response rate of the 2579 included studies is 58.6% ± 24.0% (mean ± SD), which is 70.0% ± 18.4% over 811 studies in patients and 53.3% ± 24.5% over 1746 health care professionals’ surveys.

Figure 1 shows the average response rate per mode of survey of patients and health care professionals. In-person studies yielded the highest average response rate: 77.8% ± 18.0% and 74.5% ± 18.7% for patients and health care professionals, respectively. Postal studies average a 68.0% ± 17.0% and 60.4% ± 18.1% response rate. Email studies give an average response rate of 68.0% ± 17.1% for patients and 50.5% ± 23.3% for health care professionals. Web-based surveys offer an average response rate of 59.3% ± 18.9% and 45.8% ± 25.0% for patients and health care professionals, respectively. In the mixed methods group the average response rate for patients is 68.7% ± 20.0% and for health care professionals 62.0% ± 23.0%.

FIGURE 1.

FIGURE 1

Open in a new tab

Mean response rate and standard deviation per mode of survey for patients and healthcare professionals.

No statistically significant difference in response rate between health care professionals and patients was found in “in person" surveys (P = 0.12). Patients respond significantly more often than health care professionals to surveys by mail (P < 0.001), Email (P = 0.003), web-based surveys (P < 0.001), and mixed mode surveys (P = 0.006). This effect is consistent over the whole study inclusion period (Fig. 2).

FIGURE 2.

FIGURE 2

Open in a new tab

Response rate per type of interviewee (patient or health care professional) over a thirteen-year interval.

Response Rates Relative to Follow-Up

Figures 3 and 4 show the response rate per mode of survey according to number of interventions, for patients and health care professionals, respectively. The Email and web-based surveys are mostly directed at health care professionals (312 vs 789 studies, respectively) and less at patients (13 vs 30 studies). Additional contacts significantly improve response rate in email (P = 0.26) and web-based (P = 0.041) surveys in health care professionals. A similar trend is seen for 1 and 2 follow-up contacts in email and web-based studies in patients, although overall follow-up is not statistically significant in the Email (P = 0.22) and web-based (P = 0.46) group. Online surveys with follow-up are not often used for patients (3 Email and 15 web-based studies). Follow-up has a significant negative effect in “in person" studies (P = 0.013), where sample size is also small for ≥2 follow-up contacts (8 studies).

FIGURE 3.

FIGURE 3

Open in a new tab

Response rate per number of contacts per mode of survey for patients.

FIGURE 4.

FIGURE 4

Open in a new tab

Response rate per number of contacts per mode of survey for healthcare professionals.

For the survey studies distributing questionnaires to patients by person (P = 0.76) or by mail (P = 0.65) and for surveys given to health care professionals by mail (P = 0.936), there is no significant difference in response rate with or without follow-up.

Geographical Differences

Figure 5 shows response rates (mean, SD) per country of origin. Patients partake more often than health care professionals in survey studies around the world.

FIGURE 5.

FIGURE 5

Open in a new tab

Response rate and standard deviation per country, region, or continent of origin.

The high patient response rate in Africa (88.1% ± 12.0%), Asia (83.9% ± 16.4%), Middle-East (80.1% ± 15.0%), China (82.3% ± 12.4%), India (93.3% ± 5.4%), and Saudi-Arabia (89.4%) reflects solely postal and in-person questionnaires. The United States has the lowest average respondent score over 225 patient surveys (64.2% ± 19.5%), with a high proportion of Email and web-based studies.

The highest response rates for health care professionals were found in Finland (85.2% ± 7.9%), Africa (77.5% ± 16.0%), China (74.7% ± 23.3%) and Norway (71.5% ± 11.6%), with only Norway reporting on Email and web-based surveys. Lowest response rates for health care professionals are found in Belgium (38.4% ± 14.0%), France (47.3% ± 25.8%), United States (48.0% ± 23.3%), and Intercontinental studies (48.8% ± 24.9%). Intercontinental studies (91%), Belgium (80%), United States (78%), and France (57%), mainly report email and webbased studies.

DISCUSSION

Our analysis is a global representation of survey studies in the surgical field and the largest systematic review to date in this field. We found an average response rate of 70.0% ± 18.4% (mean ± SD) in 811 patient surveys and 53.3% ± 24.5% in 1746 health care professional surveys. Health care professionals were found to have lower response rates, which has been reported before.12,22–24 Our review confirms that health care professionals participate less often in postal and online surveys than patients do, which is consistent over time. Health care professionals are probably a very specific group prone to satisficing, where time spent and a lack of benefit are key factors. Lowering both effort and time can be achieved in a variety of ways such as shortening a survey, shortening the questions or offering yes/no options, allowing the health care professional to decide when to fill it in (postal vs face to face), pre-stamping the return envelope, and/or providing an online survey option.12,22

Our analyses show that in-person surveys yield an average 76% response rate, where postal (65%) and online (46% webbased vs 51% email) survey response is lower on average. We therefore suggest to appraise response rate on type of survey, that is, a 65% response rate in an in-person survey represents a below average statistic for reviewers. However, a 65% response rate in a postal study parallels the average for that type of survey and should be aimed for when attempting a postal survey.

These results are in line with studies from other nonsurgical medical fields where usually a higher response rate is reported for in-person versus postal and for postal versus online surveys.11,13,25–26 Real-time data tracking, immediate survey delivery, and low costs have led to a rise in online surveys, but response rates tend to be lower and methodologies questionable.27,28 Nowadays, with the general overflow of Email contact, respondents’ willingness to partake in email surveys or satisficing could be negatively affected. It is a general consensus that a more personal face-to-face or telephone interview will reach a higher response rate, but such surveys weigh more heavily on time and resources.15,17,18,26,9–31

Additional contacts are frequently used to generate a higher response rate. Extensive research by Dillman et al has shown that great administrative detail for survey personalization, including additional mailing, boosts response rates.7,32–40 Our study shows that additional contacts do not significantly raise response rates compared to a single questionnaire in postal and in-person surveys, contradicting the findings of Dillman et al.27 This difference could be explained by a general trend of declining response rates around the turn of the century.10,41–43 The method used by Dillman et al, however, encompasses more than just a reminder letter. The total design method includes a series of personal approach measures resulting in better response rates.8,10,27,44 Hence, additional contacts in postal or in-person surveys by themselves do not enhance response rates. However, mailings as part of a personalization process could be beneficial.30

Interestingly, for health care professionals we do see a significant effect of additional contacts on response rates in Email and web-based surveys. A systematic review of 69 Internet-based surveys of health care professionals in 48 studies also reported a significant increase in response rates after sending reminder Emails.11 Notably, no additional contact appears to generate the highest response rate in our comprehensive analysis. This could be due to selection bias where researchers achieving a high response rate are less inclined to send follow-up Emails. There is also a heterogeneity in this group because of likely nonreporting of reminder Emails, so there might be a (stronger) beneficial effect on response rates from reminder emails for online or email questionnaires which we cannot identify. In our series, follow-up negatively impacts response rate in “in person" patient surveys. This is possibly an effect of the very small sample size and thereby more pronounced survey-specific factors.

Although guidelines exist, survey study methodology is often still questionable or at least not reported. The American Association for Public Opinion Research (AAPOR) has published a code of ethics and minimum disclosures for researchers.45 A separate checklist for internet surveys (CHERRIES) was presented by the Journal of Medical Internet Research.46 The “Strengthening the Reporting of observational studies" (STROBE) statement does offer checklists for epidemiological cross-sectional studies, but these do not offer reporting characteristics unique to surveys.47 There is considerate literature in the social sciences on study design and reporting, but a considerate amount of surveying attempts do not adhere to these guidelines.48 For example, even response rate itself is reported ambiguously. Does one include all the returned questionnaires or only the completed ones? A 2011 review showed that 154 of 165 journals do not provide guidance on survey reporting, whereas 82% have published survey research.49 These results show that, although separate guidelines exist, there is little control on survey reporting and the need for a well-developed widely adopted reporting guideline is there.

Our analysis presents a unique global overview of reported response rates in surgical survey studies and shows what response rates depend on and are influenced by. In-person surveying has the best results, but is time-consuming and relatively expensive. Postal surveying delivers consistent response rates but is more rigid, depends on accurate mailing lists, offers less certainty about who completed the survey, and is more susceptible to literacy bias.50,51 Ubiquitous digital connectivity promises fast, low-cost, real-time monitored surveying but is seriously threatened by low response rates and often flawed survey design.

In the era of high patient awareness and increasing demand from government and insurance carriers, the need for quality control has pushed the limits of survey attempts and will continue to do so. Expert consultation should be sought before attempting a survey. Well-defined questions, survey composition, and sample selection can add much needed value to conclusions drawn from survey studies. The variance in reported response rates, signifying the heterogeneity in survey response, shows that it is imperative to reach each interviewee personally and in the right manner. Mixed-mode designs (ie, an email followed by a telephone call) tailored to the targeted population (ie, student vs old age pensioner) will improve response rates significantly.23,44,52–54 Finally, a clear study design and description will help compare survey attempts and identify key influencing factors on survey outcome.

This study has a few shortcomings that need to be addressed. Our search algorithm revealed a vast amount of studies, although we realize that surveys could have been missed. Second, choosing to reply to a survey is rather personal and depends on several variables. Many aspects of survey design that influence response rates are difficult to reproduce such as wording, length and number of questions, and personalization of a cover letter.6,8–10 Salience is one of the key factors to influence response rates.55–62 No review can account for these factors, and to maximize response rates future studies should consider that. We identified those aspects of survey design that can be monitored and reproduced. Finally, surveys often lack a properly defined methodology, which hinders objective comparison of outcomes. The type of questionnaire or follow-up is not always mentioned. Our analysis is limited by its data, which is heterogeneous at best. Uniform reporting of outcomes will help improve the predictive value of future survey study analysis.

In conclusion, the quality of a survey depends on how its questions are answered and how often it is replied to. Response rate is measurable and is influenced by many amendable factors. Overall, patients partake more often in surveys then health care professionals regardless of country, survey mode, or follow-up. Follow-up appears to improve response rate in online surveys aimed at health care professionals, whereas effect on patient surveys remains unclear. Personal and postal surveys do not seem to benefit from follow-up. Our global review provides a first overview of surgical survey response rate and can be used as a quality reference in peer review. This review will aid researchers in future survey study design; it is up to the surveyor to choose depending on their specific goals and resources.

Supplementary Material

Supplemental Digital Content
ansu-275-e75-s001.doc (195.5KB, doc)

Acknowledgments

The authors would especially thank Rahul Gannamani (RG) for his help in the literature search.

Footnotes

Competing interest statement: All authors declare no support from any organization for the submitted work; no financial relationships with any organizations that might have an interest in the submitted work; no other relationships or activities that could appear to have influenced the submitted work.

Authors’ contributions and sources: V.M.M. [surgical resident (vincentmeyer@gmail.com)] was involved in both data-collection and data-analysis, and also wrote the initial and revised version of the presented manuscript; S.B. [(guarantor): medical student and PhD candidate (s.benjamens@umcg.nl)] was involved in data-analysis and designed the presented figures, and reviewed the final version of this manuscript; M.E.M., [surgeon and epidemiologist (m.el.moumni@umcg.nl)] provided input for the statistical analysis and reviewed the final version of this manuscript; J.F.M.L. [surgeon (j.lange@umcg.nl)] was involved in the design of this study and reviewed the final version of this manuscript; R.A.P. [surgeon (pol.chirurgie@gmail.com)] initiated the presented study, was involved in both data-collection and data-analysis. and reviewed the final version of this manuscript.

The authors report no conflict of interests.

Supplemental digital content is available for this article.

REFERENCES

  • 1.Mccoll E, Jacoby A, Thomas L, et al. Design and use of questionnaires: a review of best practice applicable to surveys of health service staff and patients. Health Technol Assess 2001; 5:1–256. [DOI] [PubMed] [Google Scholar]
  • 2.Marshall G. The purpose,;1; design and administration of a questionnaire for data collection. Radiography 2005; 11:131–136. [Google Scholar]
  • 3.Phillips AW, Reddy S, Durning SJ. Improving response rates and evaluating nonresponse bias in surveys: AMEE Guide No. 102. Med Teach 2016; 38:217–228. [DOI] [PubMed] [Google Scholar]
  • 4.Krosnick JA. Survey research. Annu Rev Psychol 1999; 50:537–567. [DOI] [PubMed] [Google Scholar]
  • 5.Krosnick JA. Response strategies for coping with the cognitive demands of attitude measures in surveys. Appl Cogn Psychol 1991; 5:213–236. [Google Scholar]
  • 6.Adatia FA, Munro M, Jivraj I, et al. Documenting the subjective patient experience of first versus second cataract surgery. J Cataract Refract Surg 2015; 41:116–121. [DOI] [PubMed] [Google Scholar]
  • 7.Ferrante JM, Fyffe DC, Vega ML, et al. Family physicians’ barriers to cancer screening in extremely obese patients. Obesity (Silver Spring) 2010; 18:1153–1159. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Dillman D, Sons JW, Editor M. Mail and telephone surveys-the total design method. 1978. Available at: https://www.ncjrs.gov/app/abstractdb/AbstractDBDetails.aspx?id=59541. Accessed January 6, 2019. [Google Scholar]
  • 9.Kaya Z, Gültekin KE, Demirtaş OK, et al. Effects of targeted education for first-year university students on knowledge and attitudes about stem cell transplantation and donation. Exp Clin Transplant 2015; 13:76–81. [DOI] [PubMed] [Google Scholar]
  • 10. Dillman D. Mail and Internet Surveys: The Tailored Design Method--2007 Update with New Internet, Visual, and Mixed-Mode Guide.; 2011. Available at: https://books.google.nl/books?hl=en&lr=&id=d_VpiiWp51gC&oi=fnd&pg=PT5&ots=OjHWOw9Abr&sig=Mvu48xM_cr5ZX9rdVRAtRvzT2C0. Accessed January 6, 2019. [Google Scholar]
  • 11.Cook C, Heath F, Thompson RL. A meta-analysis of response rates in web- or internet-based surveys. Educ Psychol Meas 2000; 60:821–836. [Google Scholar]
  • 12.Kellerman S. Physician response to surveys. A review of the literature. Am J Prev Med 2001; 20:61–67. [DOI] [PubMed] [Google Scholar]
  • 13.Reinisch JF, Yu DC, Li W-Y. Getting a valid survey response from 662 plastic surgeons in the 21st century. Ann Plast Surg 2016; 76:3–5. [DOI] [PubMed] [Google Scholar]
  • 14.Boynton PM, Greenhalgh T. Selecting, designing, and developing your questionnaire. BMJ 2004; 328:1312–1315. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Edwards PJ, Roberts I, Clarke MJ, et al. Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev 2009; MR000008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Edwards P, Roberts I, Sandercock P, et al. Follow-up by mail in clinical trials: does questionnaire length matter? Control Clin Trials 2004; 25:31–52. [DOI] [PubMed] [Google Scholar]
  • 17.Jones D, Story D, Clavisi O, et al. An introductory guide to survey research in anaesthesia. Anaesth Intensive Care 2006; 34:245–253. http://www.ncbi.nlm.nih.gov/pubmed/16617649http://www.ncbi.nlm.nih.gov/pubmed/16617649. Accessed July 17, 2017. [DOI] [PubMed] [Google Scholar]
  • 18.Sprague S, Quigley L, Bhandari M. Survey design in orthopaedic surgery: getting surgeons to respond. J Bone Joint Surg Am 2009; 91: (suppl 3): 27–34. [DOI] [PubMed] [Google Scholar]
  • 19. Oppenheim AN (Abraham N, Oppenheim AN Abraham N. Questionnaire Design, Interviewing, and Attitude Measurement. New ed. London, New York; New York: Pinter Publishers; 1992. Available at: http://www.worldcat.org/title/questionnaire-design-interviewing-and-attitude-measurement/oclc/25788336. Accessed July 10, 2017. [Google Scholar]
  • 20.Shamseer L, Moher D, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015; 349:g7647–g7647. [DOI] [PubMed] [Google Scholar]
  • 21. IBM. IBM SPSS Statistics for Windows. 2013. [Google Scholar]
  • 22.Cunningham CT, Quan H, Hemmelgarn B, et al. Exploring physician specialist response rates to web-based surveys. BMC Med Res Methodol 2015; 15:32. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Scott A, Jeon S-H, Joyce CM, et al. A randomised trial and economic evaluation of the effect of response mode on response rate, response bias, and item non-response in a survey of doctors. BMC Med Res Methodol 2011; 11:126. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Braithwaite D, Emery J, De Lusignan S, et al. Using the Internet to conduct surveys of health professionals: a valid alternative? Fam Pract 2003 2019; 20:545–551. http://www.ncbi.nlm.nih.gov/pubmed/14507796http://www.ncbi.nlm.nih.gov/pubmed/14507796. Accessed January 6,. [DOI] [PubMed] [Google Scholar]
  • 25.Palmen LN, Schrier JCM, Scholten R, et al. Is it too early to move to full electronic PROM data collection?: A randomized controlled trial comparing PROM's after hallux valgus captured by e-mail, traditional mail and telephone. Foot Ankle Surg 2016; 22:46–49. [DOI] [PubMed] [Google Scholar]
  • 26.Harrison S, Henderson J, Alderdice F, et al. Methods to increase response rates to a population-based maternity survey: a comparison of two pilot studies. BMC Med Res Methodol 2019; 19:65. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Hoddinott SN, Bass MJ. The dillman total design survey method. Can Fam Physician 1986; 32:2366–2368. [PMC free article] [PubMed] [Google Scholar]
  • 28.Watt JH. Internet systems for evaluation research. New Dir Eval 1999; 1999:23–43. [Google Scholar]
  • 29.Iglesias CP. Increasing response rates to postal questionnaires. Bmj 2002; 325:444–1444. [PMC free article] [PubMed] [Google Scholar]
  • 30.Scott P, Edwards P. Personally addressed hand-signed letters increase questionnaire response: a meta-analysis of randomised controlled trials. BMC Health Serv Res 2006; 6:111. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Hing CB, Smith TO, Hooper L, et al. A review of how to conduct a surgical survey using a questionnaire. Knee 2011; 18:209–213. [DOI] [PubMed] [Google Scholar]
  • 32.Hammer A, Ommen O, Röttger J, et al. The relationship between transformational leadership and social capital in hospitals—a survey of medical directors of all German hospitals. J Public Health Manag Pract 2012; 18:175–180. [DOI] [PubMed] [Google Scholar]
  • 33.Levitt C, Hanvey L, Bartholomew S, et al. Use of routine interventions in labour and birth in Canadian hospitals: comparing results of the 1993 and 2007 Canadian hospital maternity policies and practices surveys. J Obstet Gynaecol Can 2011; 33:1208–1217. [DOI] [PubMed] [Google Scholar]
  • 34.Quinn GP, Vadaparampil ST, Malo T, et al. Oncologists’ use of patient educational materials about cancer and fertility preservation. Psychooncology 2012; 21:1244–1249. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Colakoglu S, Khansa I, Curtis MS, et al. Impact of complications on patient satisfaction in breast reconstruction. Plast Reconstr Surg 2011; 127:1428–1436. [DOI] [PubMed] [Google Scholar]
  • 36.Alderman AK, Hawley ST, Morrow M, et al. Receipt of delayed breast reconstruction after mastectomy: do women revisit the decision? Ann Surg Oncol 2011; 18:1748–1756. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Lee BT, A. Adesiyun T, Colakoglu S, et al. Postmastectomy radiation therapy and breast reconstruction: an analysis of complications and patient satisfaction. Ann Plast Surg 2010; 64:679–683. [DOI] [PubMed] [Google Scholar]
  • 38.Jagsi R, Abrahamse P, Morrow M, et al. Patterns and correlates of adjuvant radiotherapy receipt after lumpectomy and after mastectomy for breast cancer. J Clin Oncol 2010; 28:2396–2403. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Yueh JH, Slavin SA, Bar-Meir ED, et al. Impact of regional referral centers for microsurgical breast reconstruction: the New England perforator flap program experience. J Am Coll Surg 2009; 208:246–254. [DOI] [PubMed] [Google Scholar]
  • 40.Waljee JF, Hu ES, Newman LA, et al. Predictors of re-excision among women undergoing breast-conserving surgery for cancer. Ann Surg Oncol 2008; 15:1297–1303. [DOI] [PubMed] [Google Scholar]
  • 41.Robson EJ, Campbell JP, Yentis SM. The value of surveys in obstetric anaesthesia. Int J Obstet Anesth 2015; 24:46–52. [DOI] [PubMed] [Google Scholar]
  • 42.Kellerman SE, Herold J. Physician response to surveys. A review of the literature. Am J Prev Med 2001; 20:61–67. [DOI] [PubMed] [Google Scholar]
  • 43.Hohwü L, Lyshol H, Gissler M, et al. Web-based versus traditional paper questionnaires: a mixed-mode survey with a nordic perspective. J Med Internet Res 2013; 15:e173. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Dillman DA. The design and administration of mail surveys. Annu Rev Sociol 1991; 17:225–249. [Google Scholar]
  • 45.Bennett C, Khangura S, Brehaut J, et al. Reporting guidelines for surveys: limited guidance and little adherence. Int Congr Peer Rev Biomed Publ 2009. [Google Scholar]
  • 46.Eysenbach G. Improving the quality of web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES) | Eysenbach. J Med Internet Res 2004; 6:e34+.doi:10.2196/jmir.6.3.e34. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.von Elm E, Altman DG, Egger M, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet 2007; 370:1453–1457. [DOI] [PubMed] [Google Scholar]
  • 48.Weisberg H, Krosnick J, Bowen B. An Introduction to Survey Research, Polling, and Data Analysis; 1996. Available at: https://books.google.com/books?hl=en&lr=&id=Zh2shPkPzUwC&oi=fnd&pg=PP13&dq=Weisberg,+H.+F.,+Krosnick,+J.+A.,+%26+Bowen,+B.+D.+(1996).+An+introduction+to+survey+research,+polling,+and+data+analysis+(3rd+ed.).+Sage+Publications,+Inc.&ots=nFo0u7Rc9Q&sig=uueLSLDL9ZUj-_jiJ33Nopce1I8. Accessed February 26, 2020. [Google Scholar]
  • 49.Bennett C, Khangura S, Brehaut JC, et al. Reporting guidelines for survey research: an analysis of published guidance and reporting practices. PLoS Med 2011; 8:e1001069. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.McMahon SR, Iwamoto M, Massoudi MS, et al. Comparison of e-mail, fax, and postal surveys of pediatricians. Pediatrics 2019; 111 (4 pt 1):e299–303. http://www.ncbi.nlm.nih.gov/pubmed/12671142http://www.ncbi.nlm.nih.gov/pubmed/12671142. Accessed April 5,. [DOI] [PubMed] [Google Scholar]
  • 51.Routledge, Moser CA, Kalton G, Kalton G. Survey Methods in Social Investigation. 2017; doi:10.4324/9781315241999. [Google Scholar]
  • 52.Brtnikova M, Crane LA, Allison MA, et al. A method for achieving high response rates in national surveys of U.S. primary care physicians. PLoS One 2018; 13:e0202755.doi:10.1371/journal.pone.0202755. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Mauz E, von der Lippe E, Allen J, et al. Mixing modes in a population-based interview survey: comparison of a sequential and a concurrent mixed-mode design for public health research. Arch Public Heal 2018; 76:8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Beebe TJ, Jacobson RM, Jenkins SM, et al. Testing the impact of mixed-mode designs (mail and web) and multiple contact attempts within mode (mail or web) on clinician survey response. Health Serv Res 2018; 53:3070–3083. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Meza JM, Rectenwald JE, Reddy RM. The bias against integrated thoracic surgery residency applicants during general surgery interviews. Ann Thorac Surg 2015; 99:1206–1212. [DOI] [PubMed] [Google Scholar]
  • 56.Maisonneuve JJ, Lambert TW, Goldacre MJ. UK doctors’ views on the implementation of the European Working Time Directive as applied to medical practice: a quantitative analysis. BMJ Open 2014; 4:e004391. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Hailu D, Berhe H. Knowledge about obstetric danger signs and associated factors among mothers in Tsegedie district, Tigray region, Ethiopia 2013: community based cross-sectional study. PLoS One 2014; 9:e83459. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Kloek CE, Borboli-Gerogiannis S, Chang K, et al. A broadly applicable surgical teaching method: evaluation of a stepwise introduction to cataract surgery. J Surg Educ 2014; 71:169–175. [DOI] [PubMed] [Google Scholar]
  • 59.Spanager L, Dieckmann P, Beier-Holgersen R, et al. Comprehensive feedback on trainee surgeons’ non-technical skills. Int J Med Educ 2015; 6:4–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60.Ahmed K, Aydin A, Dasgupta P, et al. A novel cadaveric simulation program in urology. J Surg Educ 2015; 72:556–565. [DOI] [PubMed] [Google Scholar]
  • 61.Kim JJ, Gifford ED, Moazzez A, et al. Program factors that influence American Board of Surgery in-training examination performance: a multi-institutional study. J Surg Educ 2015; 72:e236–e242. [DOI] [PubMed] [Google Scholar]
  • 62.Touma NJ, Siemens DR. Attitudes and experiences of residents in pursuit of postgraduate fellowships: a national survey of Canadian trainees. Can Urol Assoc J 2014; 8:437–441. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Digital Content
ansu-275-e75-s001.doc (195.5KB, doc)

Articles from Annals of Surgery are provided here courtesy of Wolters Kluwer Health

RESOURCES