Research Methods in Healthcare Epidemiology: Survey and Qualitative Research

Abstract

Surveys are one of the most frequently employed study designs in healthcare epidemiology research. Generally easier to undertake and less costly than many other study designs, surveys can be invaluable to gain insights into opinions and practices in large samples and may be descriptive and/or be used to test associations. In this context, qualitative research methods may complement this study design either at the survey development phase and/or at the interpretation/extension of results stage. This methods article focuses on key considerations for designing and deploying surveys in healthcare epidemiology and antibiotic stewardship, including identification of whether or not de novo survey development is necessary, ways to optimally lay out and display a survey, denominator measurement, discussion of biases to keep in mind particularly in research using surveys, and the role of qualitative research methods to complement surveys. We review examples of surveys in healthcare epidemiology and antimicrobial stewardship and review the pros and cons of methods used. A checklist is provided to help aid design and deployment of surveys in healthcare epidemiology and antimicrobial stewardship.

Surveys are a commonly used tool in healthcare epidemiology and antimicrobial stewardship research. Surveys allow selection of a relatively large sample of people from a predetermined population, followed by collection of data from those individuals, and may be exploratory, descriptive, or explanatory. The key considerations for research using surveys are that as with other methods, developing and refining a central research question are important, followed by careful consideration of the population being sought. A search of the literature for existing survey tools that might be adapted or used as is (with the necessary permissions) is essential. If no such instrument exists, developing a de novo survey may be necessary. In this case, attention to question choice, length, comprehensibility, display, validity, reproducibility, and choice of responses is critical to successfully deploying a survey in healthcare epidemiology and antimicrobial stewardship. If not enough is known about the general question of interest, qualitative methods may be useful to identify key areas to focus on in a survey. Alternatively, survey responses may be followed up with more in-depth qualitative interviewing (for example) to add depth and other useful information but at the cost of increasing participant burden. General guidance on survey development is available online at https://oqi.wisc.edu/resourcelibrary/uploads/resources/Survey_Guide.pdf, and several texts provide detailed useful information.^1–3 This article focuses on important and unique aspects of surveys in healthcare epidemiology and antimicrobial stewardship.

ADVANTAGES AND DISADVANTAGES OF SURVEYS

There are several advantages to surveys (Table 1). The ease and cost-efficient nature of this study design is very appealing. While each mode offers its own unique advantages and disadvantages, in general, the most frequent type of survey in healthcare epidemiology and antimicrobial stewardship is the online survey. The availability of user-friendly sophisticated software to develop and deploy surveys has made it very convenient, even for surveys that require complex programming. Since there is no interviewer involved, respondents may be more willing to share information but conversely, there is no possibility for probing or clarifying responses. Although there is considerable variation,⁴ average response rates for online surveys range from 20% to 30%. Nonresponse bias stemming from a low response rate can lead to biased survey results if the respondents have differing characteristics than nonrespondents. For example, in a survey to examine antimicrobial stewardship practices, institutions with robust stewardship programs may be more likely to respond than institutions with little or no antimicrobial stewardship activity. There is no universally agreed-upon minimum response rate for online surveys but capturing data to the extent possible on the characteristics of the respondents and the nonrespondents may allow an assessment of the impact of response rates on the study results.

Table 1.

Advantages and Disadvantages of Using Surveys in Healthcare Epidemiology Research

Survey type	Advantages	Disadvantages
Mail	Easy and cost-efficient No interviewer, respondents may be more willing to share information	Response rates are typically low Not appropriate for low literacy audiences No interviewer, respondents cannot be probed
Phone	Large-scale accessibility in many countries Rapid data collection, particularly with the integration of computer-assisted telephone-interviewing systems Quality control Anonymity Flexibility	Lack of visual materials Call screening is common Limited open-ended questions or time constraints due to limited survey length Wariness Inattentiveness
Online	Low costs Automation and real-time access Less time needed Convenience for respondents Design flexibility, surveys can be programmed even if they are very complex No interviewer, respondents may be more willing to share information	Limited sampling and respondent availability Possible cooperation problems No interviewer, respondents cannot be probed
Face to face	Good response rates Longer interviews more likely to be tolerated Attitude can be observed	Expensive Time-consuming May produce a nonrepresentative sample

NOTE. Adapted from http://www.fao.org/ag/humannutrition/33251-0a0b0928c3532ed36c3b6624d54d73f86.pdf.

Interventions that have been shown to increase response rates for online surveys include clarity in the instructions, motivation for participation, choosing a topic that is highly relevant to the audience, brevity, survey length and ease of completing the survey, incentives and rewards for participation, and repeat reminders to nonrespondents.^5–7 Useful resources include http://www.qualtrics.com/blog/how-to-increase-online-survey-response-rates/ and http://socialnorms.org/what-is-an-acceptable-survey-response-rate/.

Other modes of survey administration that may have applicability for healthcare epidemiology and antimicrobial stewardship are phone interviews and face-to-face surveys. The advantage of phone interviews is that they are less likely to be self-selective. Also, there is a higher completion rate and quality control since the interviewer can guide the respondents to any questions not understood as opposed to leaving them unanswered. The main disadvantage is that they are generally limited to a maximum of about 15 minutes. The longer a phone survey continues, the more people will “drop out” and not fully answer all the questions. Questions must be simple and clearly stated. Since those responding cannot see or read the questions, complicated or long questions are not appropriate for telephone surveys. Respondents might be biased toward giving socially desirable answers to the interviewer. For face-to-face sessions, response rates are better but this is the most expensive and time-consuming method of administering surveys.

PITFALLS AND TIPS

As with other research designs, many pitfalls in surveys are avoidable through adequate planning and development of a specific research question prior to design and deployment and careful consideration of the population under study (Table 2). Each question in a survey must be deliberately positioned and included only if the responses generated will contribute to answering the research question. Ambiguous, complex, or vague questions and inconsistency in definitions, scale, or wording can compromise the validity of a survey. For questionnaire design, attention should be paid to the layout, the length and general organization, and skip patterns. All too often, surveys that are too long and poorly organized are doomed to fail because of the participant burden they entail.

Table 2.

Pitfalls in Questionnaire Design and Development

Source	Bias or other pitfall
1. Question design
Problems with wording	Ambiguous question Complex question Double-barreled question (2 questions in 1) Short question Technical jargon Uncommon word Vague word
Missing or inadequate data for intended purpose	Belief vs behavior (hypothetical question, personalized question) Data degradation Insensitive measure
Faulty scale	Forced choice (insufficient category) Missing interval Overlapping interval Scale format
Intrusiveness	Reporting (self-report response) Sensitive question
Inconsistency	Case definition Change of scale Change of wording
2. Questionnaire design
Formatting problem	Horizontal response format Juxtaposed scale (questionnaire format) Left alignment and right alignment
Questionnaire too long	No-saying (nay-saying) and yes-saying (yea-saying) Open question (open-ended question) Response fatigue
Flawed questionnaire structure	Skipping question
3. Administration of questionnaire
Interviewer not objective	Interviewer Nonblinding
Respondent’s subconscious reaction	End aversion (central tendency) Positive satisfaction (positive skew)
Respondent’s conscious reaction	Faking bad (hello-goodbye effect) Faking good (social desirability, obsequiousness) Unacceptable disease Unacceptable exposure Unacceptability Underlying cause (rumination)
Respondent’s learning	Learning Hypothesis guessing
Respondent’s inaccurate recall	Primacy (respondents remember choices that appear first in a given list, and may be more likely to select these response options) Recency (respondents are more likely to remember the last choices of a list, and may be more likely to select a choice from the latter part of a response list) Proxy respondent (surrogate data) Recall bias

NOTE. Adapted from Choi et al.¹⁷

The layout of the survey needs to be consistent, respondent-friendly, and not laborious. Several excellent texts on survey design provide guidance on the preferred layout of surveys.¹

The mode of survey should be planned well in advance, and the pros and cons of each mode should be considered, depending upon the aim of the survey and the population being covered. Many surveys in healthcare epidemiology and antimicrobial stewardship are electronic and self-administered. Although that option is convenient, it may lead to issues if respondents have different interpretations of questions than intended.

The study question should drive consideration of whether a survey’s shortcomings, such as recall bias and social desirability bias, may be important enough to consider an alternative study design. For example, a survey in healthcare epidemiology that asks detailed questions regarding extensive recall may not provide the information that is being sought.

Pilot testing of a newly developed or adapted survey instrument is essential. At a minimum, pilot testing should be done for comprehensibility, face validity, participant burden, and layout, including accuracy of skip patterns.

Many of these pitfalls related to survey design and layout can be avoided by using an existing standardized, validated survey if it meets the needs of the study. It can be very time-consuming and burdensome to develop questions from scratch. Several research groups have used the Society for Healthcare Epidemiology of America Research Network to conduct surveys in healthcare epidemiology and antimicrobial stewardship, and it would be worthwhile to peruse the Society for Healthcare Epidemiology of America Research Network website for an existing survey instrument that could lend itself to adaptation.

A second major category of pitfalls is in the deployment of surveys. The measurement and consideration of the denominator are important to be able to calculate response rates and understand how nonrespondents will be identified and compared with respondents (eg, demographic or other characteristics to decrease selection bias).

A common issue that should be addressed is the effect of respondent identifiability. This can be separated into 2 categories: anonymity and confidentiality. The study question should drive decision-making regarding whether a survey should be anonymous, confidential, or neither. If the survey contains sensitive questions, anonymity is the preferred approach since it may lead to more honest responses. However, an anonymous survey cannot be authenticated, meaning that responses cannot be restricted to be one per individual. For online surveys, survey software can be used to make responses anonymous. If it is desirable to track participant responses, such as in longitudinal surveys, confidential surveys may be a better option.

Sample size considerations for surveys require knowledge of the population size, the margin of error, and the confidence interval. Survey development and deployment software often come with an online calculator to determine sample size (https://www.surveymonkey.com/mp/sample-size-calculator/), which is a useful early step as in any other research method.

Statistical analysis of surveys ranges from relatively easy with descriptive analyses to quite complex in case of weighted population surveys. For existing surveys such as (for example) the Agency for Healthcare Research and Quality Culture of Safety Survey, review of documentation and understanding the questions, the outcomes, and the data collected are especially essential. An analysis plan should be created ahead of time. Creating table shells that can be completed later with data is a useful exercise for planning the analysis phase. There are no current national guidelines or standards regarding the reporting of survey results.

QUALITATIVE RESEARCH IN THE CONTEXT OF SURVEYS

Qualitative research is a vast science with its own sets of methods, so an in-depth discussion of this science is beyond the scope of this paper. Here we discuss only those aspects of qualitative research that may complement survey design and/or analysis. Qualitative approaches are becoming more common in health-care research and are dependent on conceptual, rather than numerical, analysis. Qualitative research is used to discern complex social processes, understand how a particular phenomenon is perceived by the study patients, and discover beliefs, values, or motivations underlying their behavior. ^8,9 Methods have expanded over the past 2 decades to include ethnography, grounded theory, case studies, phenomenological research, narrative research, participatory action research, and discourse analysis. These methods can be employed when little knowledge exists about a particular research area and can also generate hypotheses for future quantitative research.⁹ In situations for which quantitative data alone will not suffice, the addition of qualitative approaches can be helpful; the combination is called the “mixed methods” approach.⁹

Philosophically, qualitative research takes a constructivist approach and tends to be participatory and collaborative in nature. Qualitative methods usually occur within a natural setting with the researcher as key to the process. Qualitative information can validate accuracy of quantitative data from surveys or can be used in advance of survey development as a step to identify key themes. For example, a researcher can hold focus groups of caregivers for those with “lived experience” of a healthcare-associated infection to examine key themes important to caregivers. These key themes can then be used as a starting point for survey development.

The methodology for conducting unbiased, credible, and rigorous qualitative research has been published elsewhere.^9,10 One critical decision is thinking about whom or what to include in the sample, and in contrast to quantitative research, there is no predetermined sample size. Rather, sampling stops when the saturation end point, which is defined as the point at which the encounters no longer provide new information, has been reached.¹¹ Sampling should include negative or deviant cases to ensure the inclusion of all perspectives.¹² Details regarding criteria for selecting and recruiting study participants, the research setting, ethical governance (eg, informed consent, maintenance of confidentiality), methods for recording and transcribing data, data analysis (including whether it was repeated by other researchers to guarantee reliability), and consideration for how the researcher(s) could have introduced bias should be provided.^12,13 Researchers should carefully examine the target audience and spend time reviewing theories, frameworks, or models that may assist in the organization and interpretation of data.

There are several advantages that qualitative research provides (Table 3). Issues and subjects can be evaluated in more depth and detail; interviews are not limited to particular questions and can be redirected or guided by researchers in real time; and the direction and framework of research can be revised quickly as soon as fresh information and findings emerge.¹² On the other hand, readers may not be familiar with the terms and concepts of qualitative research nor how to appraise such work.^13,14 Other potential pitfalls are that the quality of such studies is heavily dependent on the experience and skills of the investigators; the presence of the researcher(s) during data gathering may affect responses; the quantity of the data can make interpretation and analysis time-consuming; and the personal biases of the researcher(s) may influence data interpretation.¹² Because of the complexity and ambiguity of healthcare environments and healthcare roles, contextual factors cannot be ignored. Qualitative methods can build the context or the “story” needed to further understand a phenomenon or issue.

Table 3.

Advantages and Disadvantages of Qualitative Research^12–14

Advantages

Disadvantages

Issues and subjects can be evaluated in depth and detail. Interviews can be redirected or guided by researchers in real-time. Direction and framework of research can be revised quickly as soon as new information emerges. Complexities and nuances about the subject matter can be discovered. Data can be generalizable to other settings.

Readers may not know terms and concepts of qualitative research. Research quality is dependent on the experience and skills of investigators. Presence of researcher during data gathering can affect responses. Quantity of data can make interpretation and analysis time-consuming. Personal biases of researchers may influence data interpretation. Maintaining confidentiality during presentation of findings can be challenging.

EXAMPLES OF SURVEY METHODS IN HEALTHCARE EPIDEMIOLOGY AND ANTIMICROBIAL STEWARDSHIP

An example of a survey is one by Drees et al.¹⁵ The investigators conducted a cross-sectional survey of members of the Society for Healthcare Epidemiology of America Research Network to investigate variation in definitions and isolation procedures for multidrug-resistant gram-negative bacteria. In this study respondents were from 15 countries and 26 states in the United States, with a 39% response rate. The investigators acknowledged the low response rate and that respondents were selected from a convenience sample, which could cause selection bias. Since respondents were part of a healthcare epidemiology network, they are probably more motivated to participate and perhaps give socially desirable answers compared with nonrespondents. No information was available for nonrespondents, which is a limitation. Data on the number or rates of multidrug-resistant organism infections in each institution were not available; therefore, variations in practice were not adjusted for the frequency with which the respondents encounter these organisms. Despite the limitations, the survey provides helpful information to understand practices in a large group of hospitals and different geographic locations; the results are useful to understand baseline behaviors in a selected sample and possibly guide future studies to standardize best practices in the isolation of multidrug-resistant organisms.

An example of a mixed methods survey is the work by Glanz et al titled “A Mixed Methods Study of Parental Vaccine Decision Making and Parent-Provider Trust.”¹⁶ Seven focus groups were conducted with vaccine-hesitant parents in a single health plan in Colorado. On the basis of findings from the focus groups, the investigators developed a survey that was mailed to a stratified sample of parents who accepted, delayed, or refused vaccinations for one of their children. The study included a review of health records to target, sample, and compare distinct groups of parents on the basis of their actual vaccine behavior and identified parents who delayed and those who refused vaccinations. Strengths of this study include a good response rate of 52%. The investigators also report a subanalysis comparing responders with nonresponders, which did not reveal any significant differences and provides valuable information when interpreting the results to minimize selection bias. Performing focus group interviews allowed better understanding of the patient’s perceptions of the providers’ knowledge and balanced vs unbalanced information provided to the parents. Combining qualitative and quantitative data as performed in this study is useful in designing meaningful survey questions to address research topics and guide future targeted interventions for specific populations. Limitations of these types of surveys include the fact that these are more time-consuming, and funding is usually needed in order to conduct these studies because resources are required for training and conducting focus groups.

MAJOR TAKE-HOME POINTS

Researchers embarking on a survey in healthcare epidemiology or antimicrobial stewardship must consider multiple issues (outlined in a checklist in Table 4). All surveys require a specific research question and planning for the type of subjects who will form the survey population in order to ensure that the survey population is the correct one for the questions of interest. An analysis plan should be developed that takes into account assessment of response bias. For new questionnaires, it is critical that the guidelines for layout, question design, formatting, and pilot testing area be followed.

Table 4.

Checklist of Key Considerations When Developing a Survey in Healthcare Epidemiology

What is the research question you hope to answer? Establish the study hypothesis before initiation of the study. Is there an existing survey that can be used or adapted? Search the literature for existing survey tools that might be adapted or used as is (with the necessary permissions). How will the survey be deployed? Think about your audience and the information you want to obtain, along with the best way to obtain the information whether phone, mail, online, or face to face. What type of subjects or groups will form the survey population? Clearly define and identify the type of individual subjects or groups you will want to survey and make decisions about inclusion and exclusion criteria. What about developing a new questionnaire? Be sure that guidelines for layout, question design, comprehensibility, order of questions, etc., have been followed. What outcomes will be evaluated? Along with your study hypothesis, determine primary and secondary outcomes before developing survey questions. What about a pilot survey? Your survey should be tested on a pilot sample of members of the target audience to ensure questions of interest are easily understood and the order and length of questions are reasonable. How will the results be analyzed? Develop a statistical analysis plan before collection of data. We strongly recommend involving a statistician at the start of your study. How will nonresponse bias be assessed? An analysis plan should be developed that takes into account assessment of response bias. The statistician can assist with nonresponse bias. How will the results be reported? Report all study findings along with details related to subjects (contacting subjects, consents, response rates, methods used) and be sure to include a description of the survey tool and the testing of the tool.

CONCLUSIONS

Surveys in healthcare epidemiology and antimicrobial stewardship research are essential to advance these fields and to generate and test hypotheses, and are likely to remain a frequently utilized research study design. Often, a survey is a key step toward collecting data on practices and perceptions and can help identify potential interventions to be tested in a randomized controlled trial. Despite the limitations inherent to surveys, the examples described above have provided important information on infection prevention that could not practically have been generated any other way. Low response rate issues plague most surveys, and there is no easy solution. However, planning for that eventuality by accounting for it in the sample size calculation and collecting some information on the nonrespondents may mitigate this limitation to some extent. As institutions come under increasing scrutiny for healthcare-associated infection prevention practices and rates, social desirability bias may become more important. Qualitative research methods to gain depth in survey responses may pose a solution in such cases. Future surveys in healthcare epidemiology should take these issues into consideration.