Abstract
Objective
To understand public attitudes about and recommendations to address antibiotic overuse by employing public deliberation (a method for eliciting informed input on value‐laden issues).
Data Sources/Study Setting
Participants in 24 Community Deliberation groups (CD; n = 263), four Citizens’ Panel groups (CP; n = 96), and a control group (n = 348). Data were collected in 2012 in four U.S. locations.
Study Design
Using mixed methods, we analyzed quantitative and qualitative data from a randomized control trial.
Data Collection/Extraction Methods
Using pre/postdeliberation surveys, we compared CD and CP participant attitude changes regarding antibiotic use to the control group. We analyzed deliberation transcripts using qualitative techniques to provide context for survey results.
Principal Findings
Compared to control group participants, CD and CP participants had a larger postdeliberation shift in attitudes toward support of government limits on when doctors can prescribe antibiotics. Participants described unawareness about antibiotic overuse and called for education. When discussing prescription limits, participants debated tensions between preserving patient/doctor autonomy and protecting society from antibiotic‐related harms. Participants saw patient, physician, and government roles in antibiotic stewardship policies/programs.
Conclusion
When informed about individual and social consequences of antibiotic overuse, patients may be more receptive to antibiotic prescription limits. Community‐physician‐government partnerships are needed to create solutions.
Keywords: antibiotic stewardship, evidence‐based practice, health promotion, patient engagement, public deliberation
1. INTRODUCTION
Antibiotic overuse in health care is widespread and well‐documented. Each year U.S. physicians write about 47 million unnecessary antibiotic prescriptions in outpatient settings and emergency departments.1 Additionally, 30 percent of all oral antibiotics prescribed in outpatient settings may be inappropriate.2 Most of these unnecessary prescriptions are for respiratory conditions (eg, colds or bronchitis), which are usually caused by viruses and cannot be treated with antibiotics.1 About half of the antibiotics prescribed for acute respiratory conditions may be unnecessary.1, 2 Children, especially those under 2 years of age, are particularly likely to receive antibiotic prescriptions.1 Although antibiotic use among children has declined over the past decade, evidence suggests that this trend has reached a plateau and that antibiotic overuse among children is still quite common.1, 3, 4, 5
Widespread antibiotic overuse has important consequences.1 Antibiotic use increases the risk that organisms will develop antibiotic resistance and that future infections will not respond to antibiotic treatment.1 For example, methicillin‐resistant Staphylococcus aureus (MRSA) is resistant to many different antibiotics.6 MRSA infections can be life‐threatening and have become increasingly common.6 Antibiotic use also increases the risk of other serious infections, such as Clostridium difficile, which can be life‐threatening and difficult to treat.7 Recent estimates suggest that treatment for antibiotic‐resistant infections costs the U.S. over $2 billion dollars annually.8 Overusing antibiotics also unnecessarily places patients at risk for antibiotic‐related side effects (eg, diarrhea) and allergic reactions.1
Previous research has identified several factors that contribute to antibiotic overuse. Providers report their own clinical uncertainty about whether illnesses will respond to antibiotics, perceived patient/parent demand for antibiotics, and—in some cases—limited knowledge about consequences of antibiotic overuse.9, 10, 11, 12 Limited patient/parent awareness of relevant risks may also contribute to antibiotic overuse.13, 14 And, parents may perceive a viral diagnosis—and a physician's subsequent refusal to prescribe antibiotics—as trivializing their concerns over their child's illness.15
Interest is growing in efforts to improve public awareness of this issue and to develop solutions for reducing antibiotic overuse in various medical settings.1, 16, 17, 18, 19 For example, the Centers for Disease Control and Prevention (CDC) sponsors an annual antibiotic awareness week and has developed resources to support stewardship efforts in outpatient, hospital, and nursing home settings.1, 20, 21, 22 Medical and public health organizations also call for greater public input and involvement in research, intervention development, and policy making efforts that address important issues such as antibiotic use.23, 24, 25, 26
Public deliberation is one method for eliciting informed public input on value‐laden, complex topics such as antibiotic overuse. In public deliberation, participants learn about a topic and make informed recommendations about issues that cannot be solved by simple technical solutions.27, 28, 29, 30, 31 Participants receive information covering multiple aspects of and perspectives on the topic, often through written educational materials and interactions with experts.31 Facilitators encourage participants to explain their perspectives, listen to others’ perspectives, consider all evidence presented, and collaborate to develop policy recommendations that benefit society.
Although previous studies have assessed public knowledge and attitudes about antibiotic overuse,13, 32, 33, 34 no studies to our knowledge have sought informed public input regarding policy recommendations to address this problem. Furthermore, no studies to our knowledge have assessed how attitudes about antibiotic use may change after participants engage in public deliberation. We used data from a randomized control trial (RCT) of public deliberation to address these gaps in the literature and gather informed public input about reducing antibiotic overuse. For this analysis, our research questions were as follows:
What are public perceptions and attitudes regarding antibiotic overuse?
How, if at all, do attitudes regarding antibiotic overuse shift after participants engage in public deliberation?
What solutions does an informed public recommend to reduce antibiotic overuse?
2. METHODS
We analyzed data from the Community Forum Deliberative Methods Demonstration, a five‐arm RCT conducted in 2012. The overarching goals of this RCT were to evaluate the effectiveness of public deliberation and elicit public input about whether and how evidence of medical effectiveness should be used in medical decision making. We randomly assigned participants to one of four different deliberative methods or to a control group that only received written educational materials. Each deliberative method varied in number of participants, session length, mode (online, in‐person, or both), and use of experts who provided background information and varying opinions to inform discussions. These variations allowed us to compare outcomes across deliberative methods in the larger study. Details about each deliberative method and the overall RCT are reported elsewhere.30, 31, 35, 36, 37, 38 All authors were part of the original team that conducted the Community Forum Deliberative Methods Demonstration. For this article, we analyzed pre‐ and postdeliberation surveys and transcripts from the two deliberative methods that discussed antibiotic overuse—Citizens’ Panel (CP) and Community Deliberation (CD).
2.1. Participant eligibility criteria
Participants were eligible for the study if they were 18 years old or older, comfortable reading and conversing in English, and never employed as a health care professional. Participants were required to have Internet access because CD required online participation (described below). Participants resided in one of four study locations: Chicago, Illinois; Sacramento, California; Silver Spring, Maryland; and Durham, North Carolina. We recruited participants to match the demographics of each location in terms of age, gender, race, and ethnicity according to U.S. Census estimates. We selected these locations to recruit a diverse participant sample that came from various sociodemographic backgrounds. The American Institutes for Research Institutional Review Board approved this study.
2.2. Description of Citizens’ Panel and Community Deliberations
CP participants deliberated in person over 2.5 consecutive days (two 8‐hour days and one 4‐hour day). Approximately 24 people participated in each CP group. CD participants deliberated in person over two sessions, each 2.5 hours long, one week apart. Approximately 12 people participated in each CD group. In the week between the in‐person CD sessions, participants interacted online through a discussion board (described below). We conducted four CP sessions and 24 CD sessions: one CP group and six CD groups in each location.
CP and CD participants deliberated about the following overarching question: “Should individual patients and/or their doctors be able to make any health decisions no matter what the evidence of medical effectiveness shows, or should society ever specify some boundaries for these decisions?” We did not ask participants to reach consensus about this question to maximize free‐ranging discussion.
To help participants prepare for deliberations, we provided written materials about medical evidence, comparative effectiveness research, health care quality, and U.S. health care costs. Control group participants also received these written materials. These materials were designed to provide neutral and unbiased background information to inform deliberation. Before distributing materials, we conducted three rounds of consumer testing to assess whether people understood the materials and viewed them as unbiased.
Facilitators—who had backgrounds in health care and prior experience with public deliberation and/or other qualitative research methods (eg, focus groups)—received a two‐day training to prepare for deliberations. This training covered topics such as encouraging equal participation and expression of opposing views while remaining impartial.31, 39
CP, CD, and control group participants received a case study about antibiotic overuse among children with upper respiratory infections. CP and CD participants discussed this case study with each other, whereas control group participants read the materials on their own. The case study had two parts. The first part presented information about how overusing antibiotics contributes to antibiotic resistance and may harm individual children as antibiotics may not work as well in the future to stop bacterial infections. This first part also described reasons why doctors may prescribe antibiotics unnecessarily (eg, pressure from parents or beliefs that no harm arises from prescribing antibiotics). The second part of the case study presented information about societal consequences of antibiotic overuse, such as increasing frequencies of MRSA infection. This part of the case study facilitated participant discussions about when—if at all—societal interests should take priority over individual patient/physician desire for antibiotics.
CD participants discussed both case study parts during the first session. Participants continued discussing the case study using an online discussion board with each other and with two clinician experts between the first and second session. Each expert presented different sides of the antibiotic overuse issue and answered participant questions. When CD participants reconvened a week after the first session, they continued discussing the antibiotic case study at the beginning of the second session.
Because CP had a larger agenda with more discussion topics than CD,31 CP participants discussed this case study for less time than CD participants. Accordingly, CP participants discussed both parts of the case study together and focused on whether the societal harm that might result from antibiotic overuse should warrant rules that limit when antibiotics can be prescribed. CP participants discussed this case study on day 1 of their deliberations and did not discuss the case study with experts or use an online discussion board. Table 1 presents an overview of the CD and CP methods.
Table 1.
Overview of the Community Deliberation and Citizens’ Panel methods
| Characteristic | Community Deliberation (CD) | Citizens’ Panel (CP) |
|---|---|---|
| Number of groups | 24 groups | Four groups |
| Average number of participants per group | 12 participants per group | 24 participants per group |
| Structure and intensity | 6 h total; two 2.5‐h sessions 1 wk apart with 1‐h average online time between sessions | 20 h total over 2.5 consecutive days; two 8‐h days and one 4‐h day |
| Mode of communication | Face to face; asynchronous online communication between sessions | Face to face |
| Facilitation | Active facilitation by individuals trained on topics such as encouraging equal participation, eliciting opposing viewpoints, and remaining impartial. Facilitators had backgrounds in health care but were not content experts in antibiotic overuse | |
| Use of background materials | Participants received written materials about medical evidence, comparative effectiveness research, health care quality, and U.S. health care costs. Materials were designed to provide neutral and unbiased background information to inform deliberation | |
| Discussion of case study about antibiotic overuse | Discussed during the first session in person and on the online discussion board. Discussed again at the beginning of the second session. Facilitators encouraged participants to weigh the medical evidence about antibiotic overuse with parent and physician autonomy to use antibiotics. Facilitators also prompted discussion about whether the societal harm that might result from antibiotic overuse should warrant rules that limit when antibiotics can be prescribed | Discussed on the first day of deliberation for less time than CD participants. Facilitators prompted discussion that focused on whether the societal harm that might result from antibiotic overuse should warrant rules that limit when antibiotics can be prescribed |
| Use of content experts for antibiotic‐related discussion | In the week between the face‐to‐face sessions, participants communicated with two content experts via an online discussion board about antibiotic overuse. Experts presented different sides of the antibiotic overuse issue and answered participant questions | CP participants did not discuss the antibiotic case study with content experts |
| Other case studies/topics discussed during deliberation that are not included in this analysis a | Hospital quality at high and low volume facilities | Hospital quality at high and low volume facilities, obesity treatment and prevention, heart disease treatment, prevention of a fictional illness |
aThese additional case studies were designed to help participants grapple with the complex overarching question posed in the larger study. Additional details about these case studies are available elsewhere.31
2.3. Quantitative data collection and analysis
We used pre‐ and postdeliberation survey data to assess whether CP and CD participant attitudes regarding antibiotic use shifted after deliberation when compared to the control group.30, 31 We conducted two rounds of cognitive interviews with consumers to assess participant understanding of the survey and to improve item validity. Experts also reviewed survey items. Additional details about this survey are provided elsewhere.30, 31 Participants completed the presurvey before receiving written educational materials; they completed the postsurvey approximately 2 weeks after deliberation. Control group participants completed both surveys at the same time as deliberation participants in each geographic location.
Participants responded to three items regarding antibiotic use: (a) people should be able to get an antibiotic if they want it, even if it might not help, (b) the doctor should be able to prescribe an antibiotic to treat infections, even if there is a small chance it would help, and (c) the government should limit when doctors can prescribe antibiotics. Participants responded using a five‐point Likert scale ranging from 1 = disagree strongly to 5 = agree strongly. We reverse‐coded the first two items so higher scores indicated stronger agreement with limiting antibiotic use/prescriptions.
We used SAS version 9.2 to conduct an analysis of covariance (ANCOVA) comparing pre‐ and postsurvey attitude changes between the CP, CD, and control groups among participants who completed both surveys. We controlled for presurvey outcome scores, overall health status, health care system experience, gender, age, marital status, education, employment status, bilingual status, and geographic location.
The randomization process led to an approximately equal distribution of participants in each deliberative method. Yet, the process resulted in a different distribution of participants in the control group with respect to income and racial/ethnic background. In the larger study, we also observed health insurance status differences between deliberation and control groups. To account for this imbalance, we weighted participants’ responses to the U.S. census distribution on insurance status, income, and race/ethnicity and applied these weights in ANCOVA analyses.31
2.4. Qualitative data collection and analysis
All CP and CD sessions were recorded and professionally transcribed. Everyone who participated in a CP or CD group was included in the qualitative analysis, regardless of whether they completed both the presurvey and postsurvey. We uploaded transcripts into NVivo version 9, a qualitative analysis software program. We developed an initial codebook based on notes from pilot deliberations, which included codes such as patient/provider education and protection from harm. Four trained research analysts coded deliberation transcripts and reached 80 percent intercoder agreement.40 While coding, analysts met three times per week to discuss the process and suggest emerging themes or new codes.
Analysts used output from codes to write memos using well‐established qualitative analysis techniques (eg, by identifying main concepts, patterns, and relationships between concepts and by exploring exceptions/alternative explanations to test findings).41, 42 We reviewed the memos and transcripts to describe how participants responded to the antibiotic case study.
2.5. Mixed methods integration
We employed a concurrent mixed methods research process, where quantitative and qualitative data were collected and subsequently analyzed in a similar timeframe.41, 43 We integrated the findings using a narrative approach.41 We used the qualitative findings to expand on and provide context for quantitative results.
3. RESULTS
Ninety‐seven people participated in CP, and 96 of these participants completed both the presurvey and postsurvey. A total of 269 people participated in CD, and 263 of these participants completed both surveys. A total of 348 control group participants completed both surveys. We recruited a diverse sample, including in terms of gender, race/ethnicity, educational attainment, and income. Table 2 presents participant demographics.
Table 2.
Demographics of participants in the Citizens’ Panel (CP), Community Deliberation (CD), and Control Groups a
| Characteristic | CP (n = 96) | CD (n = 263) | Control (n = 348) | χ2 (P‐value) |
|---|---|---|---|---|
| Gender | ||||
| Female | 55 (57%) | 144 (55%) | 196 (56%) | 0.24 (P = 0.89) |
| Age | ||||
| Mean ± SD | 48.89 ± 12.96 | 47.74 ± 14.06 | 47.57 ± 13.24 | 0.44 (P = 0.80) |
| Under 65 y | 83 (86%) | 229 (87%) | 308 (89%) | |
| 65 y and older | 13 (14%) | 34 (13%) | 40 (11%) | |
| Race | ||||
| White | 47 (49%) | 159 (61%) | 252 (72%) | 24.07 (P < 0.0001) |
| Black or African American | 43 (45%) | 87 (33%) | 75 (22%) | |
| Other race | 6 (6%) | 17 (6%) | 21 (6%) | |
| Ethnicity | ||||
| Hispanic (any race) | 10 (10%) | 30 (11%) | 17 (5%) | 9.43 (P = 0.009) |
| Bilingual status | ||||
| Speaks language other than English at home | 6 (6%) | 27 (10%) | 26 (8%) | 2.17 (P = 0.33) |
| Education b | ||||
| Some high school, but did not graduate | 1 (1%) | 1 (0%) | 2 (1%) | 14.78 (P = 0.06) |
| High school graduate or GED | 21 (23%) | 47 (19%) | 39 (12%) | |
| Some college or 2‐y degree | 34 (37%) | 80 (32%) | 107 (32%) | |
| Four‐year college degree | 19 (21%) | 64 (26%) | 96 (29%) | |
| More than 4‐y college degree | 16 (18%) | 56 (23%) | 93 (28%) | |
| Employment status | ||||
| Employed | 56 (58%) | 164 (62%) | 231 (66%) | 2.96 (P = 0.56) |
| Unemployed | 19 (20%) | 49 (19%) | 62 (18%) | |
| Retired or unable to work | 21 (22%) | 50 (19%) | 55 (16%) | |
| Marital status | ||||
| Never married | 27 (28%) | 66 (25%) | 87 (25%) | 1.66 (P = 0.80) |
| Married or living with a partner | 55 (57%) | 146 (56%) | 203 (58%) | |
| Separated, divorced, or widowed | 14 (15%) | 51 (19%) | 58 (17%) | |
| Self‐rated health status | ||||
| Excellent, very good, or good | 80 (83%) | 232 (88%) | 303 (87%) | 1.48 (P = 0.48) |
| Fair or poor | 16 (17%) | 31 (12%) | 45 (13%) | |
| Health care system experience | ||||
| Seen a doctor three or more times in past 12 mo for the same condition c | 39 (41%) | 92 (35%) | 110 (32%) | 2.87 (P = 0.24) |
| Has a close friend or family member who has seen a doctor three or more times in past 12 mo for the same condition c | 61 (64%) | 156 (59%) | 201 (58%) | 1.05 (P = 0.59) |
| Health insurance status | ||||
| Insured | 71 (74%) | 181 (69%) | 260 (75%) | 2.73 (P = 0.25) |
| Income (annually) | ||||
| $29 999 or less | 19 (20%) | 57 (22%) | 57 (16%) | 14.53 (P = 0.02) |
| $30 000‐$59 999 | 35 (36%) | 84 (32%) | 90 (26%) | |
| $60 000‐$100 000 | 28 (29%) | 70 (27%) | 103 (30%) | |
| More than $100 000 | 14 (15%) | 52 (20%) | 98 (28%) | |
aDemographic characteristics are presented for participants who completed both the pre‐ and postsurvey.
bAmong participants age 25 and older.
cNot including pregnancy.
Our analysis highlighted participant desires for antibiotic‐related policies and programs that preserve patient/physician autonomy while also protecting society from harm. Below, we describe how these findings arose in the quantitative and qualitative data.
3.1. Survey results: attitudes regarding antibiotic use
At baseline, participants disagreed that the government should limit when doctors can prescribe antibiotics (premean for CP = 2.0, CD = 2.0, and control = 1.9 on the five‐point Likert scale). Participants on average neither agreed nor disagreed that the doctor should not be able to prescribe an antibiotic if there is a small chance it would help (premean for CP = 2.6, CD = 2.5, and control = 2.4 on the five‐point Likert scale). Yet, participants on average agreed at baseline that people should not be able to get an antibiotic if it might not help (premean for CP = 3.9, CD = 3.5, and control = 3.6 on the five‐point Likert scale).
After deliberation, CP and CD participants had a larger shift in attitudes toward support of government limits on when doctors can prescribe antibiotics when compared to the control group (β = 0.40, P < 0.01 for CP and β = 0.57, P < 0.01 for CD). However, attitudes did not significantly shift for the other two items. Table 3 summarizes these results.
Table 3.
Effect of deliberative methods compared to control group on attitudes toward the use of medical evidence to limit antibiotic use
| Outcomea | Method | Premean | Postmean | β | P‐value |
|---|---|---|---|---|---|
| People should [not] be able to get an antibiotic if they want it if it might not helpb | Control | 3.6 | 3.6 | ||
| CP | 3.9 | 3.9 | 0.16 | 0.34 | |
| CD | 3.5 | 3.7 | 0.11 | 0.24 | |
| The doctor should [not] be able to prescribe an antibiotic to treat infections if there is a small chance it would helpb | Control | 2.4 | 2.6 | ||
| CP | 2.6 | 2.8 | 0.19 | 0.17 | |
| CD | 2.5 | 2.7 | 0.17 | 0.06 | |
| Government should limit when doctors can prescribe antibiotics | Control | 1.9 | 2.1 | ||
| CP | 2.0 | 2.6 | 0.40 | <0.01 c | |
| CD | 2.0 | 2.8 | 0.57 | <0.01 c |
Premean indicates the mean score prior to deliberation; postmean indicates mean score postdeliberation; the estimate is the difference between the pre/postchange for the deliberative groups and the pre/postchange for the control group.
Abbreviations: CD, Community Deliberation; CP, Citizens’ Panel.
Outcomes were assessed using a five‐point Likert scale ranging from 1 = disagree strongly to 5 = agree strongly.
Item was reverse‐coded so that a higher score indicates stronger agreement with limiting antibiotic use/prescriptions.
Indicates a significant difference between deliberative method and control at p < 0.05.
3.2. Qualitative results: participant perceptions about and recommendations to reduce antibiotic overuse
Qualitative results highlight how participants debated tensions between preserving patient/physician autonomy and protecting society from antibiotic‐related harms. With one exception (described below), themes were consistent in CP and CD. Also, themes that emerged during CD online discussions mirrored those that emerged during in‐person discussions. Table 4 summarizes key themes and subthemes that arose from the qualitative data.
Table 4.
Key qualitative themes, tensions, and participant quotes
| Key themes and tensions | Description and subthemes | Example quotes |
|---|---|---|
| Participant unawareness about antibiotic overuse | Participants described initial unawareness about antibiotic overuse but largely agreed that it is a problem after learning about the issue in deliberation. However, a minority of participants initially questioned whether antibiotic overuse causes harm. | “I think it's a lack of education on my part. I didn't know. I had no clue it was dangerous.” (CD group in CA) |
| Participant 1: “…I think it's better to be safe than sorry. If the kid's sick and you start with the antibiotic, kid's better in a couple days, no harm done.” | ||
| Participant 2: “But is there no harm done?” | ||
| Participant 1: “I don't know. I don't know what the research actually is.” | ||
| Participant 2: “We're being told that it's harm.” | ||
| (CD group in MD) | ||
| Patient/physician education as a potential solution to antibiotic overuse | Participants collectively called for patient/physician education about antibiotic overuse through methods such as public service announcements and patient‐provider conversations. | “I don't think all patients are necessarily educated enough to make the decision. At the same time, I don't think all doctors are making the correct recommendation…” (CP group in CA) |
| “I do believe there are some doctors that don't keep their level of education up and so they're still using old methods or old philosophies in treating illnesses.” (CD group in MD) | ||
| “I think the doctor has got to be both the health care provider and also the educator.” (CD group in MD) | ||
| “The education could come through public broadcasting, PBS, television or something.” (CP group in CA) | ||
| Patient responsibility to seek education and make informed decisions | Participants noted that education is especially critical so that patients can fulfill their responsibility to make informed health care decisions. | “You have to be your own advocate. You try to get out what your options are and go home and do some research. And I've disagreed with my doctors several times.” (CD group in NC) |
| “… People in this country need to stop thinking of themselves as patients. We're customers. So you need to do the research and go to your doctor and know what you're talking about. Just like you take your car to a mechanic, if you don't know what you're talking about you may not get the service you want. You go in as a customer, you say, ‘I looked this up on the Internet. These are the issues I have’. And you talk to the doctor informed. Then you can make informed decisions, I think get a better decision for you or your child.” (CD group in CA) | ||
| Tension between the theme of physician responsibility to make appropriate prescribing recommendations and the theme of holding doctors accountable for their actions | Participants described physicians’ responsibility to appropriately prescribe antibiotics but recognized that individual physicians might act inappropriately. Participants offered ways to hold physicians accountable for this responsibility. Examples include oversight or guidelines from medical authorities and professional associations, prescription audits or monitoring, and physician reprimands for inappropriate prescribing. | “If the doctor knows that he's going to be held accountable for making bad decisions such as the infections or prescribing the antibiotics in the wrong case he would be less likely to do a bad job of that.” (CD group in CA) |
| “If this is a proven fact that the antibiotics are becoming resistant because of over‐prescribing then the doctors need to make that decision and [the] ones that are not… need to be fined…” (CP group in CA) | ||
| “If [doctors are] just quick to write a prescription and say ‘Oh, antibiotic, antibiotic, antibiotic…’ then that should be some kind of like red flag to the government to kind of investigate that. Or they have stricter symptoms that they have to go by.” (CD group in IL) | ||
| “When we do our taxes sometimes, we can just be randomly audited. So if they had it to where doctors could realize that they could be randomly audited or checked up or investigated, just them individually, then maybe they would make sure that they…were doing things right.” (CD group in CA) | ||
| Tension between the theme of protecting children from immediate and future harms and the theme of preserving patient/physician autonomy | Participants wanted to protect children from harm, including harm that might result from failing to get needed antibiotics. Participants also wanted to preserve patient/physician autonomy in making decisions about antibiotic use. Tensions arose when participants considered how the risk of immediate harms versus future consequences (eg, the risk of later developing antibiotic‐resistant infections) should factor into decision making. | If my child's sick and the doctor is prescribing an antibiotic and I know that antibiotic is going to help my child, I'm not too concerned about what may or may not happen ten years from now. I'm only concerned about getting my child well.” (CD group in MD) |
| “What if someone was bitten by a tic. Within 24 h they start having fevers and rashes. They go to the doctor and they don't test positive for bacteria and they keep getting worse. And it's Lyme's disease but the test didn't show it immediately. They had to wait like a week or so, so this person by that time could be terribly damaged by the bacteria, whereas the doctor had every reason to believe that this could be caused by Lyme disease… Some of these bacterias if you don't treat them right away they have very serious consequences. But if you treat them within the first few hours it turns into nothing.” (CD group in NC) | ||
| Participant 1: “We're more informed and know the MRSA virus and personally I was affected by that… My mother‐in‐law died from that. I'm very sensitive to the overuse of antibiotics. But I also on the flip side have friends that every time they're sick… [with] the sniffles, they're going to call, whether for them or their kids and they want something….” | ||
| Participant 2: Give me something now. | ||
| Participant 1: That's kind of what this is saying… who should make that call? | ||
| Participant 2: I really think the… doctors and the patients or the parents… should have a talk about it. Communication… | ||
| Participant 3: “If you know… the long term results is not going to be a good one then… sometimes the parent needs to step back.” | ||
| (CP group in CA) | ||
| Tension between the theme of appropriate regulation and the theme of preserving patient/physician autonomy drove participant desires for flexible antibiotic regulation that allows exceptions when needed | Participants recognized that regulations might be appropriate, but worried that antibiotic regulation may result in unintended consequences, such as reduced physician autonomy to make recommendations based on their training. Accordingly, participants at times described ideas for flexible limits that would allow necessary exceptions (eg, prescription oversight and enforcement of guidelines that allows for exceptions or appeals when a physician's judgment deems an antibiotic necessary). | “The doctors are educated and they make their recommendations and they should be making them based on the information at hand…. If their hands just continually get tied with more and more and more regulations, they'll be less able to do that.” (CD group in NC) |
| “There should be restrictions… But the restrictions should be made in the doctor's office between the doctor and the patient where the doctor can refuse to prescribe and say, ‘No, you don't need that. You want it. I'm not going to prescribe it for you.’ But if the doctor thinks it's necessary and the patient agrees, there should be no outside forces restricting that medication.” (CP group in IL) | ||
| “My opinion would be the medical evidence should be the guideline… If it is a viral infection, you don't give antibiotics… with always the exception… You always have to have that exception. But if it's viral, you don't give an antibiotic.” (CP group in NC) | ||
| “When you say stricter rule, I don't think that assumes all the tying the doctor's hands… I think stricter would be a, more of a form of having a little more control. Right now it's really no control as to how much antibiotic they're using… OK good example, certain medications the insurance companies may not pay for, say a cancer med. However, if you do an appeal and you can prove why it's needed, the necessity of it, then that appeal could possibly overturn. So when we say – they say stricter rules am I misunderstanding that they're not saying totally tie the doctor's hands but let more force or more rules be enforced versus so much freedom.” (CD group in NC) | ||
| Support for antibiotic regulation varied among participants and throughout deliberation | Some participants were initially comfortable with regulating antibiotic prescriptions. Others gradually became somewhat more accepting of antibiotic regulation during deliberation. Participants’ support for regulation varied as they grappled with the themes of protecting children and society from harm, holding physicians accountable to antibiotic prescribing guidelines, and preserving patient/physician autonomy. | “After reading the information that the experts gave, just listening to their opinions and then I thought about everybody's side. I put myself in everybody's position and… I feel as though everybody has a role in this, so everybody should have a say, but it needs to be a say based on the guidelines, stricter rules and guidelines.” (CD group in IL) |
| “I change my mind… I'm still very resistant to it… Because [at the] last meeting I want[ed] to have the freedom to put my foot down and say I want this antibiotic. But… deaths of MRSA has just been rolling around in my head and maybe—it's hard because I don't like the idea of regulation and things like that. Although you're right, the vaccinations and things. I was vaccinated for polio and I think everyone should.” (CD group in MD) | ||
| Participant 1: “There are ways to regulate what physicians prescribe. If you look at narcotics, prescriptions are triplicate and they're controlled substance. So if it comes down to it, the government can regulate how a physician prescribes something.” | ||
| Facilitator: “Should they?” | ||
| Participant 1: “I think they should…” | ||
| Participant 2: “It's going to affect our whole human race. Future generations.” (CP group in CA) |
Abbreviations: CD, Community Deliberation; CP, Citizens’ Panel.
After discussing the first part of the case study, participants noted their lack of awareness. One participant explained, “I think it's a lack of education on my part. I didn't know. I had no clue it was dangerous,” (CD group in CA). Another participant echoed this sentiment:
We grew up to know, you get a toothache, ‘You got some antibiotics?’ That's the first thing we said… That's all you grow up knowing, antibiotic… My baby get a cold, he needs some antibiotics… They cure everything. (CP group in NC)
As deliberation progressed and participants learned more about potential harms from antibiotic overuse, they grappled with how to balance protecting children from harm while also protecting patient/physician autonomy. One participant drew a parallel to protecting children with peanut allergies in schools: “Because one child has a severe peanut allergy, nobody's allowed to bring a peanut butter sandwich to school. I want to protect all the children, but I don't know if I want to protect this one child at the loss to my own. So it's a lot harder when you slide the scales down,” (CD group in NC). Participants wanted to protect children from harm but struggled when weighing immediate harms to children (eg, having a child's illness worsen or last longer than necessary if parents/physicians cannot access antibiotics) with future consequences (eg, the risk of the child later developing an antibiotic‐resistant infection). One participant explained: “If my child's sick and the doctor is prescribing an antibiotic and I know that antibiotic is going to help my child, I'm not too concerned about what may or may not happen ten years from now. I'm only concerned about getting my child well,” (CD group in MD). Yet, another participant noted: “If you know… the long‐term results is not going to be a good one then… sometimes the parent needs to step back,” (CP group in CA).
In response to these competing priorities, participants deliberated about solutions. Generally, participants called for patient/physician education about antibiotic overuse (eg, via patient‐provider discussions or public service announcements). One participant noted, “I don't think all patients are necessarily educated enough to make the decision. At the same time, I don't think all doctors are making the correct recommendation…,” (CP group in CA). Participants also noted that education is especially critical so that patients can fulfill their responsibility to make informed health care decisions.
Although participants generally agreed that education could reduce antibiotic overuse, tensions arose when participants discussed setting limits (eg, through government regulations on antibiotic use). These tensions were prominent in the CD groups, where participants spent more time discussing the case study. Participants worried that regulating antibiotic use may not be feasible or result in unintended consequences, such as hindering physicians’ ability to treat patients. One participant noted, “The doctors are educated and they make their recommendations and they should be making them based on the information at hand…. If their hands just continually get tied with more and more and more regulations, they'll be less able to do that,” (CD group in NC). Participants worried that enacting regulations to prevent one type of harm could result in a “slippery slope” that sets precedence for more extreme, intolerable regulations in the future. Furthermore, participants worried that regulation would prevent children from receiving antibiotics when needed, such as when the parents (who “know [their] kid best”) and physicians agree antibiotics are necessary.
Some participants were comfortable from the outset with regulating antibiotic use and drew analogies to public health problems (eg, narcotic prescription control) where the government intervened to reduce potential harm. Other participants gradually became more accepting of antibiotic regulation during deliberation. Participants at times acknowledged changing their minds as they considered the public health benefits that resulted from other regulations (eg, vaccinations, traffic lights, and speed limits). One participant explained:
I change my mind… I'm still very resistant to it… Because [at the] last meeting I want[ed] to have the freedom to put my foot down and say I want this antibiotic. But… deaths of MRSA has just been rolling around in my head and maybe—it's hard because I don't like the idea of regulation and things like that. Although you're right, the vaccinations and things. I was vaccinated for polio and I think everyone should. (CD group in MD)
Participants who acknowledged the need for prescribing limits suggested that medical authorities and professional associations should provide oversight and guidelines related to antibiotic overuse. Participants called for increased education for and accountability of physicians to ensure appropriate antibiotic use (eg, through random audits or regular monitoring of physician prescribing practices). These participants also called for physician “reprimands” for inappropriate antibiotic prescriptions, which could involve action from medical associations and licensing agencies (eg, implementation of probation policies). Yet, participants noted that even when limits are set, physicians should be able to rely on their training/experience to make exceptions for necessary antibiotic prescriptions. As one participant noted:
There should be restrictions… But the restrictions should be made in the doctor's office between the doctor and the patient where the doctor can refuse to prescribe and say, ‘No, you don't need that. You want it. I'm not going to prescribe it for you.’ But if the doctor thinks it's necessary and the patient agrees, there should be no outside forces restricting that medication. (CP group in IL)
4. DISCUSSION
These results highlight participant desires for antibiotic stewardship policies and programs that preserve patient/physician autonomy while also protecting society from harm. In the presurvey, participants generally agreed that people should not be able to get an antibiotic if it might not help. And, during deliberation participants recognized that many people are unaware of antibiotic‐related risks and called for patient education about when antibiotics may cause harm. Therefore, it is unsurprising that after deliberation participants remained convinced that people should not be able to get antibiotics if it might not help.
We also found no significant attitude shifts regarding whether the doctor should be able to prescribe an antibiotic if there is a small chance it would help. Qualitative results suggest that participants viewed physicians as experts capable of making informed decisions about whether antibiotics are necessary. Yet, as deliberation progressed, participants called for physician education about this topic and grappled with desires to protect physician autonomy while also holding them accountable for inappropriate antibiotic prescriptions.
Conversely, deliberation participants’ attitudes shifted significantly toward support of government limits on when doctors can prescribe antibiotics, mirroring participant calls for increased antibiotic use oversight to protect society from potential harms. Despite these attitude shifts, participants still did not strongly agree after deliberation that the government should limit when doctors can prescribe antibiotics, which may reflect the belief that patients and physicians share responsibility for addressing overuse.
Indeed, these results collectively suggest that participants see roles for patients, physicians, and the government in addressing antibiotic overuse. For patients, this role may involve staying informed about this issue and considering potential benefits and harms when making decisions about antibiotic use. For physicians, this role may involve staying up‐to‐date about and complying with antibiotic prescribing guidelines and remaining accountable to medical authorities or government organizations that may monitor their prescribing practices. For the government, this role may involve funding awareness campaigns, providing antibiotic prescription oversight, and instituting penalties when appropriate. Participants desired programs that supported individuals in these roles, protected society from harm, and preserved patient/physician autonomy. For example, participants advocated for patient/physician education and flexible systems of physician accountability that ensure patients can receive necessary antibiotics (eg, prescription oversight that allows exceptions when a physician's clinical judgment deems an antibiotic necessary).
Our findings regarding limited public understanding about the potential harms of antibiotic overuse are reflected in other studies, which suggest that patients may lack awareness and/or may not view this issue as personally relevant.13, 14, 33, 44 Furthermore, two systematic reviews of interventions aimed at reducing antibiotic prescriptions for children with respiratory tract infections found that the most effective interventions simultaneously target parents and clinicians.45, 46 Deliberation participants echoed that interventions should not target patients or physicians alone but also noted that government stakeholders have roles in controlling this problem too.
Future work is needed to develop effective antibiotic stewardship policies/programs that consider patient, physician, and government knowledge and perspectives while preserving autonomy as much as possible. For example, community, physician, and government stakeholders might collaborate to design antibiotic stewardship programs. To enhance public understanding and buy‐in, these programs should aim for partnership and leadership with patients and strive to avoid dominance by physicians, government leaders, or other technical experts.24 Additionally, partnering with patients in program design could help address public concerns about the potential loss of patient/physician autonomy that may result from antibiotic stewardship efforts. Furthermore, community‐physician‐government partners can draw on existing evidence‐based resources (like the CDC's Core Elements of Outpatient Antibiotic Stewardship framework) to address the challenges they may face when designing programs.21
Future research should also explore whether attitudes about antibiotic overuse vary by participant characteristics (eg, race/ethnicity, age, and education) using quantitative and qualitative methods. Such analyses may, for example, identify populations that report stronger opposition to limits on antibiotic use as well as factors influencing this opposition (eg, mistrust of entities with the power to set limits, such as the government). Results from the larger study suggested that the size and direction of attitude shifts about the use of medical evidence in health care decision making were similar across demographic groups.35 However, African Americans had significantly smaller attitude changes than others regarding whether doctors and people should consider cost when making decisions. Future research is needed to understand whether demographic characteristics are similarly associated with attitude shifts about antibiotic use after individuals participate in public deliberation, educational interventions, or other interventions/programs frequently found in the literature. Results from such studies could have implications for health communication efforts that aim to disseminate messages and raise awareness about antibiotic overuse in diverse communities to advance health equity.
4.1. Limitations
Although we recruited a sample to match demographics in the four study locations, we did not intend to draw a nationally representative sample and results may not generalize to other populations. We also excluded participants without Internet access, which may affect generalizability. However, most U.S. households had Internet access at the time of data collection.47 Additionally, the study randomization process resulted in a different distribution of participants in CP, CD, and the control group with respect to health insurance status, income, and racial/ethnic background, which could bias results. However, we weighted participants’ responses to the U.S. census distribution with respect to these demographic variables and subsequently identified no significant differences between CP, CD, and the control group regarding baseline scores on the antibiotic survey items.31
We also did not purposively sample participants with recent exposure to this topic (eg, parents making a decision about antibiotic use for their child); results may have varied if the sample were limited to parents with experience making this decision. Furthermore, the antibiotic case study discussed some individual harms that children may experience from antibiotic overuse (eg, a higher risk that they will develop future antibiotic‐resistant infections), but it did not describe other more immediate harms (eg, diarrhea and other side effects from an unnecessary medication). Results may have varied if the case study and deliberations focused more on these immediate harms that children can experience from unnecessary antibiotics as participants identified protecting children from harm as a key impetus for considering limits on antibiotic use. Despite these limitations, our analysis included a large, diverse sample and highlighted in‐depth participant perspectives on antibiotic overuse.
5. CONCLUSION
Patients may resist antibiotic stewardship policies or programs that limit antibiotic prescribing because of the perceived threat to patient/physician autonomy and to what they perceive as necessary antibiotic access. However, patients who are informed about the limited clinical value of antibiotics for conditions such as upper respiratory infections and about the harms of overuse may be more receptive to government programs that limit antibiotic prescriptions. Such programs may be particularly effective when they are designed with patient, physician, and government partners. These partnerships may produce antibiotic stewardship programs/policies that are more publicly acceptable and that address the patient concerns highlighted in this study (eg, diminished patient/physician autonomy).
Supporting information
ACKNOWLEDGMENTS
Joint Acknowledgment/Disclosure Statement: This study was supported by the Agency for Healthcare Research and Quality (Contract No. 290‐2010‐00005CAHRQ). Support for Jennifer Richmond was provided by a grant from the Robert Wood Johnson Foundation Health Policy Research Scholars program (Grant Number 73921). The content of this paper is solely the responsibility of the authors and does not represent the official views of the funding agencies and organizations.
We thank the Community Forum Deliberative Methods Demonstration project team for their input and expertise in designing and evaluating the study. We also thank the study participants who taught each other and the authors so much.
Disclosures: None.
Richmond J, Mangrum R, Wang G, et al. An informed public's views on reducing antibiotic overuse. Health Serv Res. 2019;54:1283–1294. 10.1111/1475-6773.13175
REFERENCES
- 1. Centers for Disease Control and Prevention . Antibiotic Use in the United States, 2017: Progress and Opportunities. Atlanta, GA: Centers for Disease Control and Prevention; 2017. [Google Scholar]
- 2. Fleming‐Dutra KE, Hersh AL, Shapiro DJ, et al. Prevalence of inappropriate antibiotic prescriptions among US ambulatory care visits, 2010‐2011. JAMA. 2016;315(17):1864‐1873. [DOI] [PubMed] [Google Scholar]
- 3. Vaz LE, Kleinman KP, Raebel MA, et al. Recent trends in outpatient antibiotic use in children. Pediatrics. 2014;133(3):375‐385. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Hersh AL, Jackson MA, Hicks LA. Principles of judicious antibiotic prescribing for upper respiratory tract infections in pediatrics. Pediatrics. 2013;132(6):1146‐1154. [DOI] [PubMed] [Google Scholar]
- 5. Sarpong EM, Miller GE. Narrow‐ and broad‐spectrum antibiotic use among U.S. children. Health Serv Res. 2015;50(3):830‐846. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Zeller JL, Golub RM. MRSA infections. JAMA. 2011;306(16):1818. [DOI] [PubMed] [Google Scholar]
- 7. Surawicz CM, Brandt LJ, Binion DG, et al. Guidelines for diagnosis, treatment, and prevention of Clostridium difficile infections. Am J Gastroenterol. 2013;108(4):pp. 478–498; quiz 499. [DOI] [PubMed] [Google Scholar]
- 8. Thorpe KE, Joski P, Johnston KJ. Antibiotic‐resistant infection treatment costs have doubled since 2002, now exceeding $2 Billion annually. Health Aff (Project Hope). 2018;37(4):662‐669. [DOI] [PubMed] [Google Scholar]
- 9. Md Rezal RS, Hassali MA, Alrasheedy AA, Saleem F, Md Yusof FA, Godman B. Physicians’ knowledge, perceptions and behaviour towards antibiotic prescribing: a systematic review of the literature. Expert Rev Anti Infect Ther. 2015;13(5):665‐680. [DOI] [PubMed] [Google Scholar]
- 10. Coxeter P, Del Mar CB, McGregor L, Beller EM, Hoffmann TC. Interventions to facilitate shared decision making to address antibiotic use for acute respiratory infections in primary care. Cochrane Database Syst Rev. 2015. (11):Cd010907. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Heritage J, Elliott MN, Stivers T, Richardson A, Mangione‐Smith R. Reducing inappropriate antibiotics prescribing: the role of online commentary on physical examination findings. Patient Educ Couns. 2010;81(1):119‐125. [DOI] [PubMed] [Google Scholar]
- 12. Stivers T, Mangione‐Smith R, Elliott MN, McDonald L, Heritage J. Why do physicians think parents expect antibiotics? What parents report vs what physicians believe. J Family Practice. 2003;52(2):140‐148. [PubMed] [Google Scholar]
- 13. Price EL, Mackenzie TD, Metlay JP, Camargo CA Jr, Gonzales R. A computerized education module improves patient knowledge and attitudes about appropriate antibiotic use for acute respiratory tract infections. Patient Educ Couns. 2011;85(3):493‐498. [DOI] [PubMed] [Google Scholar]
- 14. Lee GM, Friedman JF, Ross‐Degnan D, Hibberd PL, Goldmann DA. Misconceptions about colds and predictors of health service utilization. Pediatrics. 2003;111(2):231‐236. [DOI] [PubMed] [Google Scholar]
- 15. Cabral C, Ingram J, Hay AD, Horwood J. “They just say everything's a virus”–parent's judgment of the credibility of clinician communication in primary care consultations for respiratory tract infections in children: a qualitative study. Patient Educ Couns. 2014;95(2):248‐253. [DOI] [PubMed] [Google Scholar]
- 16. Riggs KR, Ubel PA, Saloner B. Can appealing to patient altruism reduce overuse of health care services? An experimental survey. J Gen Intern Med. 2017;32(7):732‐738. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Coxeter PD, Mar CD, Hoffmann TC. Parents’ expectations and experiences of antibiotics for acute respiratory infections in primary care. Ann Family Med. 2017;15(2):149‐154. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Feldman MD. Altruism and medical practice. J Gen Intern Med. 2017;32(7):719‐720. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Guidos RJ, Infectious Diseases Society of America . Combating antimicrobial resistance: policy recommendations to save lives. Clin Infect Dis. 2011;52(Suppl 5):S397‐S428. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Centers for Disease Control and Prevention . The core elements of antibiotic stewardship for nursing homes. 2017. https://www.cdc.gov/longtermcare/prevention/antibiotic-stewardship.html. Accessed June 5, 2018.
- 21. Sanchez GV, Fleming‐Dutra KE, Roberts RM, Hicks LA. Core elements of outpatient antibiotic stewardship. MMWR Recommend Rep. 2016;65(6):1‐12. [DOI] [PubMed] [Google Scholar]
- 22. Centers for Disease Control and Prevention . Core elements of hospital antibiotic stewardship programs. 2015. https://www.cdc.gov/antibiotic-use/healthcare/implementation/core-elements.html. Accessed June 5, 2018. [DOI] [PMC free article] [PubMed]
- 23. Boivin A, Lehoux P, Lacombe R, Burgers J, Grol R. Involving patients in setting priorities for healthcare improvement: a cluster randomized trial. Implement Sci. 2014;9:24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Carman KL, Dardess P, Maurer M, et al. Patient and family engagement: a framework for understanding the elements and developing interventions and policies. Health Aff (Project Hope). 2013;32(2):223‐231. [DOI] [PubMed] [Google Scholar]
- 25. Domecq JP, Prutsky G, Elraiyah T, et al. Patient engagement in research: a systematic review. BMC Health Services Res. 2014;14:89. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Ewers T, Knobloch MJ, Safdar N. Antimicrobial stewardship: the role of the patient. Curr Treat Options Infect Dis. 2017;9(1):92‐103. [Google Scholar]
- 27. Abelson J, Forest PG, Eyles J, Smith P, Martin E, Gauvin FP. Deliberations about deliberative methods: issues in the design and evaluation of public participation processes. Soc Sci Med. 2003;57(2):239‐251. [DOI] [PubMed] [Google Scholar]
- 28. Delli Carpini MX, Cook FL, Jacobs LR. Public deliberation, discursive participation, and citizen engagement: a review of the empirical literature. Ann Rev Polit Sci. 2004;7:315‐344. [Google Scholar]
- 29. Burkhalter S, Gastil J, Kelshaw T. A conceptual definition and theoretical model of public deliberation in small face‐to‐face groups. Commun Theory. 2002;12(4):398‐422. [Google Scholar]
- 30. Carman KL, Mallery C, Maurer M, et al. Effectiveness of public deliberation methods for gathering input on issues in healthcare: results from a randomized trial. Soc Sci Med. 2015;133:11‐20. [DOI] [PubMed] [Google Scholar]
- 31. Carman KL, Maurer M, Mallery C, et al. Community Forum Deliberative Methods Demonstration: Evaluating Effectiveness and Eliciting Public Views on Use of Evidence. Rockville, MD: Agency for Healthcare Research and Quality; 2014. [Google Scholar]
- 32. Vaz LE, Kleinman KP, Lakoma MD, et al. Prevalence of parental misconceptions about antibiotic use. Pediatrics. 2015;136(2):221‐231. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Carter RR, Sun J, Jump RLP. A survey and analysis of the American Public's perceptions and knowledge about antibiotic resistance. Open Forum Infect Dis. 2016;3(3):ofw112. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Groshek J, Katz JE, Andersen B, Cutino C, Zhong Q. Media use and antimicrobial resistance misinformation and misuse: survey evidence of information channels and fatalism in augmenting a global health threat. Cogent Med. 2018;5(1):1460898. [Google Scholar]
- 35. Wang G, Gold M, Siegel J, et al. Deliberation: obtaining informed input from a diverse public. J Health Care Poor Underserved. 2015;26(1):223‐242. [DOI] [PubMed] [Google Scholar]
- 36. Maurer M, Mangrum R, Carman KL, et al. Setting boundaries: public views on limiting patient and physician autonomy in health care decisions. J Health Polit Policy Law. 2017. [DOI] [PubMed] [Google Scholar]
- 37. Carman KL, Maurer M, Mangrum R, et al. Understanding an informed public's views on the role of evidence in making health care decisions. Health Aff (Millwood). 2016;35(4):566‐574. [DOI] [PubMed] [Google Scholar]
- 38. Richmond J, Powell W, Maurer M, et al. Public mistrust of the U.S. Health Care System's profit motives: mixed‐methods results from a Randomized Controlled Trial. J Gen Intern Med. 2017;32(12):1396‐1402. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Carman KL, Heeringa JW, Heil SKR, et al. The Use of Public Deliberation in Eliciting Public Input: Findings from a Literature Review. Rockville, MD: Agency for Healthcare Research and Quality; 2013. [Google Scholar]
- 40. Kurasaki KS. Intercoder reliability for validating conclusions drawn from open‐ended interview data. Field Methods. 2000;12(3):179‐194. [Google Scholar]
- 41. Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs—principles and practices. Health Serv Res. 2013;48(6 Pt 2):2134‐2156. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42. Ryan GW, Bernard HR. Techniques to identify themes. Field Methods. 2003;15(1):85‐109. [Google Scholar]
- 43. Zhang W, Creswell J. The use of “mixing” procedure of mixed methods in health services research. Med Care. 2013;51(8):e51‐e57. [DOI] [PubMed] [Google Scholar]
- 44. Davis ME, Liu TL, Taylor YJ, et al. Exploring patient awareness and perceptions of the appropriate use of antibiotics: a mixed‐methods study. Antibiotics (Basel, Switzerland). 2017;6(4). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45. Vodicka TA, Thompson M, Lucas P, et al. Reducing antibiotic prescribing for children with respiratory tract infections in primary care: a systematic review. Br J Gen Pract. 2013;63(612):e445‐e454. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46. Hu Y, Walley J, Chou R, et al. Interventions to reduce childhood antibiotic prescribing for upper respiratory infections: systematic review and meta‐analysis. J Epidemiol Community Health. 2016. [DOI] [PubMed] [Google Scholar]
- 47. Ryan C, Lewis JM. Computer and Internet use in the United States: 2015. American Community Survey Reports. 2017.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials