Surveys of parents and clinicians concerning the minimally important difference of probiotic therapy for prevention of paediatric antibiotic-associated diarrhoea

Abstract

Objectives

To establish the minimally important difference (MID) that would prompt parents and clinicians to use probiotics for prevention of paediatric antibiotic-associated diarrhoea (AAD) and to obtain parent and clinician opinion about the most important outcomes in clinical trials of AAD.

Methods

In this survey, parents of children presenting to the emergency department of a Canadian tertiary care children’s hospital and paediatricians working in that hospital were approached. A range of potential MIDs were presented and participants selected one that they would require to use probiotics for AAD prevention. In addition, participants were asked to rate a list of outcomes they would consider to be important in clinical trials of AAD.

Results

In total, 127 parents and 45 paediatricians participated. About 51% (64/125) of parents and 51% (21/41) of clinicians responding to the MID question reported they would use probiotics if it reduced the risk of AAD by 39% (ie, reduce the risk of AAD from 19% to 12%). The most important outcomes to parents, in descending order, were need for hospitalisation, prevention of dehydration, disruption of normal daily activities, diarrhoea duration and physician revisit. Paediatricians considered need for hospitalisation along with physician revisit as the most important outcomes. They rated prevention of dehydration, diarrhoea duration and stool frequency as important outcomes as well.

Conclusion

There is good agreement between parents and clinicians regarding how effective probiotics would need to be in preventing AAD in order to warrant use. This information, along with outcomes perceived to be most important, will help in the design of future clinical trials.

Strengths and limitations of this study.

• This is the first study to seek parent and clinician opinions about minimally important difference of probiotic therapy for preventing paediatric antibiotic-associated diarrhoea.

• Face validity and comprehensibility of the survey were tested.

• Response rate of parents/guardians was very high. However, response rate of clinicians was low.

• Restricting our participants to English-speaking population and parents of children presenting to a children’s hospital emergency department might affect the generalisability of our findings.

Background

Probiotics are defined as ‘live microorganisms which, when administered in adequate amounts, confer a health benefit on the host’.^{1 2} Research shows a substantial increase in probiotic use in clinical and research settings and among the general public in the last three decades.³

According to a 2015 Cochrane systematic review,⁴ probiotics may be effective for prevention of antibiotic-associated diarrhoea (AAD) in children (pooled relative risk (RR)=0.46, 95% CI 0.35 to 0.61), AAD can be delayed up to 8 weeks after initiation of antibiotics.⁵ Its incidence varies considerably (5%–62%) depending on the patient population, setting, type and duration of antibiotics.^6–12 Although mild-to-moderate diarrhoea is more common, serious complications such as dehydration and Clostridium difficile infection can result.^{11 12} The proposed mechanism for the development of AAD is that antibiotics influence the gut microbial balance, altering its protective functions and leading to diarrhoea.¹³ AAD is particularly important in children as antibiotics are frequently prescribed in this population¹⁴ and they are more likely to develop dehydration from diarrhoea than are adults.

As the gold standard for determining treatment efficacy,¹⁵randomised clinical trials (RCTs) are powered to detect the difference or change in the outcome of interest between study groups.¹⁶ However, this difference or change must outweigh the risks, costs and inconvenience of the intervention in order to warrant implementation. The smallest difference or change that meets these criteria is called the minimally important difference (MID).^{17 18} MID also informs the sample size calculation of RCTs.¹⁷

Historically, MID was determined by healthcare providers; more recently, patient or parent input on MID is being sought.^{18 19} Recent calls to establish patient-determined MID are especially relevant for therapies that are accessed by consumers without a prescription (eg, probiotics).

To date, more than 20 RCTs^20–39 have studied the effectiveness and safety of probiotics for prevention of AAD in children. None of these studies reported seeking the perspective of children or parents about the most relevant outcomes and associated MID.

Different methods, including surveys, Delphi methods and interviews, can be used to elicit opinions about the change or difference in an outcome that is perceived to be important.^{16 40} Accordingly, we conducted a survey to establish the MID in diarrhoea incidence that would lead parents/guardians to use probiotic therapy for prevention of AAD in their children.

As our secondary objective, we also obtained the opinions of clinicians and compared them with the opinions of parents/guardians. Factors associated with the size of MID (demographics, previous familiarity and experience with probiotics and AAD) in each group were explored. Furthermore, parents/guardians and clinicians rated the importance of outcomes that should be measured in AAD trials, other than the risk of AAD.

Methods

Sampling frame and administration

Parents/guardians

We approached parents/guardians of children in the waiting room of the emergency department at the Stollery Children’s Hospital, a large urban tertiary care hospital in Edmonton, Canada. They were eligible if their children were less than 17 years old and had taken antibiotics at least one time in their lives. Exclusion criteria were inability to communicate in English or previous participation in the study. Participants were provided a paper-based survey (online supplementary appendix A) by a study team member (SKA) who obtained consent for participation and provided help to understand the questions as required.

Clinicians

We approached a convenience sample of general paediatricians and all sub-specialists from gastroenterology, infectious diseases and emergency medicine in active practice at the Stollery Children’s Hospital. Clinicians were given electronic surveys (online supplementary appendix B) using REDCap;⁴¹; paper surveys were provided to those who did not respond to the electronic surveys.

Development of survey

Validated surveys were developed based on the literature, discussion with experts, and consultation with parents and paediatricians. Clinical sensibility and pilot testing were performed on a group of parents (n=5) and clinicians (n=5) with diverse demographic characteristics to ensure face validity, comprehensiveness, clarity, acceptability and ease of administration of the surveys. The surveys (online supplementary appendices A and B) consisted of two sections: in the first section, we asked participants for their opinions and behaviour about probiotics. In the second section, we introduced a trade-off tool consisting of potential advantages and disadvantages of probiotic therapy.^{4 12 42} For parents/guardians, this was complemented by presentation of a scenario wherein the risk of developing AAD in children was shown to be 19% as stated in a 2015 systematic review.⁴ Then, a range of higher and lower MIDs were presented. These options were calculated based on the pooled RR of probiotics to reduce the incidence of paediatric AAD and the corresponding lower and upper limits of 95% confidence interval (pooled RR=0.46, 95% CI 0.35 to 0.61). We asked participants to select the MID that was closest to what they would require in order to use probiotics for AAD prevention. The rationale of presenting limited response options was to obtain the opinions of parents and clinicians for the range of treatment effect that was realistic and in keeping with the published literature. A research team member was available to respond to any questions that parents/guardians might have had and to make sure that they had a good understanding of the concept of the question. For parents/guardians, risks were expressed as frequencies per 100 patients to facilitate ease of understanding.⁴³ Positive and negative wording with corresponding visual illustration (ie, happy and sad faces) were used to promote clarity.⁴⁴ Format and questions of clinician survey were mainly adapted from the survey study carried out by Li et al.⁴⁵

Finally, we asked participants to score a list of outcomes they would consider important to be measured in clinical trials of AAD. We used the nine-point scale suggested by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) group to score the importance of outcomes.⁴⁶ In this scale, scores of 1–3 represents outcomes of limited importance, 4–6 important but not critical and 7–9 indicates outcomes that are of critical importance.

Sample size justification

A sample size of 122 parents/guardians and 44 clinicians would achieve 80% power to detect an effect size of 0.3 (medium effect) and 0.5 (large effect), respectively, using a 3 df Χ² test with a significance level (alpha) of 0.05, two-sided.⁴⁷ According to Cohen et al,⁴⁷ effect size is the measure of the magnitude of the Χ² that is to be detected.

Statistical analysis

Frequencies of MID estimates of parents/guardians and clinicians were reported as n (%). MID estimates derived from clinicians and parents/guardians were compared using Χ ² test. Participant opinions and behaviours were reported as frequencies for each question. A multinomial logistic regression model was conducted to determine factors associated with the size of MID in clinicians and parents/guardians. P value <0.05 was considered statistically significant. Data were analysed using SPSS V.16.0.⁴⁸

Patient and public involvement

Parents were involved in the comprehensibility and feasibility testing of the surveys. The results of the study reflects parents and clinicians views which can be used to designing and interpreting the findings of intervention studies of probiotic therapy for prevention of AAD.

Results

We approached 145 families and 125 clinicians of which 127 parents/guardians (87.5%) and 45 paediatricians (36%) responded. Lack of time or interest was the main reason of refusal among families, and respondents did not answer all questions (1%–13% missing values across different questions and participants). The mean age of children presenting to the emergency room on whose behalf their parents/guardians completed our survey was 6.5 years (66/124, 53% female). According to the parents/guardians, 39 out of 127 (31%) of the children had previous experience of AAD. Most of the responding clinicians were general paediatricians (17/39, 44%) or paediatric emergency medicine sub-specialists (15/39, 38%). Tables 1 and 2 show the general characteristics of responding parents/guardians and paediatricians, respectively.

Table 1.

Responding parent/guardian general characteristics

Child’s age (years), n=126
Mean (SD)	6.5 (4.9)
Child’s gender, n=124
Female	66 (53%)
Parent’s age (years), n=125
20 or less	8 (6%)
21–30	20 (16%)
31–40	57 (46%)
41–50	31 (25%)
Over 50	9 (7%)
Parent’s gender, n=126
Female	98 (77.8%)
Parent’s ethnicity, n=123
White/European/Caucasian	80 (65%)
Asian (East, Southeast)	15 (12%)
Middle Eastern/South or West Central Asian	8 (7%)
Black	6 (5%)
Latin American	4 (3%)
North American Aboriginal	5 (4%)
Other	5 (4%)
Parent’s education, n=124
Did not finish high school	6 (5%)
High school diploma	22 (18%)
Post-secondary education without a bachelor’s degree	39 (31%)
Bachelor’s degree or higher	57 (46%)

Categorical variables are presented as n (%).

Table 2.

Clinicians general characteristics

Gender, n=39
Female	19 (49%)
Specialty, n=39
General paediatricians	17 (44%)
Sub specialists
Paediatric emergency medicine	15 (38%)
Paediatric gastroenterology	5 (13%)
Paediatric infectious disease	2 (5%)
Years since graduation, n=39
Mean (SD)	10.05 (6.3)
Median (Q1, Q3)	10 (5, 15)
Number of AAD patients in a typical month, n=39
Mean (SD)	4.5 (3.8)
Median (Q1, Q3)	4 (2, 5)

Categorical variables are presented as n (%).

Parent/guardian and clinician knowledge and behaviour regarding probiotics

Parents/guardians

One-hundred and twelve (88%) of the 127 parents/guardians were familiar with probiotics before doing the survey and 106/127 (84%) had previously given their children probiotics, mostly as foods containing naturally occurring probiotics (eg, regular yoghourt, kefir, sauerkraut and kimchi) (81/106; 76%) and foods containing supplemental probiotics (eg, yoghourts and drinks containing added probiotics) (64/106; 60%). Thirty-two of 106 parents (30%) had given their children probiotics in the form of supplements (eg, powder, capsule, chewable pill and drop/liquid). When asked which formulation their child would prefer (choose all that apply), most parents favoured drops/liquid form (63%) of probiotic supplements. Chewable pills (48%) and powder/sachet (42%) were the next favourite options, followed by capsules (26%); 3% selected none of the options.

Clinicians

Thirty-two of the 45 (71%) paediatricians recommended probiotics for specific indications, 9/45 (20%) selected ‘other’ (eg, ‘If they want to take them I do not object’, “I state that the current evidence for its use is limited and that there is a cost associated with their use. It could help and likely would not harm their child but could harm their pocket book.”), 3/45 (7%) did not know enough about probiotics to make any recommendations and 1/45 (2%) did not recommend probiotics at all. Thirty-eight of the 45 paediatricians (84%) stated that they recommended probiotics without parents asking them. Paediatricians mainly recommended probiotic supplements (29/45, 64%) or foods containing supplemental probiotics (23/45, 51%). The the most common indication for which they had recommended probiotics was prevention and treatment of AAD (31/45, 69%). Other indications were treatment (23/45, 51%) and prevention (12/45, 27%) of non-specific diarrhoea, prevention of necrotizing enterocolitis (2/45, 4%) and other conditions (10/45, 22%) (eg, functional abdominal pain, functional constipation, inflammatory bowel disease, irritable bowel syndrome, infantile colic and cold).

Parent/guardian and clinician opinions regarding probiotics for prevention and treatment of AAD

Compared with parents, paediatricians more frequently agreed or strongly agreed that probiotics were effective (77 vs 48%, p=0.001) and safe (98 vs 62%, p<0.001) for prevention of AAD. Three (2%) parents and none of the clinicians disagreed or strongly disagreed that probiotics were safe for prevention of AAD (table 3).

Table 3.

Parent/guardian and clinician opinions about effectiveness and safety of probiotics for prevention and treatment of antibiotic-associated diarrhoea

			Strongly agreed	Agreed	Neutral	Disagreed	Strongly disagreed	Do not know	P value
Prevention	Effective	Parents n=126	19 (15%)	41 (33%)	22 (17%)	5 (4%)	6 (5%)	33 (26%)	0.001
		Clinicians n=44	10 (23 %)	24 (54%)	6 (14%)	4 (9%)	0	0
	Safe	Parents n=123	38 (31%)	38 (31%)	26 (21%)	0	3 (2%)	18 (15%)	0.000
		Clinicians n=44	17 (39%)	26 (59%)	1 (2%)	0	0	0
Treatment	Effective	Parents n=126	17 (14%)	37 (29%)	23 (18%)	3 (2%)	6 (5%)	40 (32%)	0.000
		Clinicians n=43	6 (14%)	23 (53%)	11 (26%)	3 (7%)	0	0
	Safe	Parents n=123	34 (28%)	39 (32%)	21 (17%)	1 (1%)	4 (3%)	24 (19%)	0.000
		Clinicians n=43	13 (30%)	28 (65%)	2 (5%)	0	0	0

Data are presented as n (%).

AAD, antibiotic-associated diarrhoea; NS, non-significant.

Minimally important difference

Sixty-four out of 125 responding parents (51%) and 21 out of 41 responding clinicians (51%) reported they would use probiotics if it could reduce the RR of AAD by 39% (ie, reduce the absolute risk of AAD from 19% to 12%; yielding a number needed to treat of 13 and a RR of 0.61) (table 4). Paediatricians were most likely to choose a RR reduction of 54% or less than compared with parents (85 vs 65%; OR=3, 95% CI 1.14 to 9.54, p=0.02)

Table 4.

Parent/guardian and clinician opinions about minimally important difference

MID options—Absolute risk of diarrhoea in probiotic group, assuming 19% in control group	Parents (n=125)	Clinicians (n=41)
12% (NNT=13, RRR=0.39)	64 (51%)	21 (51%)
9% (NNT=10, RRR=0.54)	18 (14%)	14 (34%)
7% (NNT=8, RRR=0.65)	33 (27%)	6 (15%)
I would not give (recommend) probiotics for AAD prevention	10 (8%)	0

MID, minimally important difference; NNT, number needed to treat; RRR, relative risk reduction.

There was no association between parental age, gender, ethnicity, education, previous familiarity with probiotics, previous use of probiotics, child’s previous experience of AAD and parental opinion about the safety of probiotics with the choice of MID (p>0.05) (online supplementary appendix C). In addition, there was no association between clinician’s gender, specialty, years since graduation, the number of AAD patients seen per month, previous familiarity and recommendation of probiotics and clinician’s opinion about the safety of probiotics with the choice of MID (p>0.05) (online supplementary appendix C).

Important outcomes

According to GRADE,⁴⁶ outcomes should be measured in clinical trials if more than 70% of respondents rate them between 7 and 9 (critical) and less than 15% rate them between 1 and 3 (limited importance) on a scale of 1–9.

In our study, the most important outcomes to parents in descending order were - need for hospitalisation, prevention of dehydration, disruption of normal daily activities, diarrhoea duration and physician revisit (table 5). Paediatricians considered the need for hospitalisation along with physician revisit as the most important outcomes. Moreover, they also rated prevention of dehydration, diarrhoea duration and stool frequency as critical outcomes to be measured in clinical trials (table 5).

Table 5.

Parent/guardian and clinician opinions regarding importance of outcomes in clinical trials of antibiotic-associated diarrhoea

Outcomes		Limited importance	Important but not critical	Critical	P value*
Stool frequency	Parents (n=125)	17 (14%)	50 (40%)	58 (46%)	0.002
	Clinicians (n=40)	1 (2%)	8 (20%)	31 (78%)
Stool consistency	Parents (n=125)	6 (5%)	38 (30%)	81 (65%)	0.03
	Clinicians (n=40)	2 (5%)	21 (53%)	17 (42%)
Duration of diarrhoea	Parents (n=125)	3 (2%)	26 (21%)	96 (77%)	NS
	Clinicians (n=40)	1 (2%)	7 (18%)	32 (80%)
Dehydration	Parents (n=125)	3 (2%)	15 (12%)	108 (86%)	NS
	Clinicians (n=40)	1 (2%)	7 (18%)	32 (80%)
Effect on normal daily activities (eg, eating, sleeping and playing)	Parents (n=125)	0	19 (15%)	106 (85%)	0.004
	Clinicians (n=40)	1 (2%)	14 (35%)	25 (63%)
Child absence from daycare or school	Parents (n=125)	19 (15%)	31 (25%)	75 (60%)	NS
	Clinicians (n=40)	3 (7%)	16 (40%)	21 (53%)
Parental absence from work	Parents (n=125)	30 (24%)	31 (25%)	64 (51%)	NS
	Clinicians (n=40)	4 (10%)	14 (36%)	21 (54%)
Need for hospitalisation	Parents (n=125)	3 (2%)	8 (7%)	113 (91%)	NS
	Clinicians (n=40)	1 (2%)	4 (10%)	35 (88%)
Need for outpatient or emergency department visit	Parents (n=125)	7 (6%)	23 (18%)	95 (76%)	NS
	Clinicians (n=40)	1 (2%)	4 (10%)	35 (88%)

*For the comparison between parents and clinicians.

NS, non-significant.

Discussion

Our study showed that half of the parents of children presenting to the emergency department of a Canadian tertiary care children’s hospital and half of the paediatricians working in that hospital required at least a 39% reduction in the RR of paediatric antibiotic-associated diarrhoea (ie, decrease the absolute AAD risk from 19% to 12%) to consider it worthwhile to consume/recommend probiotics. No associated factors (eg, demographic characteristics, previous experience of AAD and familiarity with probiotics) were found to be related with the choice of MID in either group.

There are multiple approaches to establish MID in the current literature: anchor-based, distribution-based, health economic, pilot studies, review of the existing evidence and opinion-seeking.¹⁶ However, most of them are not considered patient-centred approaches. Although anchor-based methods reflect patients’ views about the amount of experienced change, most often researchers decide on the threshold scores for MID. In addition, this method usually relies on change of symptoms over time rather than differences between patients with and without intervention.¹⁹

To obtain parent preferences about MID, we used the benefit-harm trade off tool providing advantages and disadvantages (eg, side effects, costs and inconvenience) of the intervention. This method has been used in various studies in other settings.^49–54 In addition to considering the patient’s perspective, this method is specific to the intervention and is based on between-group comparisons.^{18 19}

In the majority of the previous studies comparing patient and healthcare provider opinions, patients wanted larger effect sizes before opting for an intervention than did healthcare providers.^55–60 In our study, although clinicians were more convinced than were parents that probiotics are safe and effective, the MIDs were relatively similar. Only 8% of parents and none of the clinicians were unwilling to use probiotics for AAD. The high rates of familiarity and use of probiotics, limited costs and inconvenience and the favourable safety profile of probiotics may explain this preference.

The level of familiarity (88%) and use of probiotics (84%) by parents/guardians were high in our study compared with others. Chin-Lee et al in 2014,⁶¹ reported that 65% of their respondents were familiar with the term ‘probiotics’ and only 30% had used them before. Another study in New Zealand (2011)⁶² also showed a low rate (25%) of probiotic use. Studies in the Netherlands in 2013 (50%),⁶³ Brazil in 2008 (29%)⁶⁴ and Greece in 2005 (18%–29%)⁶⁵ reported even less familiarity with the term and meaning of probiotics. It is possible that the general population has greater awareness about the potential health benefits of probiotic products over time, but it also seems parents in Canada have a more positive attitude towards probiotics than do those in other countries.

In 2014, a core outcome set was developed for clinical trials of acute diarrhoea in children.⁶⁶ Outcomes included prevention of hospitalisation, diarrhoea and dehydration, similar to the outcomes of greatest importance to our participants. Employing outcomes that reflect patient/parent and clinician opinions will increase the acceptability and relevance of these studies.

Strengths and limitations

Our study is the first to seek parent and clinician opinions about MID of probiotic therapy for preventing paediatric AAD. Before recruitment, a pilot and clinical sensibility testing were conducted to ensure the comprehensibility and feasibility of the surveys, and revisions were made based on the results. Response rate of parents/guardians was very high since the survey was conducted in the emergency department waiting room with in-person support.

Our study has some limitations. It was restricted to individuals who could communicate in English. In addition, we only recruited parents of children presenting to a children’s hospital emergency department, which represent a small fraction of children who are prescribed antibiotics. These might affect the generalisability of our findings. The level of education in our participating parents was higher than the level of education in people living in Alberta (Canada).⁶⁷ Although, education level and previous familiarity with probiotics were not correlated with the choice of MID in our study, these characteristics might affect the generalisability of our findings. In addition, all our participating clinicians were paediatricians who may be more familiar with probiotics than other medical specialists. Similar to previous studies,⁶⁸ there was a low response rate from clinicians despite sending two reminders after the first invitation. According to VanGeest et al,⁶⁹ the most common reasons for non-responders are being busy and considering surveys as a low priority task compared with their other duties. Moreover, in our study, the administration method was different for parents/guardians (in-person) and clinicians (online) which might have an effect on their response rate.

Implication

Findings of our study regarding MID will inform future RCTs to calculate sample size and interpret findings informed by parental and clinician perspectives. Given that parents/caregivers are the ultimate decision-makers about their child’s health, especially for treatments that are easily available without a prescription, employing the outcomes that are most important to them will also improve the applicability and relevance of future studies.

Conclusion

There is a good agreement between parents and clinicians regarding how effective probiotics need to be in preventing AAD in order to warrant use. This information, along with the outcomes they perceived important, will help designing future clinical trials.