Response to “Questions on “Intervention effects of a kindergarten-based health promotion programme on obesity related behavioural outcomes and BMI percentiles”” by Vorland et al.

Evaluating large interventions can pose a considerable challenge to researchers and requires a substantial investment of time and resources. However, there is growing interest among policymakers, practitioners, and researchers in evaluating multi-component randomised controlled trials due to the need of further evidence-based effectiveness of public health interventions. We all are aware that evaluating large interventions becomes more challenging as they become more complex (Datta and Petticrew, 2013). The aspects raised by Vorland et al. (2019) point this out and confirm that even well planned public health research sometimes leads to unexpected results.

This has neither to do with changing outcomes, nor with reporting bias. The study protocol (Kobel et al., 2017) clearly states “The results of all statistical tests will be interpreted in an explorative sense and not in a confirmatory way.”. The article in question (Kobel et al., 2019) never aimed to report only primary outcomes, nor did it stated that. Rather, it is one of several manuscripts about the programme reporting its findings. Other aspects – primary and secondary outcomes, as well as other investigative findings based on subjective as well as objective assessment methods – will be published by the research team in time whereas its sequence is mainly due to the publication processes of the journals.

Regarding the addressed methodological concerns about using self-reported outcomes we emphasize say that objective assessment methods have been used in the study (e.g. for physical activity, nutritional knowledge, anthropometrics, motor skills), some of which have also been reported (Kobel et al., 2019). Others will – as mentioned above – follow in other manuscripts. Yet, the majority of large public health studies are conducted on the basis of self-report data (Carson et al., 2017), mainly due to cost and feasibility. The used well-established instruments include only validated questions and the readers were made aware of the possible biases attached to subjective assessment methods (Kobel et al., 2019) and can therefore interpret the data that way.

No interpretation, however, is needed when carefully reading the results; all variables were reported as initial changes and in the tables controlled for all relevant variables. Also, clustering was not ignored, neither was the plan to evaluate the data using intention-to-treat. The study protocol agreed with the University’s Ethics Committee states: “The evaluation of the study follows the intention-to-treat approach, but there will be no replacement of missing values”, which is what literature suggests in such contexts (Dong and Peng, 2013, Jakobsen et al., 2017). We would like to emphasize that the study (Kobel et al., 2019) was no quasi-experimental research with close contact to the research team but a real-life roll-out of a multi-component investigation in a whole state for one year. It, therefore, reports a realistic low-level, long-term intervention in pre-school children, performed by trained teachers developed during a structured scientific process (Bartholomew et al., 2006).

The power of such interventions is naturally low and Table 1 shows an analysis of drop-outs, which were expected previously due to the children’s age and setting and were included in the initial power calculations. Drop-outs, realistic for such population-based interventions, were mainly due to three kindergartens leaving the intervention completely and the older children leaving for school. Additional missing values are due to unreturned parental questionnaires.

Table 1.

Participant’s characteristics at baseline, split into drop-outs and children who stayed in the intervention, including statistical differences between those groups.

	Missing Values	Non-Drop-outs (n = 558)	Drop-outs (n = 415)	Total (n = 973)	p
Age, years [m (sd)]		3.6 (0.6)	4.5 (0.7)	4.0 (0.8)	0.01
Boys, n (%)*		292 (52.3)	205 (49.4)	497 (51.1)	0.37
Anthropometry
BMIPCT, m (sd)	118	50.3 (25.9)	49.0 (27.3)	49.7 (26.5)	0.47
Overweight and obesity, n (%)	118	28 (5.8)	33 (8.8)	61 (7.1)	0.09
Socio-economic characteristics
Migration background, n (%)	232	151 (33.4)	85 (29.4)	236 (31.8)	0.25
Nutrition
Portions of fruit and vegetables/day, m (sd)	218	2.3 (1.7)	2.4 (1.9)	2.3 (1.8)	0.45
Sugar-sweetened beverages ≥ daily, n (%)	202	44 (9.5)	13 (4.2)	57 (7.4)	0.01
Screen media use
Screen media use ≥60 min/day, n (%)	311	201 (49.5)	132 (51.6)	329 (49.7)	0.90
Physical activity
Days of MVPA ≥60 min/day, m (sd)	232	2.7 (2.0)	2.6 (1.9)	2.7 (2.0)	0.87
7 days of MVPA ≥60 min/day, n (%)	232	32 (7.2)	15 (5.1)	47 (6.3)	0.24
Endurance
Completed meters in 3 min, m (sd)	144	251.5 (47.6)	272.6 (51.0)	260.6 (50.2)	0.01

m (sd), mean (standard deviation); BMIPCT = BMI percentiles; MVPA = moderate to vigorous physical activity; bold = significant difference between drop-outs and non-drop-outs.

Despite these conditions, we were able to demonstrate significant impact on outcomes using linear-regression models showing no significantly different results when clustering was taken into account (Table 2). Therefore, the study adds significantly to existing literature and after acknowledging and explaining the above mentioned points of concern, we see no need to reanalyse the data in order to clarify other potentially undermining factors. The bespoke article (Kobel et al., 2019) went through a standardised three-stage review process during which it was peer-reviewed by qualified, recognised researchers, which we thank for their valuable inputs prior to publication.

Table 2.

Results from the final regression model and the GEE model for health outcomes at follow up for the intervention group.

	na	Coefficientb	p	95% CI
Physical activity (60 min MVPA/day (days/week)	279
Linear regression model		0.102	0.049	[0.01; 0.81]
GEE model		0.102	0.025	[0.05; 0.76]
Fruit and vegetable intake (portions of fruit and vegetables/day)	286
Linear regression model		0.006	0.907	[−0.34; 0.39]
GEE model		0.006	0.920	[−0.40; 0.44]
Sugar-sweetened beverages (Daily consumption of sugar-sweetened beverages)	295
Linear regression model		−0.098	0.070	[−0.10; 0.01]
GEE model		−0.098	0.103	[−0.11; 0.01]
Screen media use (≥60 min/day)	256
Linear regression model		−0.002	0.964	[−0.27; 0.26]
GEE model		−0.002	0.951	[−0.20; 0.19]
Endurance capacity (meters completed in 3 min)	466
Linear regression model		0.201	0.001	[10.31; 27.80]
GEE model		0.201	0.049	[0.06; 38.05]
Weight status (BMI percentiles)	480
Linear regression model		−0.032	0.153	[−3.98; 0.62]
GEE model		−0.032	0.121	[−3.86; 0.45]

CI = confidence interval for regression coefficient, MVPA = moderate to vigorous physical activity; Bold = significant difference between control and intervention group.

^aOnly cases with baseline and follow-up data.

^bStandardised regression coefficients adjusted for baseline values, gender, age, BMI percentiles (except weight status), and migration status.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.