Abstract
Objective
To our knowledge, the effect of front-of-pack nutrition labels such as the Nutri-Score on food purchases has never been assessed among individuals suffering from nutrition-related chronic diseases specifically, while dietary modifications are generally part of their care. This study aimed to investigate the effect of the Nutri-Score on the nutritional quality of purchasing intentions among adults suffering from a cardiometabolic disease, compared with no label and the Reference Intakes (RIs), a label already implemented by some food manufacturers in France.
Setting
Secondary prevention—mainland France.
Participants
2431 eligible participants were randomly assigned and 1180 participants (65.5% women, mean age 65.0±7.1 years) completed the shopping task and were included in the analyses.
Intervention
A three-arm randomised controlled trial using an experimental online supermarket was conducted in 2017. Participants with cardiometabolic diseases were invited to simulate food purchases with the Nutri-Score, the RIs or no label.
Primary and secondary outcome measures
The primary outcome was the nutritional quality of the shopping cart, estimated using the French-modified Food Standard Agency Nutrient Profiling System (FSAm-NPS), and secondary outcomes included the nutrient content of purchases.
Results
The mean (SD) FSAm-NPS score was significantly lower in the Nutri-Score arm (1.29 (3.61) points), reflecting a higher overall nutritional quality of purchasing intentions, compared with the RIs (1.86 (3.23) points) and no label (1.92 (2.90) points) arms (p=0.01). Moreover, the Nutri-Score led to significantly lower content in calories and saturated fatty acids compared with the two other arms. These differences resulted from participants avoiding some packaged products (sweets, dairy and starches) and purchasing larger amounts of fresh fruit and meat.
Conclusions
The Nutri-Score exhibited a significant higher nutritional quality of purchasing intentions, encouraging healthier food choices among individuals suffering from cardiometabolic chronic diseases.
Trial registration number
Keywords: epidemiology, health policy, nutrition & dietetics
Strengths and limitations of this study.
Inclusion of a rarely explored population in a randomised controlled trial pertaining to the effectiveness of front-of-package labelling on food choices.
This controlled experimental environment allowed assessing the effect of the Nutri-Score in standardised conditions and optimising internal validity of the study.
Limitation pertaining to a high rate of participants who did not complete the shopping task.
The trial investigated purchasing intentions rather than actual food purchases.
Introduction
Non-communicable diseases (NCDs), such as obesity, type 2 diabetes, cardiovascular diseases and cancer have become a major burden for the current health systems.1 For these diseases, dietary factors have been recognised to be one of the major leading risk factors in developed countries, resulting in 11 million deaths worldwide in 2017, and represent modifiable determinants through primary prevention.2 In France, cardiovascular diseases remain the second leading cause of deaths by NCDs, accounting for 30% approximately of mortality.3 Regarding obesity, the prevalence was estimated at 17% within the French adult population in 2015,4 and the prevalence of type 2 diabetes was around 5% in 2016.5
Hence, in the context of secondary or tertiary prevention, many treatment guidelines highlight the importance to modify dietary habits to improve the nutritional status of individuals and thus control these nutrition-related NCDs.1 For example, controlling for saturated satty acids (SFA), sugars and salt intakes and increasing fruits and vegetables, pulses and fibres consumption are encouraged in the management of several NCDs or risk factors such as obesity, arterial hypertension and diabetes.1 Nutritional labelling has been suggested to be an interesting tool in helping individuals suffering from NCDs achieve balanced nutritional intakes.6 However, it has been shown that nutritional information on the back of packages was poorly understood and used during food choices.7 While few studies have suggested that individuals suffering from nutrition-related NCDs would pay more attention to nutritional information and check for specific nutrients,8 9 another study has observed no difference of nutritional information use between patients and individuals with no chronic condition.6
In the last decade, Front-of-Pack nutrition Labels (FoPLs) have been identified to improve the nutritional quality of food choices at the point of purchase in the general population,10–19 and to encourage reformulation and innovation of food products.20 21 In France, the summary FoPL Nutri-Score has been adopted in October 2017 (and then in several European countries) to indicate the nutritional quality of products in supermarkets.22 By the end of 2019, the brands which adopted the Nutri-Score represented approximately 25% of the volume of prepacked foods sales with more than 300 manufacturers engaged.23 The Nutri-Score has been demonstrated to be well perceived, understood and to have a positive effect on food purchases in the general French population14 18 24–27 and students.28 However, as the measure is implemented on a voluntary basis, it coexists on the French market with the Reference Intakes label (RIs),29 used by multiple food manufacturers since 2006 in Europe, and the absence of any front-of-pack labelling.
To our knowledge, no study has specifically investigated the effect of FoPLs, including the Nutri-Score, on food purchasing intentions of patients suffering from nutrition-related NCDs only. Thus, the study aimed to determine the effect of the Nutri-Score on purchasing intentions of individuals suffering from nutrition-related cardiometabolic chronic diseases, compared with the current French labelling situations, that is, the RIs or no FoPL, as a secondary or primary prevention tool.
Methods
Trial design and participants
A three-arm parallel group randomised trial was conducted in 2017 targeting individuals suffering from cardiometabolic NCDs. Electronic consent was obtained from each participant. A methodology similar to a trial targeting students was used.28
Participants were recruited from the NutriNet-Santé cohort by a targeted emailing campaign in 2016, using the following criteria: age, body mass index and the declaration of one of the diseases included in the present study. Briefly, the NutriNet-Santé is an ongoing web-based prospective observational cohort study launched in France in May 2009, including adult volunteers recruited by multimedia campaigns.30 Each individual who agreed to participate was asked to fulfil an inclusion questionnaire and provide information on gender, age, occupation, educational level, household composition and weekly budget for grocery shopping. They were also asked to self-estimate their nutrition knowledge level on a 4-point scale (between ‘I am very knowledgeable about nutrition’ and ‘I do not know anything about nutrition’), and to provide information on their grocery shopping frequency in general and online (‘Always’, ‘Often’, ‘Sometimes’ and ‘Never’). Finally, they were invited to declare if they had been diagnosed or were currently under medical supervision for at least one of the following nutrition-related chronic diseases: obesity, type 2 diabetes, dyslipidaemia, arterial hypertension and cardiovascular disease. Thus, individuals involved in grocery shopping, over 50 years old, and with at least one of the chronic diseases from the list above, were eligible to participate.
Patient and public involvement
The research question underlying the study was driven by considerations regarding tools to improve patients’ empowerment concerning their diets. Patients were not directly involved in the development of the protocol or in recruitment of participants. Dissemination of the research results will be done through the NutriNet-Santé cohort platform, with an abstract in the French language, allowing for all participants to be informed.
Randomisation and blinding
Eligible participants were randomly allocated to one of the three arms using a random block method with permuted blocks of size 3, 6, 9 and 12, without stratification. The randomisation list was only available to the independent statistician who generated the randomisation sequence and the computer programmer who uploaded the list on the secured platform. Given the nature of the intervention, participants could not be blinded of the intervention; however, they were only informed about the main objectives of the experimental online supermarket, aiming to investigate determinants of purchasing behaviour. No information was given on the FoPLs or the explicit purpose of the trial.
Intervention and procedure
Experimental arm
The experimental arm consisted on the Nutri-Score applied on the front of package of all prepacked foods included in the online supermarket. The Nutri-Score is a summary FoPL characterising the overall nutritional quality of foods. The label is based on the Food Standards Agency Nutrient Profiling System, modified by the High Council of Public Health to better discriminate foods from specific categories (cheese, fats and beverages) consistently with nutritional recommendations (FSAm-NPS).18 The FSAm-NPS is calculated for 100 g (or 100 mL) of food, and allocates from 0 to 10 points for each nutrient which should be limited (energy (kJ), SFA (g), sugars (g) and sodium (mg)) and from 0 to 5 points to each favourable nutrient which should be encouraged (proteins (g), fibres (g) and the content in fruits, vegetables, legumes and nuts (%)). A discrete score is finally obtained by subtracting the favourable points from the unfavourable points, ranging therefore between a minimum of −15, for food products with higher nutritional quality, to a maximum of+40 points for food products with lower nutritional quality. Hence, the lower the FSAm-NPS score, the healthier the products. Then, the Nutri-Score is represented by a 5-colour scale with a corresponding letter, from dark green (A) indicating the highest nutritional quality to dark orange (E) for products with the lowest nutritional quality.
Control arms
Two control arms were also included: (1) the RIs FoPL was affixed on all prepacked food items and (2) no front-of-pack nutritional labelling at all. The RIs are a nutrient-specific monochromatic label endorsed by some manufacturers, indicating the kilocalories and the amount of fat, SFA, sugars and sodium in gram per serving, and their contribution in percentages to the guideline-based daily intakes.29 In the no label arm, no nutrition label was applied on the front of food packages on the experimental online supermarket.
The experimental online supermarket was composed of three sections. First, the upper section included the logo of the supermarket, a search bar, an access to the shopping cart and the tabs for the different food categories. Second, a central section displaying advertisements and showing shoppers in a supermarket aisle was included. The rotating banner ad on the left side of the central section included one specific ad and four ads on non-dietary information such as information on national campaigns of health promotion. In the two arms with a FoPL, the specific ad drew awareness on the label with additional information on its computation and use. In the no label arm, additional information was provided on the proper conservation of fresh food products. On the central section, the participant could also view the different products depending on the food categories, and access the information (name, brand, price, nutritional information, etc) by clicking on the product. For the two label arms, the nutritional label was affixed on the front of the package and next to the product on a larger scale to improve its readability. Third, the lower section included links to the various food categories, links for information and links towards account information. An example of a food item included in the experimental online supermarket with its three versions depending on the trial arm is shown in figure 1 and a picture of the experimental online supermarket is presented in online supplemental figure S1.28
bmjopen-2021-058139supp001.pdf (147.1KB, pdf)
Procedure
For this specific purpose, an experimental online supermarket was developed, similar to previous trials.18 28 Eligible participants were invited to simulate a shopping task as if they were in their usual supermarket, but without any payment required and no instruction on the amount, the duration or the number of participants they were asked to shop for. The experimental online supermarket resembled existing grocery shopping websites with a virtual shopping cart, a virtual payment procedure, a search tab and promotional banners. As in real shopping websites, participants could choose products categorised in multiple food groups and subgroups, using a hierarchical structure and names of the categories similar to existing online supermarkets. The food offer was a representative sample of the products commonly sold on French online supermarkets and included 751 foods and beverages (prepacked products carrying a FoPL on the Nutri-Score and RIs arms, and raw products without any label in the three arms according to the European regulation), divided into 20 food categories. For all products, name, brand, price (per unit and per kg or litre), a picture of the product (with or without a FoPL, depending on the arm) and the nutritional composition as well as the list of ingredients were provided. For each food item, at least two different products were proposed, including a national brand and a retailer’s brand. The number of brands proposed balanced the nutritional variability observed for a given type of food.
Outcomes
The primary outcome was the overall nutritional quality of the shopping cart, assessed by the mean of the FSAm-NPS score across all the items in the cart, computed for 100 g. A lower overall FSAm-NPS score of the shopping cart reflects a higher nutritional quality of the entire selection of products within the cart. Minimal theoretical value is −15, maximal theoretical value is +40.
Secondary outcomes were, by order of importance, the content of the shopping cart in energy, SFA, sugars, sodium, fibres, fruits and vegetables, and proteins, for 100 g of the shopping cart.
Statistical analyses
The final sample size was calculated for an effect size of 0.2 (for the main outcome, FSAm-NPS score, calculated by minimisation of estimates from previous studies showing a 0.62 point difference between Nutri-Score and control arm, with an SD of 2.55 of the average FSAm-NPS of the shopping cart26), a power of 90% and a p value of 0.02 considering the three-arm design, resulting in 1956 individuals, that is, 652 participants per arm. To reach this final sample size while considering non-respondents, 2431 individuals were initially randomised and the number of individuals validating their shopping cart was monitored.
Per protocol analyses were carried out, given that only one measure was collected for the outcome. All participants meeting the inclusion criteria and who completed the shopping task were included in the analyses. The primary outcome was compared between the three trial arms using one-way analysis of variance (ANOVA) (p≤0.05 significant). Pairwise comparisons among FoPLs were performed using Tukey tests to consider multiple comparisons (p≤0.05 significant). Then, secondary outcome variables were also compared between the three arms using a hierarchical gatekeeping strategy28 with the following order: (1) energy, (2) SFA, (3) sugars, (4) sodium, (5) fibres, (6) fruits and vegetables, (7) proteins. When the comparison across the three arms for a component was not significant, the comparison of following secondary outcomes was stopped. The gatekeeping strategy order was determined using the relative importance of the various nutrients to health (with the most unfavourable elements first) and the results of previous studies assessing FoPL effects on the nutritional quality of food purchases.18 Analyses were performed considering the FSAm-NPS score of all products from the experimental supermarket, including also raw items that were not labelled in any trial arm (ie, fruits, vegetables, meat and poultry). Multiple sensitivity analyses were then performed. First, sensitivity analyses were computed (1) including only labelled food products (ie, prepacked foods and beverages), (2) excluding participants whose spending amount was below the 5th percentile or over the 95th percentile of the distribution of the cost of the shopping carts in the sample and (3) using multiple imputations on missing outcomes (25 imputed sets) to consider the non-response rate and thus provide intention-to-treat estimates. Missing primary and secondary outcomes of non-respondents were imputed using the individual characteristics of the individuals, including sociodemographic and nutrition-related lifestyle data collected in the inclusion questionnaire. The total quantities of calories, SFA, sugars, sodium, fibres and proteins in the shopping carts were also calculated and compared across the three arms using ANOVA. The composition of the shopping cart across the different food categories was calculated in percentage of the total number of products in the cart (mean and SE). The contributions of each food group to the nutrient amounts in the shopping carts were then calculated and expressed a mean percentage and SE. Finally, the distribution of the products across the different Nutri-Score classes was also compared between the three arms, taking into account all food products including raw foods that were non-labelled.
All tests of significance were two sided, and analyses were carried out with the SAS software (V.9.4; SAS Institute).
Results
Among 3728 individuals with chronic diseases assessed for eligibility, 1297 did not meet inclusion criteria, resulting in 2431 participants randomly assigned to one of the three arms (figure 2). Among them, 1180 individuals with a nutrition-related chronic disease fully completed the shopping task and were finally included in the analyses. The other subjects who did not complete their shopping cart were excluded from the analyses, as their purchasing behaviour may not be representative of their habits. Overall, participants of the trial included 65.5% of women, 27.8% of subjects with primary educational level and their mean age was 65.0±7.1 years (table 1). Regarding purchasing behaviour, 61.2% declared doing always their grocery shopping and 29.7% reported having purchased foods online at least once. Among them, 16.2% reported purchasing online at least one time per week. 57.2% of the included participants declared having an intermediate self-estimated nutrition knowledge level, and 51.4% often reading the nutrition facts. The two main chronic diseases represented in the trial were arterial hypertension (65.7%) and dyslipidaemia (33.9%), then followed by cardiovascular diseases (15.2%), type 2 diabetes (14.7%) and obesity (13.8%). Approximately 30% of participants reported having more than one of the diseases included in the trial. Individual characteristics of participants were globally similar between the three arms. The mean cost of the shopping cart was €75.0±51.5 overall, €80.0±57.8 in the Nutri-Score arm, €73.9±48.3 in the RIs arm and €71.2±47.3 in the no label arm. The mean weight of the shopping carts was 16.6±14.3 kg in the Nutri-Score arm with 22.9±21.9 products on average, 24.2±14.7 kg in the RIs arm with 33.6±22.0 products on average, and 22.7±14.2 kg in the no label arm with 31.1±21.3 products on average.
Table 1.
Nutri-Score | Reference Intakes | No label | Total | |
Total (n) | 394 | 392 | 394 | 1180 |
Gender, n (%) | ||||
Men | 131 (33.3) | 124 (31.6) | 152 (38.6) | 407 (34.5) |
Women | 263 (66.7) | 268 (68.4) | 242 (61.4) | 773 (65.5) |
Age, years | 64.8±6.9 | 64.8±7.3 | 65.4±7.1 | 65.0±7.1 |
Educational level, n (%) | ||||
Primary | 122 (31.0) | 102 (26.0) | 104 (26.4) | 328 (27.8) |
Secondary | 53 (13.4) | 51 (13.0) | 74 (18.8) | 178 (15.1) |
University, undergraduate degree | 103 (26.1) | 122 (31.2) | 99 (25.1) | 324 (27.4) |
University, postgraduate degree | 98 (24.9) | 102 (26.0) | 103 (26.1) | 303 (25.7) |
Other | 18 (4.6) | 15 (3.8) | 14 (3.6) | 47 (4.0) |
Grocery shopping frequency, n (%) | ||||
Always | 231 (58.6) | 252 (64.3) | 239 (60.6) | 722 (61.2) |
Often | 122 (31.0) | 107 (27.3) | 113 (28.7) | 342 (29.0) |
Sometimes | 41 (10.4) | 33 (8.4) | 42 (10.7) | 116 (9.8) |
Online grocery shopping, yes n (%) | 119 (30.2) | 129 (32.9) | 103 (26.1) | 351 (29.7) |
Online grocery shopping frequency, n (%) | ||||
At least one time per week | 16 (13.4) | 20 (15.5) | 21 (20.4) | 57 (16.2) |
One or two times per month | 22 (18.5) | 26 (20.1) | 15 (14.5) | 63 (18.0) |
One time every 2 or 3 months | 29 (24.4) | 33 (25.6) | 17 (16.5) | 79 (22.5) |
One or two times per year | 23 (19.3) | 21 (16.3) | 29 (28.2) | 73 (20.8) |
Less than one time per year | 29 (24.4) | 29 (22.5) | 21 (20.4) | 79 (22.5) |
Weekly budget for grocery shopping (€), n (%) | ||||
<€30 | 13 (3.3) | 17 (4.3) | 16 (4.1) | 46 (3.9) |
€30–50 | 76 (19.3) | 74 (18.9) | 63 (16.0) | 213 (18.0) |
€50–100 | 151 (38.3) | 168 (42.9) | 160 (40.6) | 479 (40.6) |
>€100 | 151 (38.3) | 130 (33.1) | 147 (37.3) | 428 (36.3) |
Missing | 3 (0.8) | 3 (0.8) | 8 (2.0) | 14 (1.2) |
Perceived nutritional knowledge, n (%) | ||||
High | 38 (9.6) | 38 (9.7) | 22 (5.6) | 98 (8.3) |
Intermediate | 222 (56.4) | 220 (56.1) | 233 (59.1) | 675 (57.2) |
Low | 125 (31.7) | 125 (31.9) | 124 (31.5) | 374 (31.7) |
No | 9 (2.3) | 7 (1.8) | 9 (2.3) | 25 (2.1) |
Missing data | 0 | 2 (0.5) | 6 (1.5) | 8 (0.7) |
Nutrition facts reading frequency, n (%) | ||||
Always | 63 (16.0) | 55 (14.0) | 54 (13.7) | 172 (14.6) |
Often | 202 (51.3) | 199 (50.8) | 206 (52.3) | 607 (51.4) |
Sometimes | 117 (29.7) | 122 (31.1) | 119 (30.2) | 358 (30.3) |
Never | 12 (3.0) | 14 (3.6) | 9 (2.3) | 35 (3.0) |
Missing data | 0 | 2 (0.5) | 6 (1.5) | 8 (0.7) |
Chronic disease diagnosed, n (%) | ||||
Arterial hypertension | 265 (67.3) | 256 (65.3) | 254 (64.5) | 775 (65.7) |
Diabetes mellitus | 51 (12.9) | 55 (14.0) | 67 (17.0) | 173 (14.7) |
Cardiovascular disease | 65 (16.5) | 48 (12.2) | 66 (16.8) | 179 (15.2) |
Dyslipidaemia | 141 (35.8) | 127 (32.4) | 132 (33.5) | 400 (33.9) |
Obesity | 43 (10.9) | 58 (14.8) | 62 (15.7) | 163 (13.8) |
Total cost of the shopping cart (€) | 80.0±57.8 | 73.9±48.3 | 71.2±47.3 | 75.0±51.5 |
Number of products in the shopping cart | 22.9±21.9 | 33.6±22.0 | 31.1±21.3 | 29.2±22.2 |
Weight of the shopping cart (kg) | 16.6±14.3 | 24.2±14.7 | 22.7±14.2 | 21.2±14.8 |
Values are mean±SD deviation or n (%) as appropriate.
According to the flow diagram, approximately 50% of participants did not complete the virtual shopping task. Individual characteristics between respondents and non-respondents were compared for intention-to-treat analyses and results are displayed in online supplemental table S1. Even if non-respondents had some small disparities on their sociodemographic and lifestyle characteristics compared with respondents, this potential bias was similar in the three arms. Indeed, the interaction term between each individual characteristic and the arm to model the probability of no response was not statistically significant (p≥0.1).
Outcomes
The FSAm-NPS score was lower in the Nutri-Score arm (1.29±3.61 points), reflecting a higher overall nutritional quality of the shopping carts, followed by the RIs arm (1.86±3.23 points) and no label (1.92±2.9 points) (table 2). The difference of FSAm-NPS scores were statistically significant between the Nutri-Score and the RIs groups (mean difference=−0.57 (−1.11; −0.02); p=0.04), and between the Nutri-Score and no label (−0.63 (−1.17; −0.08); p=0.02). No significant difference was observed between the RIs and no label (−0.06 (−0.61; 0.48); p=1.0). Furthermore, the Nutri-Score label led to a significantly lower content of the shopping carts in calories and SFA, compared with the RIs and no label (p≤0.0001 for comparisons of calories between the Nutri-Score and both RIs and no label; p=0.01 for comparisons of SFA between the Nutri-Score and both RIs and no label). The differences between the RIs and no label arms were not significant. The differences of sugars content between the three arms were not significant; then comparisons of subsequent secondary outcomes were stopped.
Table 2.
Nutri-Score | Reference Intakes | No label | P value | Nutri-Score versus no label | Nutri-Score versus Reference Intakes | Reference Intakes versus no label | ||||
n=394 | n=392 | n=394 | Difference* | P† | Difference* | P† | Difference* | P† | ||
Overall nutritional quality (FSAm-NPS score/100 g) | 1.29±3.61 | 1.86±3.23 | 1.92±2.9 | 0.01 | −0.63 (−1.17; −0.08) | 0.02 | −0.57 (−1.11; −0.02) | 0.04 | −0.06 (−0.61; 0.48) | 1.0 |
Calories (kcal/100 g) | 153.53±76.96 | 184.06±64.38 | 175.38±64.22 | <0.0001 | −21.85 (−33.35; −10.35) | <0.0001 | −30.53 (−42.05; −19.02) | <0.0001 | 8.68 (−2.83; 20.20) | 0.2 |
Saturated fatty acids (g/100 g) | 3.24±3.13 | 3.78±2.13 | 3.77±2.36 | 0.004 | −0.53 (−0.96; −0.10) | 0.01 | −0.53 (−0.96; −0.10) | 0.01 | 0.01 (−0.42; 0.44) | 1.0 |
Sugars (g/100) | 5.92±3.58 | 5.89±3.25 | 5.65±3.81 | 0.5 | 0.27 (−0.32; 0.87) | 0.5 | 0.03 (−0.56; 0.63) | 1.0 | 0.24 (−0.35; 0.84) | 0.6 |
Sodium (mg/100 g) | 189.83±200.21 | 195.51±104.13 | 212.73±158.16 | |||||||
Fibres (g/100 g) | 1.37±0.99 | 1.89±1.17 | 1.65±0.97 | |||||||
Fruits and vegetables (%) | 34.12±22.87 | 29.51±16.03 | 28.90±14.81 | |||||||
Proteins (g/100 g) | 7.36±3.43 | 7.29±2.20 | 7.58±3.33 |
*Mean difference (95% CI).
†P value using Tukey’s multiple comparisons tests. Boldface indicates statistical significance (p≤0.05).
FSAm-NPS, modified Food Standards Agency Nutrient Profiling System.
When analyses considered prepacked products only, the overall difference of shopping carts’ FSAm-NPS score between the three arms was no longer significant suggesting inter- food group substitutions (online supplemental table S2). However, results for the secondary outcomes remained consistent with the main analyses. In sensitivity analyses excluding outliers on the spending amount, similar results were observed for primary and secondary outcomes (online supplemental table S3). Results of the sensitivity analyses using multiple imputations and providing intention-to-treat estimates are presented in online supplemental table S4 for analyses considering all food products and online supplemental table S5 for analyses considering only labelled food items. Results using multiple imputations were consistent with the main analyses; however, the amplitude of differences between arms was lower and comparisons were no longer significant, except for calories for which the Nutri-Score also led to lower contents compared with the two other arms (online supplemental tables S4 and S5). The participants in the Nutri-Score arm purchased less calories, SFA, sugars, sodium, fibres and proteins compared with the two other arms (online supplemental table S6).
Online supplemental table S7 describes the shopping carts composition in terms of the mean number of products per food category in each of the three arms. In the Nutri-Score arm, participants tended to purchase more products from the fruits (especially fresh fruit), meat and water categories (compared with the RIs), and fewer products from vegetables, dairy products, cheeses, sweets and starchy foods such as pasta, rice, rush potatoes and semolina. The average percentages of raw products (ie, not labelled in the label arms) purchased by participants were 32.9%±18.4% in the no label arm, 33.2%±18.2% in the RIs arm and 42.0%±28.1% in the Nutri-Score arm. The percentage contributions of food groups to nutrient intakes in the overall shopping carts are presented in online supplemental table S8 (only for nutrients where a difference between arm was observed in the main analyses). Thus, the lower calorie and SFA contents of the shopping carts in the Nutri-Score arm compared with the RIs arms could be explained by fewer products purchased in the dairy products, cheese, but also sweets and starchy foods. Finally, the proportion of healthier food products in the shopping carts classified as A was significantly higher in the Nutri-Score arm compared with the two other arms (difference=5.63 (2.02; 9.24), p=0.0008 compared with no label; difference=4.85 (1.24; 8.47), p=0.005 compared with the RIs), which can be partly explained by the higher proportion of raw fruits and meats in the shopping carts of participants from the Nutri-Score group—corresponding to products with higher nutritional quality (online supplemental table S9). On the contrary, the proportion of unhealthier products classified as D or E was significantly lower in the Nutri-Score arm compared with the two other arms or the RIs only. No significant difference was observed between the RIs and no label.
Discussion
Results of the present study showed that the Nutri-Score label significantly led to an improvement of the overall nutritional quality of food purchasing intentions in individuals with cardiometabolic chronic disease. Moreover, the Nutri-Score led to lower contents of the shopping carts in energy and SFA compared with the two other arms. Similar trends were observed with multiple imputations; nevertheless, differences were no longer statistically significant. No significant difference was observed between the RIs and no label. Moreover, in both FoPLs arms, and particularly in the Nutri-Score arm, substitutions between food groups were observed, with more raw products purchased—corresponding mainly to fruits and butcher’s meats from higher nutritional quality. It appeared that the participants exposed to the Nutri-Score purchased less products and from higher overall nutritional quality (ie, lower FSAm-NPS score).
The present findings are consistent with studies which observed a positive effect of interpretive FoPLs and especially the Nutri-Score on the nutritional quality of intentional or real food purchases, while the RIs demonstrated a limited or non-significant effect in the general population14 18 or students.28 This could be partly explained by the features of the schemes. Indeed, the summary indicator of the Nutri-Score, combining colours and text, would be easier to read and understand.16 18 19 31–37 On the contrary, the RIs with its nutrient-specific and monochromatic format have been shown to be more complicated to identify and understand in the general population,18 36 37 creating notably potential decisional conflicts and prioritisation of nutrients.38 Nevertheless, to our knowledge, this is the first study to assess the effect of FoPLs on purchasing intentions among individuals suffering from nutrition-related NCDs. Only one study investigated the effect of the Traffic Lights nutrient-specific label and the three-stars summary label on food purchases in vending machine among patients in an Australian hospital and observed a positive effect of the labels to identify healthier products. However, the experiment was performed in a specific context and no focus was made on patients suffering specifically from nutrition-related NCDs.12
Interestingly, while previous studies among patients with hypertension, hypercholesterolaemia, type 2 diabetes or hyperlipidaemia found that they were more likely to read information on salt and SFA respectively,39 and have lower intakes in energy and SFA,9 in the present study, the RIs did not help consumers to select products with significantly less SFA compared with no label. On the contrary, the Nutri-Score which does not provide numerical data but rather summarised information led to significantly lower contents of the shopping carts in SFA compared with no label and the RIs. These results on the Nutri-Score effect are particularly important, given that a decrease of the intakes in energy, SFA and salt with an increase of fruits and vegetables consumption are recommended among patients suffering from nutrition-related NCDs.1 Moreover, despite these recommendations, it has been observed in a study within the NutriNet-Santé cohort that adults with a cardiometabolic disease tended to have unhealthier dietary habits overall (eg, lower intakes of fruits, higher intakes of meat, processed meat and added fats) compared with healthy controls,40 which supports the interest of public health measures encouraging healthier food choices among these individuals.
When analyses were restricted to labelled items only, no significant difference of the overall nutritional quality between the Nutri-Score and the other arms was found. These results reflect that the use of the Nutri-Score may encourage also substitutions between food categories. Indeed, participants who were exposed to the Nutri-Score tended to purchase more non-labelled raw products, in particular fruits, meat and poultry, characterised by healthier nutritional quality. This substitution between food categories has been observed in other populations under the same or similar experimental conditions.28 41 42 Some hypotheses could explain these results. In general, the impact of front-of-pack labelling has been found to vary according the food category,43 partly in relation to consumer motivation.44 More specifically, the Nutri-Score provides an explicit comparative scale of the nutritional quality of prepacked foods and may have raised awareness as to the lower nutritional value of some prepacked products. By comparison, this may have heightened the perceived healthiness fruit or meat products, even in the absence of any labelling. As to beverages, water being the only beverage receiving a ‘A’ Nutri-Score, its promotion is particularly straightforward in the system. Another hypothesis relates to the overall awareness to the importance of food choices that the presence of the Nutri-Score may have spurred, acting as a global reminder of previously received nutritional education in patients. In doing so, the Nutri-Score scheme may cue concerns/motivations about eating healthier products overall.45 Finally, the choice to purchase more fresh fruits, meats, and water (perceived as A-grade products) may also suggest compensatory behaviours designed to offset choosing some less healthy products. It may also be worth noting that in choosing more fresh fruits, meats and water (rather than increasing the purchase of vegetables), consumers may also seek to balance the perceived healthiness of their choices with perceived taste/palatability. These speculations as to the motivations and goals underlying specific food choices patterns following the introduction of a front-of-pack labelling scheme should be further explored in future studies, to devise efficient strategies to reinforce the observed trends.
The present study provides insights regarding the effect of the Nutri-Score on purchasing intentions of individuals with nutrition-related NCDs compared with the current labelling situation in France and other European countries. First, strength of the study pertained in the inclusion of a specific population rarely explored in the nutritional labelling field, and its randomised controlled design, which resulted in comparable groups allowing accurate estimations of the labels’ effect. Furthermore, the experiment was conducted on an experimental online supermarket, closed to real online grocery shopping conditions, with a range of different products with distinct nutritional profiles, brands and the use of real packaging. This controlled experimental environment allowed assessing the effect of the Nutri-Score in standardised conditions and optimising internal validity of the study. Finally, we provided intention-to-treat analyses of the participants (online supplemental table S1) and intention-to-treat estimates through multiple imputation methods. Nevertheless, some limitations should be acknowledged. First, a high rate of participants did not complete the shopping task. Hence, respondents may have different individual characteristics, leading however to a potential non-differential bias which could limit the generalisability of the results. In addition, the reduced sample size could have led to a decreased statistical power preventing us from detecting some potential small differences. Moreover, it is important to notice that analyses with multiple imputations led to similar trends but with non-significant differences given the wide variance in the sample. Second, the trial involved voluntary participants, who may have greater interest and knowledge in nutrition than the French population of patients. Thus, participants in the no label arm might have made healthier food choices than the general population and the effects of FoPLs in comparison could have been underestimated. Third, despite the diversity of the food offer proposed, the number of products was somewhat limited, and some participants may not have found their usual product and chose foods they would not buy in real shopping situation. In addition, the representativity of the experimental food offer was not carefully assessed. These elements would limit the extern validity of the study and the generalisability of the results to a real online supermarket. Moreover, compared with the French average, the higher proportion of subjects who declared doing often their grocery shopping online may have led to a sample with sociodemographic differences compared with the French population of patients. Fourth, the trial investigated purchasing intentions rather than actual food purchases that may have led the participants to take the experiment less seriously or to spend more money than they would actually do. Complementary studies should be conducted in real-life settings to provide additional elements on the Nutri-Score effectiveness. Nevertheless, virtual purchasing behaviours of individuals have been suggested to be good predictors of real behaviours.46 Finally, the study included cases of self-reported cardiometabolic chronic conditions with no validation required. Therefore, we were not able to ascertain whether the participants were following specific diets or nutritional recommendations during the period of the trial, which could have modified their purchasing behaviours. The present study focused on the Nutri-Score effect as a secondary or tertiary prevention tool of NCDs, and complement previous studies which have been conducted on the general population including individual without any chronic conditions, or on specific subgroups such as students. Furthermore, it could have been interesting to also include individuals having someone in the household with a chronic condition.
These results support that the Nutri-Score may improve the nutritional quality of food choices of consumers suffering from nutrition-related chronic diseases. This is particularly important given that an improvement of the dietary habits and the nutritional status of these individuals is a major element in the secondary prevention and the management of these NCDs. These findings are complementary to studies having observed a favourable effect of the Nutri-Score or its underlying nutrient profiling system on chronic diseases risk, in a context of primary prevention, through an improvement of food purchases and nutrient intakes.18 47
Supplementary Material
Acknowledgments
The authors would like to thank especially Younes Essedik (IT manager), Nathalie Pecollo (operational coordinator) for the design of the experimental supermarket, Paul Flanzy (computer technician) for computer management, Julien Allègre (data manager) for data extraction and datamanagement, Flora Demory, Apolline Caroux, Wassila Ait Haddad, Amaia Cherbero, Aurélie Gayon for their contribution in the implementation of the experimental supermarket, and University deans for their contribution and involvement in the recruitment of participants.
Footnotes
Twitter: @HercbergS
Contributors: ME, CJ and IB wrote the statistical analysis plan, analysed the data, and drafted and revised the paper. SP, PD, MT, PG, LF, RP, PR, SH and EK-G analysed the data and critically revised the paper for important intellectual content. SH and CJ designed data collection tools, implemented the study, monitored data collection for the whole study and critically revised the draft paper for important intellectual content. All authors, external and internal, had full access to all of the data (including statistical reports and tables) in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis. All authors have read and approved the final manuscript. CJ is the garantor
Funding: This project was funded by the French National Cancer Institute (INCA DA 2015-106). The NutriNet-Santé study was supported by the following public institutions: French Ministry of Health and Social Affairs, Santé Publique France, Institut National de la Santé et de la Recherche Médicale (INSERM), Institut National de la Recherche Agronomique (INRA), Conservatoire National des Arts et Métiers (CNAM) and Université Paris 13.
Competing interests: None declared.
Patient and public involvement: Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review: Not commissioned; externally peer reviewed.
Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
Ethics statements
Patient consent for publication
Not applicable.
Ethics approval
The study was approved by the Institutional Review Bard of the INSERM (IRB no. IRB0000388 FWA00005831), the National Commission for Data Protection and Liberties (CNIL no. 909216) and the Comité consultatif sur le traitement de l'information en matière de recherche dans le domaine de la santé. Participants gave informed consent to participate in the study before taking part.
References
- 1.World Health Organization . Diet, nutrition and the prevention of chronic diseases. in who technical report series; 916. Geneva, Switzeland: WHO, 2003. [PubMed] [Google Scholar]
- 2.Afshin A, Sur PJ, Fay KA, et al. Health effects of dietary risks in 195 countries, 1990–2017: a systematic analysis for the global burden of disease study 2017. The Lancet 2019;393:1958–72. 10.1016/S0140-6736(19)30041-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.L’état de santé de la population en France - RAPPORT 2017 - Ministère des Solidarités et de la Santé. Available: https://drees.solidarites-sante.gouv.fr/etudes-et-statistiques/publications/recueils-ouvrages-et-rapports/recueils-annuels/l-etat-de-sante-de-la-population/article/l-etat-de-sante-de-la-population-en-france-rapport-2017 [Accessed 5 Feb 2019].
- 4.Santé Publique France . Etude ESTEBAN 2014-2016 – Chapitre corpulence : stabilisation du surpoids et de l’obésité chez l’enfant et l’adulte, 2017. Available: https://www.santepubliquefrance.fr/les-actualites/2017/etude-esteban-2014-2016-chapitre-corpulence-stabilisation-du-surpoids-et-de-l-obesite-chez-l-enfant-et-l-adulte [Accessed 22 Oct 2019].
- 5.Santé Publique France . Prévalence et incidence Du diabète, 2019. Available: https://www.santepubliquefrance.fr/maladies-et-traumatismes/diabete/articles/prevalence-et-incidence-du-diabete [Accessed 13 Jan 2020].
- 6.Hong S-woo, Oh S-W, Lee C, et al. Association between nutrition label use and chronic disease in Korean adults: the fourth Korea National health and nutrition examination survey 2008-2009. J Korean Med Sci 2014;29:1457–63. 10.3346/jkms.2014.29.11.1457 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Cannoosamy K, Jeewon R. A critical assessment of nutrition labelling and determinants of its use and understanding. Progress in Nutrition, 2016. [Google Scholar]
- 8.Lewis JE, Arheart KL, LeBlanc WG, et al. Food label use and awareness of nutritional information and recommendations among persons with chronic disease. Am J Clin Nutr 2009;90:1351–7. 10.3945/ajcn.2009.27684 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Post RE, Mainous AG, Diaz VA, et al. Use of the nutrition facts label in chronic disease management: results from the National health and nutrition examination survey. J Am Diet Assoc 2010;110:628–32. 10.1016/j.jada.2009.12.015 [DOI] [PubMed] [Google Scholar]
- 10.Acton RB, Jones AC, Kirkpatrick SI, et al. Taxes and front-of-package labels improve the healthiness of beverage and snack purchases: a randomized experimental marketplace. Int J Behav Nutr Phys Act 2019;16:46. 10.1186/s12966-019-0799-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Ares G, Varela F, Machin L, et al. Comparative performance of three interpretative front-of-pack nutrition labelling schemes: insights for policy making. Food Qual Prefer 2018;68:215–25. 10.1016/j.foodqual.2018.03.007 [DOI] [Google Scholar]
- 12.Carrad AM, Louie JC-Y, Milosavljevic M, et al. Consumer support for healthy food and drink vending machines in public places. Aust N Z J Public Health 2015;39:355–7. 10.1111/1753-6405.12386 [DOI] [PubMed] [Google Scholar]
- 13.Finkelstein EA, Li W, Melo G, et al. Identifying the effect of shelf nutrition labels on consumer purchases: results of a natural experiment and consumer survey. Am J Clin Nutr 2018;107:647–51. 10.1093/ajcn/nqy014 [DOI] [PubMed] [Google Scholar]
- 14.Ministère des Solidarités et de la Santé . Evaluation ex ante de systèmes d’étiquetage nutritionnel graphique simplifié - Rapport final du Comité scientifique, 2017. Available: https://solidarites-sante.gouv.fr/IMG/pdf/rapport_comite_scientifique_etiquetage_nutritionnel_150317.pdf [Accessed 26 Jul 2019].
- 15.Mora-García CA, Tobar LF, Young JC. The effect of randomly providing Nutri-Score information on actual Purchases in Colombia. Nutrients 2019;11:491. 10.3390/nu11030491 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Ruffieux B, Muller L. Etude sur l’influence de divers systèmes d’étiquetage nutritionnel sur la composition du panier d’achat alimentaire, 2011. [Google Scholar]
- 17.Thorndike AN, Riis J, Sonnenberg LM, et al. Traffic-light labels and choice architecture: promoting healthy food choices. Am J Prev Med 2014;46:143–9. 10.1016/j.amepre.2013.10.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Julia C, Hercberg S. Development of a new front-of-pack nutrition label in France: the five-colour Nutri-Score. Public Health Panor 2017;3:537–820. [Google Scholar]
- 19.Marandola G, Ciriolo E, Storcksdieck Genannt Bonsmann, S . Front-of-pack nutrition labelling schemes: a comprehensive review. Luxembourg: Publications Office of the European Union, 2020. https://ec.europa.eu/jrc/en/publication/eur-scientific-and-technical-research-reports/front-pack-nutrition-labelling-schemes-comprehensive-review [Google Scholar]
- 20.Young L, Swinburn B. Impact of the Pick the tick food information programme on the salt content of food in New Zealand. Health Promot Int 2002;17:13–19. 10.1093/heapro/17.1.13 [DOI] [PubMed] [Google Scholar]
- 21.Vyth EL, Steenhuis IH, Roodenburg AJ, et al. Front-of-pack nutrition label stimulates healthier product development: a quantitative analysis. Int J Behav Nutr Phys Act 2010;7:65. 10.1186/1479-5868-7-65 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Communiqué de presse . Marisol Touraine se félicite des résultats des études sur l’impact d’un logo nutritionnel : leur intérêt et l’efficacité du logo Nutri-score sont démontrés. Paris, Ministère des Affaires sociales et de la Santé, 2017. [Google Scholar]
- 23.Anses, INRA, Oqali . Déploiement du Nutri-Score: analyse partir des données transmises l’Oqali. Paris, 2019. [Google Scholar]
- 24.CREDOC . Les enquêtes complémentaires. Etude quantitative. Paris: Ministère des Affaires sociales et de la Santé, 2017. http://social-sante.gouv.fr/IMG/pdf/rapport-credoc-fev_2017 [Google Scholar]
- 25.Julia C, Blanchet O, Méjean C, et al. Impact of the front-of-pack 5-colour nutrition label (5-CNL) on the nutritional quality of purchases: an experimental study. Int J Behav Nutr Phys Act 2016;13:101. 10.1186/s12966-016-0416-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Ducrot P, Julia C, Méjean C, et al. Impact of different Front-of-Pack nutrition labels on consumer purchasing intentions: a randomized controlled trial. Am J Prev Med 2016;50:627–36. 10.1016/j.amepre.2015.10.020 [DOI] [PubMed] [Google Scholar]
- 27.Crosetto P, Lacroix A, Muller L. Nutritional and economic impact of five alternative front-of-pack nutritional labels: experimental evidence. Eur Rev Agric Econ 2020;47:785–818. 10.1093/erae/jbz037 [DOI] [Google Scholar]
- 28.Egnell M, Boutron I, Péneau S, et al. Front-of-Pack labeling and the nutritional quality of students' food Purchases: a 3-Arm randomized controlled trial. Am J Public Health 2019;109:1122–9. 10.2105/AJPH.2019.305115 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Food and Drink Federation . Reference intakes (previously guideline daily amounts). food label. tool help improve food Lit. Consum, 2017. Available: http://www.foodlabel.org.uk/label/reference-intakes.aspx [Accessed 5 Feb 2017].
- 30.Hercberg S, Castetbon K, Czernichow S, et al. The Nutrinet-Santé study: a web-based prospective study on the relationship between nutrition and health and determinants of dietary patterns and nutritional status. BMC Public Health 2010;10:242. 10.1186/1471-2458-10-242 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Feunekes GIJ, Gortemaker IA, Willems AA, et al. Front-of-pack nutrition labelling: testing effectiveness of different nutrition labelling formats front-of-pack in four European countries. Appetite 2008;50:57–70. 10.1016/j.appet.2007.05.009 [DOI] [PubMed] [Google Scholar]
- 32.Hersey JC, Wohlgenant KC, Arsenault JE, et al. Effects of front-of-package and shelf nutrition labeling systems on consumers. Nutr Rev 2013;71:1–14. 10.1111/nure.12000 [DOI] [PubMed] [Google Scholar]
- 33.Aschemann-Witzel J, Grunert KG, van Trijp HCM, et al. Effects of nutrition label format and product assortment on the healthfulness of food choice. Appetite 2013;71:63–74. 10.1016/j.appet.2013.07.004 [DOI] [PubMed] [Google Scholar]
- 34.Malam S, Clegg S, Kirwan S. Comprehension and use of UK nutrition Signpost labelling schemes. Guildford, UK: University of Surrey, 2009. [Google Scholar]
- 35.Mejean C, Macouillard P, Péneau S, et al. Consumer acceptability and understanding of front-of-pack nutrition labels. J Hum Nutr Diet 2013;26:494–503. 10.1111/jhn.12039 [DOI] [PubMed] [Google Scholar]
- 36.Egnell M, Ducrot P, Touvier M, et al. Objective understanding of Nutri-Score Front-Of-Package nutrition label according to individual characteristics of subjects: comparisons with other format labels. PLoS One 2018;13:e0202095. 10.1371/journal.pone.0202095 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Egnell M, Talati Z, Hercberg S, et al. Objective understanding of Front-of-Package nutrition labels: an international comparative experimental study across 12 countries. Nutrients 2018;10:1542. 10.3390/nu10101542 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Helfer P, Shultz TR. The effects of nutrition labeling on consumer food choice: a psychological experiment and computational model. Ann N Y Acad Sci 2014;1331:174–85. 10.1111/nyas.12461 [DOI] [PubMed] [Google Scholar]
- 39.Kreuter MW, Brennan LK, Scharff DP, et al. Do nutrition label readers eat healthier diets? behavioral correlates of adults' use of food labels. Am J Prev Med 1997;13:277–83. 10.1016/S0749-3797(18)30175-2 [DOI] [PubMed] [Google Scholar]
- 40.Adriouch S, Lelong H, Kesse-Guyot E, et al. Compliance with nutritional and lifestyle recommendations in 13,000 patients with a cardiometabolic disease from the Nutrinet-Santé study. Nutrients 2017;9:546. 10.3390/nu9060546 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Egnell M, Boutron I, Péneau S, et al. Randomised controlled trial in an experimental online supermarket testing the effects of front-of-pack nutrition labelling on food purchasing intentions in a low-income population. BMJ Open 2021;11:e041196. 10.1136/bmjopen-2020-041196 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Egnell M, Galan P, Fialon M, et al. The impact of the Nutri-Score front-of-pack nutrition label on purchasing intentions of unprocessed and processed foods: post-hoc analyses from three randomized controlled trials. Int J Behav Nutr Phys Act 2021;18:38. 10.1186/s12966-021-01108-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Vandevijvere S, Berger N. The impact of shelf tags with Nutri-Score on consumer purchases: a difference-in-difference analysis of a natural experiment in supermarkets of a major retailer in Belgium. Int J Behav Nutr Phys Act 2021;18:150. 10.1186/s12966-021-01207-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Talati Z, Pettigrew S, Kelly B, et al. Consumers' responses to front-of-pack labels that vary by interpretive content. Appetite 2016;101:205–13. 10.1016/j.appet.2016.03.009 [DOI] [PubMed] [Google Scholar]
- 45.Bix L, Sundar RP, Bello NM, et al. To see or not to see: do front of pack nutrition labels affect attention to overall nutrition information? PLoS One 2015;10:e0139732. 10.1371/journal.pone.0139732 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Sharpe KM, Staelin R, Huber J. Using extremeness aversion to fight obesity: policy implications of contaxt dependent demand. J Consum Res 2008. [Google Scholar]
- 47.Egnell M, Crosetto P, d'Almeida T, et al. Modelling the impact of different front-of-package nutrition labels on mortality from non-communicable chronic disease. Int J Behav Nutr Phys Act 2019;16:56. 10.1186/s12966-019-0817-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
bmjopen-2021-058139supp001.pdf (147.1KB, pdf)
Data Availability Statement
All data relevant to the study are included in the article or uploaded as supplementary information.