Assessing the construct validity of five nutrient profiling systems using diet modeling with linear programming

Abstract

Nutrient profiling classifies individual food products according to their nutrient content. According to the WHO (World Health Organization), validation is a key step in the development of a nutrient profiling system. The aim was to assess the construct validity of five European nutrient profiling systems (Choices, Keyhole, (AFSSA), European Commission (EC) system and FoodProfiler). Construct validity was assessed for each of the five-selected nutrient profiling systems by testing whether healthy foods (that is, identified as eligible by the system) make healthy diets, and unhealthy foods (that is, non-eligible) make unhealthy diets, using diet modeling. The AFSSA, EC and FoodProfiler systems were identified as valid, but differences in their levels of permissiveness suggested some misclassified food products. The two other systems failed the construct validity assessment. Among these three systems, the EC system is the less demanding in terms of nutritional information, it would, therefore, be the easiest to implement for regulating nutrition and health claims in Europe.

Introduction

Nutrient profiling classifies individual food products according to their nutrient content. In the 1924/2006 European regulation, nutrient profiling is proposed as a tool for deciding whether a given food might be eligible or not for bearing nutrition or health claims.¹ One core aim of the regulation is ‘to avoid a situation where nutrition or health claims mask the overall nutritional status of a food product, which could mislead consumers when trying to make healthy choices in the context of a balanced diet'. Report of a WHO (World Health Organization)/IASO (International Association for the Study of Obesity) joint technical meeting recently highlighted the importance of using validated nutrient profiling systems and listed a number of possible validation methods.² These include assessment of construct validity, that is, testing whether healthy foods (identified as eligible by nutrient profile system) make healthy diets, and unhealthy foods (that is, non-eligible) make unhealthy diets. The aim of the present study was to assess the construct validity of five nutrient profiling systems, using diet modeling with linear programming for designing healthy and unhealthy diets.³

Materials and Methods

The dietary data and the food database were derived from the INCA French dietary survey.⁴ Each food of the food database (n=597) was allocated to one of two classes, eligible and non-eligible, according to each of the five European nutrient profile systems tested. Their main characteristics can be found in Table 1. Two systems are currently used for food labeling, namely Choices v2.2⁵ and Green Keyhole,⁶ two have been proposed to assess food products' eligibility to claim but that are not currently in use, namely the French AFSSA system (Agence Française de Sécurité Sanitaire des Aliments)⁷ and the EC system (European Commission draft),⁸ and FoodProfiler, a system in use for self-limiting advertisement to children.⁹

Table 1. Main characteristics of the five European systems studied.

Name of the system	Field of application	Nutrients to limit	Nutrients to encourage	Reference amount	Calculation	Number of categories
Choices v2.2	Labeling (In use)	Added sugars, SFA, Na, Energy	Fiber	100 g	Threshold	>25
Green Keyhole	Labelling (in use)	Added sugars, SFA, Na	Fiber	100 g	Threshold	>25
AFSSA system	Claim Eligibility (draft)	Added sugars, SFA, Na	Fiber, Protein, Vitamin C, Vitamin D, Ca, Fe	100 kcal 100 g	Scoring	Across the board
EU system	Claim Eligibility (draft)	Added sugars, SFA, Na	None	100 g	Threshold	13
FoodProfiler	Advertising to children eligibility (in use)	Total fat, Added sugars, Added Na, SFA, TFA	Fiber, PUFAs, Ca, Fe	100 kcal	Scoring	Across the board

Abbreviation: AFSSA, Agence Française de Sécurité Sanitaire des Aliments; EC, European Commission; SFA, saturated fatty acids; TFA, trans fatty acids.

Using a previously described diet modeling approach,³ the feasibility of designing healthy or unhealthy diets with eligible foods only, or with non-eligible foods only, was tested for each nutrient profiling system. Healthy diets were defined by the fulfillment of a set of a forty nutrient constraints, and unhealthy diets by the nonfulfillment of the same set of nutrient constraints.³ To ensure realism to the modeled diets, constraints on foods and food groups were included and the possibility to design a 2000-kcal diet was tested.³ Four feasibility tests were used to assess the construct validity of each system. Tests A, eating healthily with eligible foods only, and D, eating unhealthily with non-eligible foods only, were used to assess whether the systems respected the nutritional common sense. Tests B, eating unhealthily with eligible foods only, and C, eating healthily with non-eligible foods only, assessed more finely the level of permissiveness. The following terminology is used: ‘Strict unfeasibility' means that no mathematical solution can be found at all. When solutions can be found, the range between the minimum and the maximum energy achievable is called the ‘energy range'. When the energy range includes 2000 kcal, the model is considered feasible. When the energy range excludes 2000 kcal, the model is considered unfeasible. A system is considered as valid when tests A and D are feasible, and tests B and C are unfeasible.

Results

The percentage of eligible foods varied between systems: Choices: 31.8% Green Keyhole: 35.6% AFSSA: 35.8% EC system: 46.5% and FoodProfiler: 49.5%.

Figure 1 shows that all tested systems allowed 2000 kcal healthy diets with eligible foods (test A), and 2000 kcal unhealthy diets with non-eligible foods (test D).

Figure 1.

Energy range, in kcal, to design healthy/unhealthy diets with eligible/non-eligible foods. Test A assesses whether it is feasible to design healthy diets with eligible foods (it is expected to be feasible). Test B assesses whether it is feasible to design unhealthy diets with eligible foods (it is expected to be unfeasible). Test C assesses whether it is feasible to design healthy diets with non-eligible foods (it is expected to be unfeasible). Test D assesses whether it is feasible to design unhealthy diets with non-eligible foods (it is expected to be feasible). ‘Strictly unfeasible' means that no mathematical solution can be found at all. When the energy range includes 2000 kcal, the model is considered feasible. When the energy range excludes 2000 kcal, the model is considered unfeasible.

Regarding test B, all tested systems exclude 2000 kcal from their energy range and were, therefore, considered as unfeasible: strict unfeasibility was observed for Choices and Green Keyhole systems. Unfeasibility was observed also for the FoodProfiler and EC systems, the maximum energy achievable being 1772 kcal and 1801 kcal, respectively, and for the AFSSA system, with a narrow and unrealistic energy range (587–907 kcal).

Regarding test C, strict unfeasibility was observed for FoodProfiler only. Test C was feasible for Choices and Green Keyhole (2000 kcal healthy diets allowed with non-eligible foods) and it was unfeasible for AFSSA and EC systems (the energy range for test C excluded 2000 kcal but the minimum energy achievable was close to 2000 kcal: 2018 kcal and 2162 kcal, respectively).

Discussion

One important result is that, with all tested systems, it was possible to design healthy diets with eligible products and unhealthy diets with non-eligible products (tests A&D), demonstrating that all systems satisfied the nutritional common sense. In line with the latter results, Choices was previously presented as valid based on its beneficial impact on dietary quality in modeling studies simulating the replacement of foods not complying with the Choices criteria by existing Choices-compliant products.¹⁰ Nevertheless, Choices and Green Keyhole allowed to design 2000 kcal healthy diets with non-eligible products, and were, therefore, identified as invalid according to this construct validity assessment. In contrast, FoodProfiler, AFSSA and EC systems were identified as valid because they answered as expected to the four tests.

The three valid systems showed differences in their level of permissiveness. With the AFSSA system, it was almost feasible to design a 2000-kcal healthy diet with non-eligible foods (test C), which suggests that some healthy foods are misclassified as non-eligible. Therefore, this system rather display a ‘keep the best' approach when classifying the unhealthy food products. In the opposite, with the FoodProfiler and the EC system, it was almost feasible to design a 2000-kcal unhealthy diet with eligible foods (test B), which shows that these systems rather display a ‘stop the worst' approach by misclassifying some unhealthy foods as eligible. Analyzing the foods selected by linear programming in these tests could help identify the misclassified foods, and then fine-tune the systems.

The validation method used in the present study is perfectible, for instance by improving the realism of diets using more sophisticated diet modeling techniques.¹¹ Nevertheless, this study revealed that some nutrient profiling systems, among those currently used, show weaknesses by failing this construct validity assessment.

Among the three systems identified as valid in the present study, the EC system is the less demanding in terms of nutritional information, it would, therefore, be the easiest to implement for regulating nutrition and health claims in Europe.

The authors declare no conflict of interest.