Nutrition Classification Schemes for Informing Nutrition Policy in Australia: Nutrient-Based, Food-Based, or Dietary-Based?

ABSTRACT

Background

Policy makers are increasingly using nutrition classification schemes (NCSs) to assess a food's health potential for informing nutrition policy actions. However, there is wide variability among the NCSs implemented and no standard benchmark against which their contrasting assessments can be validated.

Objectives

This study aimed to compare the agreement of nutrient-, food-, and dietary-based NCSs in their assessment of a food's health potential within the Australian food supply, and examine the conceptual underpinnings and technical characteristics that explain differences in performance.

Methods

A dataset combining food compositional data from the Mintel Global New Products Database and the Australian Food Composition Database (AUSNUT 2011–2012) (n = 7322) was assembled. Products were classified by 7 prominent NCSs that were selected as representative of one or other of 1) nutrient-based NCSs [the Chilean nutrient profile model (NPM), Health Star Rating (HSR), Nutri-Score, the WHO European Region's NPM (WHO-Euro NPM), and the Pan American Health Organization's (PAHO) NPM]; 2) food-based NCS (NOVA), and 3) dietary-based NCS [Australian Dietary Guidelines (ADGs)].

Results

The PAHO NPM classified the lowest proportion (22%) of products as “healthy”, and the HSR the highest (63%). The PAHO NPM, NOVA, WHO-Euro NPM, and the Chilean NPM classified >50% of products as “unhealthy,” and the ADGs, HSR, and Nutri-Score classified <50% of products as “unhealthy.” The HSR and Nutri-Score had the highest pairwise agreement (κ = 0.7809, 89.70%), and the PAHO NPM and HSR the lowest (κ = 0.1793, 53.22%). Characteristics of NCSs that more effectively identified ultraprocessed and discretionary foods were: category-specific assessment, the classification of categories as always “healthy” or “unhealthy,” consideration of level of food processing, thresholds for “risk” nutrients that do not penalize whole foods; and no allowance for the substitution of ingredients.

Conclusions

Wide variation was observed in agreement of the assessment of a food's health potential among the NCSs analyzed due to differing conceptual underpinnings and technical characteristics.

High disagreement in the classification of “healthiness” observed between seven different schemes that attempt to classify the health potential of individual foods for policy purposes.

Introduction

Dietary risk factors are a leading cause of the burden of disease globally, implicated in the development of noncommunicable diseases, such as cardiovascular disease, diabetes, and some cancers (1). Comprehensive policies targeting multiple dimensions of the food system are recommended to tackle these dietary risk factors. Nutrition policy actions such as food taxes and subsidies, restrictions on food advertising, public food procurement, and interpretive front-of-pack labeling (FOPL), have been consistently recommended to modify the food environment and help promote healthy dietary behaviors (2–4).

The implementation of policy actions often requires assessment of the health potential of individual foods and beverages (herein referred to as “foods”), operationalized through a nutrition classification scheme (NCS). An example would be deciding what specific foods would not be eligible to display health claims or be allowed to be advertised to children. An NCS is defined in this article as a method developed to classify individual foods or food groups in relation to their claimed health potential. Currently there are contested views on the conceptual and technical approaches to NCSs. There are conceptual differences in their alignment with 1 of 3 types of nutrition exposure—nutrients, foods, or dietary patterns—and technical differences in the way they are implemented (e.g., the nutrients considered, the thresholds used, and binary categories compared with rating scales). Consequently, there is wide variability in which type of nutrition exposure and what specific technical aspects are applied to NCSs for informing nutrition policy actions.

Nutrient-based NCSs are informed by evidence on the effects of certain nutrients and food components on metabolic processes or health outcomes, and can differ technically in the nutrients included and calculations employed. Examples include nutrient profiling models (NPMs) developed for FOPL, such as the French Nutri-Score and the Australasian Health Star Rating (HSR) system (5, 6). These schemes use an algorithm to calculate a “healthiness” score based on the amounts of certain so-called “risk” and “beneficial” nutrients present in the food. Both Nutri-Score and the HSR were adapted from the Food Standards Agency United Kingdom Nutrient Profiling Model (FSA-UK NPM) developed in 2004 for the regulation of marketing to children (7). Nutrient profiling can also take a binary approach, wherein foods exceeding upper thresholds for “risk” nutrients are considered “unhealthy” or “high in” critical nutrients. For example, the NPM developed for the Chilean Food Act, which is applied to FOP warning labels, the regulation of food products sold in schools, and restrictions to advertising targeting children, all employ an upper threshold approach (8).

Food-based NCSs are informed by evidence of a food's structure or composition and associations with health outcomes. There are multiple ways of categorizing the health potential of foods at the food-based level, for example: by contribution to certain macro- or micronutrients; by eating occasion or preparation method (e.g., home-cooked, convenience, or snack foods); or by level of processing (9, 10). The NOVA classification system, which categorizes foods into 4 groups based on the nature, extent, and purpose of their processing, has been the most widely applied scheme for this purpose, and is considered the most objective, comprehensive, and clear (11). Of particular relevance is NOVA group 4, ultraprocessed foods (UPFs), which are defined as “formulations of ingredients, mostly of exclusive industrial use, that result from a series of industrial processes” (12). There is now a substantial body of evidence showing an association between UPF consumption and adverse health outcomes (13–15). The WHO European Region's (WHO-Euro) NPM, developed to inform decisions about which foods to restrict in relation to the marketing of food to children, combines nutrient- and food-based aspects, applying specific upper nutrient thresholds depending on the food type or category (16). Similarly, the Pan American Health Organization's (PAHO's) NPM applies upper nutrient thresholds to processed (group 3) and UPFs as categorized by the food-based NOVA classification system (17).

Diet-based NCSs are informed by evidence on dietary patterns (the types, amounts, and combinations of foods over the whole diet) and health outcomes. Diet-based schemes generally consider the concepts of variety, adequacy, and moderation and include indices derived from national food-based dietary guidelines, dietary patterns such as the Mediterranean diet, and scoring indices created for evaluation purposes, such as the Diet Quality Index International (18). Variety, adequacy, and moderation are difficult concepts to incorporate into the assessment of an individual food's health potential; consequently validated diet-based NCSs that target individual foods for policy purposes do not exist. However, a binary interpretation of the Australian Dietary Guidelines (ADGs), wherein foods are classified as recommended nutritious “five food group” (FFG) foods or nonrecommended discretionary foods, is considered a diet-based NCS for the purposes of this research (19).

The vast majority of implemented nutrition policy is informed by nutrient-based NCSs (20). However, the evidence for the accuracy of nutrient-based schemes to correctly identify “healthy” and “unhealthy” food products is mixed, because currently there is no standard benchmark against which the validity of an NCS can be assessed; thus they are compared with other nutrient-based schemes or nutrient recommendations. The reductionist focus on nutrients in classification schemes reflects a wider dominant nutrient paradigm within the nutrition science discipline (21), despite a shift in thinking in recent decades toward greater consideration of the role of food processing and whole dietary patterns in health outcomes (14, 22).

A proliferation of FOPLs has emerged in recent years (23), leading to an international move to create guidelines for harmonization. The Codex Alimentarius is in the process of finalizing international guidelines for FOPL and working on potential guidelines for labeling-specific nutrient profiling (24), and a proposal is being prepared for a harmonized mandatory FOPL in the European Union (25). It is therefore timely to investigate the different conceptual and technical approaches to classifying a food's healthiness for policy action purposes. This study aimed to compare the agreement of nutrient-based, food-based, and dietary-based NCSs in their assessment of a food's health potential within the Australian food supply, and examine the conceptual underpinnings and technical characteristics that explain differences in performance.

Methods

Data collection

Food and nutrient data were obtained from a combination of the most recent Australian Food Composition Database, AUSNUT 2011–2013 (herein referred to as AUSNUT) (26), and Mintel's Global New Product Database 2014–2019 (herein referred to as Mintel) (27). The combination of these 2 databases helped to ensure a wide variety of food and beverage products available in the Australian food supply, from fresh unpackaged products to novel packaged processed products. AUSNUT provides data on a wide range of food types in the food supply between 2011 and 2013, and Mintel provides data on all new products launched in the market in subsequent years. Therefore, the final sample provides examples of food types available or launched in the Australian food supply between 2011 and 2019.

Mintel's Global New Products Database

Mintel collects detailed information on newly released packaged food and beverage products worldwide (27). The Mintel search, conducted in June 2020, was limited to: products released in the last full 5-y calendar period (January 2014 to December 2019); products released in the Australian marketplace; and food and beverage categories, excluding the subcategories “baby food” and “alcoholic beverages.” The 5-y time frame reflects products currently available in the retail marketplace with sufficient variety to ensure a diverse dataset. The search was limited to the Australian marketplace to reduce inconsistencies that could arise from variations in the cultural, market, and policy drivers of food supplies in different countries. Baby foods and alcoholic beverages were excluded from the sample because they are usually subject to specific regulations that do not require an NCS.

Product duplications due to packaging updates, variety packs, and products missing nutrient information or sufficient ingredient information, and multiple similar-type products were deleted, leaving just 1 variety for analysis. Similar products included items such as olive oil, tomato sauce, eggs, plain yogurts, cheddar cheese, and packaged fruits. These products were overrepresented in the data, and had similar or identical ingredients, level of processing, or nutrient profiles. Products were excluded if they were food items that do not provide significant energy to the diet and/or were not likely to be the target of nutrition policy actions (e.g., tea leaves and ground spices).

A stratified random sample of the full 5-y Mintel dataset was created in Stata version 16 (28). Products in each Mintel subcategory had differing numbers and/or varieties of products; for example, the “chocolate confectionery” subcategory had low variety, but a high number of products. The following logic was applied to deal with this issue:

Take a random sample of 40 items in each subcategory, or allitems in the subcategory if the total is <40.

For items with a high level of variety take a 15% random sampleOR 80 products if 15% sample is <80.

Level of variety was determined using the 9 food groups outlined in the ADGs methodology described in previous research (19) (Supplemental File 1). If a subcategory contained products from >2 ADG food groups, it was determined to have a high level of variety.

AUSNUT 2011–2013

AUSNUT was developed to analyze the results of the 2011–2013 Australian Health Survey, and reflects the food supply in Australia at that time. AUSNUT contains data for 53 nutrients and related components for foods and beverages, estimated by averaging the nutrient content of similar products available in the food supply in 2011–2013. AUSNUT has previously been classified using the NOVA scheme by Machado et al. (29). Certain items in the database were not deemed relevant for the purposes of this study and were removed, such as sugar, flour, dried spices, and alcoholic beverages, because most NCSs do not include assessment of basic culinary ingredients, ingredients with very low energy, or alcoholic beverages. Furthermore, items described as “homemade,” “from café or restaurant,” or “takeaway,” were removed because they were classified into NOVA groups by disaggregating the underlying ingredients, which are already present in the dataset. This left only food and beverage items sold in retail settings and fast-food chains.

Data analysis

Nutrition classification schemes

All products in the combined AUSNUT and Mintel dataset were classified in accordance with 7 prominent NCSs. A prominent NCS for the purposes of this study was defined as a scheme that satisfies ≥1 of the following criteria: 1) is widely evaluated in the peer-reviewed literature; 2) has been/or will be applied to public policy actions at the national level; or 3) has been developed by a reputable international health organization. The NCSs were also chosen to ensure that ≥1 of each of the 3 conceptual levels—nutrients, foods, and dietary patterns—was represented. The 7 selected NCSs and the conceptual level they represent are shown in Table 1. They were: the Chilean NPM, the HSR (30), Nutri-Score (5), the WHO-Euro NPM (16), the PAHO NPM (17), NOVA(12), and the ADGs (31). The methodology used for classifying each NCS is described below, and a detailed overview of each scheme is provided in Supplemental Table 1. A syntax created in Stata version 16 was used to automatically calculate the PAHO NPM, WHO-Euro NPM, HSR, Nutri-Score, and Chilean criteria, after the input of all necessary nutrient and categorical information.

TABLE 1.

Description of purpose and scoring criteria of each nutrition classification scheme studied¹

Conceptual basis	NCS	Description	Scoring criteria
Nutrient-based	Chilean NPM (2019 limits)	Developed for the Chilean Food Act and applied to FOPLs, the regulation of food products sold in schools, and restrictions to advertising targeting children. Came into effect in 2016 (8)	Foods are subject to regulation when they exceed thresholds for “risk” nutrients; e.g., for FOPL regulation the product would receive a “high in” warning label. Energy, added sugar, added sodium, and added saturated fat are assessed
	Nutri-Score	Designed specifically for FOPL and first implemented by the French government in 2017 (5). Foods score a rating on a scale from A to E, with A indicating a “healthier” product, and E an “unhealthier” product	Scoring depends on the food category. An algorithm balances the quantity of beneficial and risk nutrients. Risk nutrients assessed are energy, total sugar, sodium, and saturated fat. Beneficial components assessed are fiber, protein, and fruit, vegetable, nut, and legume content
	HSR	Designed specifically for FOPL and implemented in Australia and New Zealand in 2014 (6). Foods score a rating on a scale from 0.5 stars to 5 stars, in half-star increments, more stars indicating a “healthier” product	Scoring depends on the food category. An algorithm balances the quantity of beneficial and risk nutrients. Risk nutrients assessed are energy, total sugar, sodium, and saturated fat. Beneficial components assessed are fiber, protein, and fruit, vegetable, nut, and legume content
Nutrient- and food-based	WHO-Euro NPM	Published in 2015 and specifically designed for the purpose of restricting the marketing of foods to children in the European region (16)	Uses a binary approach, wherein foods exceeding upper thresholds for certain nutrients, depending on the food category, are not permitted for marketing. Assesses energy, total sugar, added or free sugars, nonsugar sweeteners, sodium, total fat, and saturated fat
	PAHO NPM	Developed in 2016 to be used in the design of all forms of regulation on food and nonalcoholic beverages in PAHO member states (17)	Processed (group 3) and ultraprocessed foods (group 4, using NOVA) that are in excess of free sugars, nonnutritive sweeteners, salt, total fat, saturated fat, and trans fatty acids are classified as “unhealthy”
Food-based	NOVA	First published in 2009 and incorporated into dietary guidelines of Brazil, Uruguay, Ecuador, Peru, Israel, and Malaysia (13), and applied to procurement guidelines for the Brazilian Ministry of Health and its entities (32)	Classifies foods into 4 groups based on the nature, extent, and purpose of industrial processing (13). Ultraprocessed foods (group 4) are identified by the presence of certain processed food substances or cosmetic additives
Dietary-based	ADGs (2013)	The ADGs are a tool to inform policy in Australia (31)	A binary classification of the ADGs identifies foods as either discretionary, for which the ADGs advise limited consumption, or recommended nutritious “five food group” foods

¹ADGs, Australian Dietary Guidelines; FOPL, front-of-pack labeling; HSR, Health Star Rating; NCS, nutrition classification scheme; NPM, nutrient profile model, PAHO, Pan American Health Organization; WHO-Euro, World Health Organization European Division.

Chilean NPM

Both the Mintel and AUSNUT datasets provided sufficient nutrient information for the calculation of the Chilean NPM. For Mintel data, ingredient lists were manually examined to determine whether critical nutrients had been added, for example, the presence of cane sugar, sugar syrups, salt, butter, or palm oil. For AUSNUT data, recipe files, ingredient lists of example products, and nutrient information were used to estimate whether critical nutrients had been added.

Nutri-Score

All products were first classified into 1 of the 6 Nutri-Score categories (Supplemental File 1). Neither dataset contained fruit, vegetable, nut, and legume (FVNL) details and Mintel did not provide fiber content. The methods to estimate FVNL content and fiber are outlined in Supplemental File 1.

Health Star Rating system

All products were classified into 1 of the 6 HSR categories (Supplemental File 1). All nutrient information was available in the datasets for the calculation of the HSR except FVNL% and fiber. The methodology for estimating FVNL% and fiber was the same as Nutri-Score (described above) (30).

WHO-Euro NPM

The calculation of the WHO-Euro NPM first involved classification of all products into 1 of the 20 categories (16). All nutrient information required was available in the AUSNUT dataset, and all nutrients except for free sugars were available in the Mintel dataset. The presence of free sugars was determined by examining product ingredient lists. Free sugars were defined using the WHO definition (33).

PAHO NPM

The classification of the PAHO NPM first involved the application of NOVA (see below). All nutrient information required was available in the AUSNUT dataset, and all nutrients except for free sugars were available in the Mintel dataset. Free sugars were estimated using the methodology recommended in the PAHO NPM guide (17). Free sugars were defined using the WHO definition (33).

NOVA

The AUSNUT dataset has previously been classified using NOVA by Machado et al. (29). For Mintel, the most recent published description of the NOVA groups (12) guided classification of groups 1, 2, and 3. Group 4 foods, or UPFs, were identified based on the presence of a processed food substance or cosmetic additive in the ingredient list. Research by Monteiro et al. (12) was referred to for the identification of UPF-characterizing ingredients, and the purpose of novel processed ingredients was discussed amongst all authors until a consensus was reached. A conservative approach to the classification of novel ingredients was employed, whereby ingredients were not considered UPF-characterizing when authors were uncertain of the purpose.

Australian Dietary Guidelines

Food products were classified by SD into 2 categories: FFGs and discretionary foods. Discretionary items had previously been flagged in AUSNUT using a methodology developed by the Australian Bureau of Statistics (ABS) for the analysis of the 2011–2012 Australian Health Survey (34). The ABS procedure was referred to for classification of ADGs in the Mintel dataset. Because not all products easily translated to an AUSNUT food item, a decision framework was created and followed when products were difficult to classify, described in previous research (19). Where there were difficulties in classification, all authors reached a consensus decision on the food's classification based on the ingredients list and food purpose.

Two of the authors (JW and PM) completed a random 5% quality control check on the classification of the WHO-Euro NPM, NOVA, and ADG categories. To simplify the interpretation of results, products in both Mintel and AUSNUT were categorized into a common food grouping system consisting of 15 categories and 68 subcategories adapted from the Global Food Monitoring Group food categorization system (35).

Healthy and unhealthy categories

The scaled ratings of HSR and Nutri-Score were grouped into a binary output for statistical analysis and comparison with the other NCSs. The HSR and Nutri-Score were both designed as comparative scales, and therefore a cutoff point delineating “healthy” and “unhealthy” food products was not specified. As interpretive FOPLs, an appropriate cutoff could be determined by understanding the point at which a consumer's perception of health is modified; however, currently there is no evidence on the cutoff at which this occurs for either scheme. In the absence of evidence, this study selected an A, B, or C rating on the Nutri-Score scale and 2.5/5 on the HSR scale as logical cutoffs for delineating between healthy and unhealthy food products. For the Chilean NPM all products that exceeded the “high in” threshold for ≥1 nutrient were categorized as unhealthy. For NOVA, groups 1–3 were classified as healthy, and UPFs as unhealthy. For the ADGs the FFG foods and culinary ingredients were classified as healthy, and discretionary foods as unhealthy. The WHO-Euro NPM and PAHO NPM already delineate binary categories in their respective classifications.

Statistical analysis

All statistical analyses were conducted using Stata version 16 (28). The frequency of food products classified as healthy or unhealthy for each NCS was produced for the total sample and per food category and subcategory. Cohen κ agreement coefficient and percentage agreement were calculated for each NCS pairing for the total sample. The κ coefficient was interpreted using Landis and Koch (36): <0.0 = poor; 0.00–0.20 = slight; 0.21–0.40 = fair; 0.41–0.60 = moderate; 0.61–0.80 = substantial; and 0.81–1.00 = almost perfect. The 95% CIs and P values (to test if κ values were significantly different from zero) were calculated for each Cohen κ coefficient. A high degree of agreement for the whole sample was defined as >50%, and high disagreement at <25%. For NCS pairings where there was a high degree of disagreement a detailed qualitative analysis was conducted, identifying why differences were occurring within each food subcategory and at the product-based level.

Qualitative analysis

This study used an inductive research design to examine how the conceptual and technical characteristics of the different NCSs might explain observed differences in performance. Examples of specific products illustrative of the differing classifications between NCSs were chosen and the ingredients and classifications extracted. Using a qualitative analysis, key themes were identified from the data and conclusions were drawn about the conceptual and technical explanations for the differences.

Results

A total of 7322 products were assessed, 3002 from AUSNUT, providing an overview of the Australian food supply in 2011–2013, and 4320 products from Mintel, representing newly released products in the Australian food supply from 2014 to 2019. The PAHO NPM classified the highest proportion of the total sample as unhealthy at 77% (Table 2). The WHO-Euro NPM, NOVA, and Chilean NPM also classified over half the total sample as unhealthy, at 69.8%, 65.6%, and 58.1%, respectively.

TABLE 2.

Frequency of products classified by each nutrition classification scheme (converted to binary output) by food category¹

	ADGs		NOVA		PAHO		WHO-Euro		HSR		Nutri-Score		Chilean
Food category	FFG	Discretionary	Non-UPF	UPF	Healthy	Unhealthy	Healthy	Unhealthy	≥2.5	<2.5	A–C	D–E	Healthy	Unhealthy
Bread and bakery products	337	379	204	512	59	657	119	597	358	358	343	373	86	630
Cereal and cereal products	535	101	190	446	224	412	306	330	563	73	541	95	306	330
Confectionery	0	244	5	239	0	244	0	244	30	214	34	210	16	228
Convenience foods	499	290	92	697	10	779	318	471	657	132	611	178	373	416
Dairy	592	330	222	700	84	838	100	822	574	348	544	378	258	664
Edible oils and oil emulsions	98	65	86	77	73	90	89	74	96	67	51	112	92	71
Eggs	21	0	20	1	20	1	21	0	19	2	17	4	19	2
Fish and seafood products	157	32	139	50	95	94	162	27	170	19	159	30	134	55
Fruit and vegetables	767	169	741	195	535	401	548	388	837	99	857	79	678	258
Meat and meat products	485	183	388	280	326	342	454	214	413	255	403	265	410	258
Nonalcoholic beverages	227	532	125	634	99	660	45	714	224	535	361	398	443	316
Sauces, spreads, and seasonings	92	462	150	404	26	528	32	522	283	271	292	262	134	420
Snack foods	97	355	106	346	26	430	12	440	295	157	211	241	80	372
Special foods	9	76	1	84	3	82	3	82	62	23	55	30	22	63
Sugar, honey, and related products	30	158	52	136	39	149	0	188	29	159	31	157	22	166
Total	3946	3376	2521	4801	1615	5707	2209	5113	4610	2712	4510	2812	3073	4249
Percentage	53.9	46.1	34.4	65.6	22.1	77.9	30.2	69.8	63.0	37.0	61.6	38.4	41.9	58.1

¹Total sample n = 7322. ADGs, Australian Dietary Guidelines; FFG, “five food group” food; HSR, Health Star Rating; NPM, nutrient profile model; PAHO, Pan American Health Organization; UPF, ultraprocessed food; WHO-Euro, World Health Organization European Division.

The HSR classified the highest proportion of the total sample as healthy (≥2.5 stars) at 63%, closely followed by Nutri-Score (A–C) at 61.6%, with both NCSs rating products similarly at the category level. The HSR and Nutri-Score had the highest overall agreement according to the Cohen κ coefficient (κ = 0.7809, 89.70%) (Table 3), interpreted as substantial agreement (Table 3). The lowest level of agreement was observed between PAHO NPM and the HSR (κ = 0.1793, 53.22%), interpreted as slight agreement, mostly occurring for products classified as unhealthy by the PAHO NPM and healthy (≥2.5 stars) by the HSR (n = 3046).

Table 3.

Agreement between each nutrition classification scheme (converted to binary output) measured by the Cohen κ coefficient (with 95% CIs), and frequency of products classified into each binary category¹

¹High degree of alignment highlighted at >50% of sample; high degree of misalignment highlighted at >25% of sample. κ coefficient interpretation using Landis and Koch (36): <0.0 = poor; 0.00–0.20 = slight; 0.21–0.40 = fair; 0.41–0.60 = moderate; 0.61–0.80 = substantial; 0.81–1.00 = almost perfect. ADGs, Australian Dietary Guidelines; FFG, “five food group” food; HSR, Health Star Rating; NPM, nutrient profile model; PAHO, Pan American Health Organization; UPF, ultraprocessed food; WHO-Euro, World Health Organization European Division.

Figure 1 presents a comparison of the proportion of products classified as healthy or unhealthy at the food category level for each NCS, and Table 4 presents a breakdown of agreement at the food category level for each NCS (95% CIs are presented in Supplemental Table 2, and subcategory agreement is presented in Supplemental File 2). The categories of eggs; fruit and vegetables; sugar, honey and related products; and confectionery had little variation in classification (in both proportions and pairwise agreement) amongst all NCSs. Conversely, the categories of convenience foods, snack foods, dairy, and cereal and cereal products had wide variation in classification.

FIGURE 1.

Percentage healthy and unhealthy by food category for each nutrition classification scheme. Nutri-Score: A–C = healthy; D–E = unhealthy. ADGs, Australian Dietary Guidelines (“five food group” foods = healthy; discretionary foods = unhealthy); HSR, Health Star Rating (healthy ≥2.5, unhealthy <2.5); NPM, nutrient profile model; PAHO, Pan American Health Organization; WHO-Euro, World Health Organization European Division.

TABLE 4.

Percentage agreement and Cohen κ by food category for each nutrition classification scheme pairing for the total sample¹

		NOVAADGs		NOVAPAHO NPM		NOVAWHO-Euro NPM		NOVAHSR		NOVANutri-Score		NOVAChilean NPM		HSRChilean NPM
Food category	n	%AG		%AG		%AG		%AG		%AG		%AG		%AG
Bread and bakery products	716	77.2	0.53	74.2	0.20	74.7	0.29	71.8	0.44	71.9	0.43	74.3	0.24	61.7	0.23
Cereal and cereal products	636	45.8	0.15	83.3	0.62	69.5	0.38	40.7	0.10	43.9	0.12	73.9	0.47	59.3	0.21
Confectionery	244	98.0	*	98.0	*	98.0	*	86.5	*	84.8	*	91.4	*	91.8	0.53
Convenience foods	789	42.3	0.04	88.3	0.08	60.7	0.08	26.6	0.03	31.4	0.04	55.3	0.07	63.0	0.29
Dairy	922	53.6	0.19	84.2	0.45	74.3	0.14	46.2	0.05	38.2	*	69.0	0.20	53.4	0.16
Edible oils and oil emulsions	163	43.6	*	92.0	0.84	45.4	*	42.3	*	52.8	0.07	62.6	0.25	76.1	0.51
Eggs	21	95.2	*	100.0	1.00	95.2	*	85.7	*	76.2	*	95.2	0.64	81.0	*
Fish and seafood products	189	88.4	0.66	76.7	0.53	76.2	0.28	69.8	0.03δ	73.5	0.22	80.4	0.51	77.8	0.33
Fruit and vegetables	936	83.8	0.48	75.6	0.47	72.0	0.38	79.3	0.23	80.6	0.25	73.2	0.27	81.5	0.43
Meat and meat products	668	75.6	0.47	90.1	0.80	75.2	0.47	62.1	0.21	63.9	0.25	83.2	0.65	76.2	0.50
Nonalcoholic beverages	759	77.6	0.39	94.2	0.77	84.2	0.23	66.9	0.09	60.2	0.18	53.9	0.17	68.3	0.41
Sauces, spreads, and seasonings	554	78.3	0.38	77.6	0.23	74.7	0.15	59.8	0.20	58.8	0.20	74.6	0.33	67.2	0.35
Snack foods	452	72.8	0.22	77.0	0.13	78.8	0.15	50.2	0.14	60.4	0.18	76.1	0.27	52.0	0.20
Special foods	85	88.2	*	95.3	*	95.3	*	28.2	0.01	36.5	0.01	75.3	0.07	50.6	0.19
Sugar, honey, and related products	188	56.4	*	89.9	0.73	72.3	*	56.9	*	56.9	*	60.6	*	86.7	0.43
		PAHO NPM ADGs		PAHO NPM WHO-Euro NPM		PAHO NPM HSR		PAHO NPM Nutri-Score		PAHO NPM Chilean NPM		WHO-Euro NPM HSR		WHO-Euro NPM Nutri-Score
Food category	n	%AG		%AG		%AG		%AG		%AG		% AG		%AG
Bread and bakery products	716	59.5	0.15	83.8	0.27	57.7	0.15	59.8	0.17	94.0	0.67	66.6	0.33	68.7	0.36
Cereal and cereal products	636	50.5	0.18	76.7	0.53	46.4	0.13	49.8	0.17	81.5	0.63	59.6	0.22	63.1	0.28
Confectionery	244	100.0	1.00	100.0	1.00	87.7	*	86.1	*	93.4	*	87.7	*	86.1	*
Convenience foods	789	37.8	0.01	60.5	0.03	17.7	0.002	23.6	0.004	53.7	0.02	57.0	0.24	62.6	0.32
Dairy	922	40.4	0.03	86.4	0.25	41.0	0.02	45.6	0.05	77.2	0.29	46.3	0.10	50.9	0.14
Edible oils and oil emulsions	163	49.1	0.003	50.9	0.03	47.9	*	58.3	0.13	68.1	0.37	95.7	0.91	73.0	0.48
Eggs	21	95.2	*	95.2	*	85.7	*	76.2	*	95.2	0.64	90.5	*	81.0	*
Fish and seafood products	189	66.1	0.32	64.6	0.29	58.2	0.16	64.0	0.28	79.4	0.59	88.4	0.46	91.0	0.65
Fruit and vegetables	936	72.9	0.40	89.1	0.78	66.9	0.25	65.6	0.22	83.4	0.65	69.0	0.28	67.1	0.23
Meat and meat products	668	76.2	0.53	80.8	0.62	66.0	0.32	67.5	0.35	87.4	0.75	82.2	0.61	82.5	0.62
Nonalcoholic beverages	759	82.4	0.50	89.2	0.38	68.5	0.10	61.3	0.20	54.7	0.19	75.6	0.24	57.8	0.12
Sauces, spreads, and seasonings	554	86.6	0.32	91.3	0.13	51.8	0.05	50.5	0.06	78.9	0.21	54.0	0.10	52.4	0.09
Snack foods	452	79.0	0.15	94.7	0.34	40.0	0.06	58.6	0.12	85.4	0.32	36.5	0.01	55.1	0.04
Special foods	85	92.9	0.47	92.9	*	25.9	*	34.1	*	75.3	0.10δ	30.6	0.03δ	38.8	0.04δ
Sugar, honey, and related products	188	63.30	*	79.26	*	63.83	*	63.83	*	70.74	*	84.6	*	83.51	*
		WHO-Euro NPM Chilean NPM		WHO-Euro NPM ADGs		ADGs Chilean NPM		ADGs HSR		ADGs Nutri-Score		Nutri-Score Chilean NPM		Nutri-Score HSR
Food category	n	% AG		%AG		% AG		% AG		%AG		% AG		%AG
Bread and bakery products	716	85.9	0.43	68.4	0.34	62.7	0.22	85.9	0.72	85.8	0.71	63.6	0.25	97.1	0.94
Cereal and cereal products	636	89.9	0.80	64.0	0.30	63.7	0.29	83.0	0.28	82.4	0.33	62.7	0.27	96.5	0.85
Confectionery	244	93.4	*	100	1.00	93.4	*	87.7	*	86.1	*	92.6	0.61	94.3	0.75
Convenience foods	789	62.6	0.32	59.6	0.23	64.0	0.29	70.9	0.29	70.3	0.31	68.6	0.39	93.4	0.79
Dairy	922	81.0	0.42	46.5	0.12	58.1	0.26	67.3	0.30	62.9	0.22	64.2	0.34	78.3	0.55
Edible oils and oil emulsions	163	77.9	0.55	85.9	0.71	77.3	0.54	87.7	0.75	66.3	0.37	71.8	0.45	69.9	0.44
Eggs	21	90.5	*	*	*	90.5	*	90.5	*	81.0	*	81.00	*	90.5	0.62
Fish and seafood products	189	83.1	0.52	81.5	0.30	77.3	0.37	79.4	0.12	78.8	0.23	80.4	0.45	94.2	0.74
Fruit and vegetables	936	85.4	0.68	74.4	0.42	79.2	0.42	87.8	0.51	86.8	0.44	79.8	0.36	95.3	0.73
Meat and meat products	668	87.4	0.73	82.2	0.57	80.4	0.56	74.6	0.43	75.5	0.46	77.1	0.52	97.6	0.95
Nonalcoholic beverages	759	47.6	0.09	75.2	0.23	67.6	0.39	57.2	*	71.0	0.41	82.6	0.66	78.8	0.57
Sauces, spreads, and seasonings	554	80.0	0.27	81.6	0.10	79.2	0.37	57.9	0.17	57.4	0.18	65.8	0.34	90.1	0.80
Snack foods	452	83.6	0.16	80.3	0.14	78.1	0.31	49.1	0.13	65.5	0.28	68.8	0.35	79.7	0.60
Special foods	85	77.7	0.19	85.9	*	68.2	*	21.2	*	29.4	*	58.8	0.28	91.8	0.81
Sugar, honey, and related products	188	88.3	*	84.0	*	87.2	0.47	83.5	0.38	85.6	0.47	86.7	0.45	95.7	0.84

¹ = Cohen κ coefficient (all values P < 0.05, unless marked with ^δ). κ coefficient interpretation using Landis and Koch (37): <0.0 = poor; 0.00–0.20 = slight; 0.21–0.40 = fair; 0.41–0.60 = moderate; 0.61–0.80 = substantial; 0.81–1.00 = almost perfect; * denotes valid κ coefficient could not be calculated. ADGs, Australian Dietary Guidelines; HSR, Health Star Rating; NPM, nutrient profile model; PAHO, Pan American Health Organization; WHO-Euro, World Health Organization European Division; %AG, percentage pairwise agreement.

Certain subcategories consistently had high agreement between all NCS pairings; for example: cakes, muffins and pastries; quinoa and other cereals; eggs; fruits; vegetables; chocolates and sweets; dessert additions; energy drinks; and fresh or frozen fish or seafood. However, the majority of subcategories had a high level of disagreement, and differing extents of disagreement between pairings. Subcategories with consistently low agreement between NCS pairings included: bread; breakfast cereals; meal kits; preprepared salads and sandwiches; ready meals; burgers; meat alternatives; milk; yogurt and yogurt drinks; and protein and diet bars.

The details of the qualitative analysis at the product-based level are presented in Supplemental File 3, and the summary of observations for each NCS are presented in Table 5. Table 5 summarizes key differences observed in the resulting classifications, and the corresponding specific technical explanation for these differences.

TABLE 5.

Key characteristic differences and the technical design elements that explain the differences for each nutrition classification scheme¹

Conceptual basis	NCS	Characteristics	Technical design explanation
Nutrient-based	Chilean NPM	• Closer agreement with NOVA compared with other nutrient-based schemes	• Thresholds only apply when “risk” nutrients are added, and these products are more likely to be ultraprocessed• Minimally processed foods with intrinsic fats and sugars are not assessed for “risk” nutrients
		• Ultraprocessed or discretionary products containing nonnutritive sweeteners can avoid regulation	• Processed versions of risk nutrients not penalized (e.g., nonnutritive sweeteners).
	HSR and Nutri-Score	• Recommended foods with intrinsic fats and sugars can score high ratings (e.g., ≥2.5 stars), and score relatively highly compared with other schemes• Basic recommended foods in the dairy and meat substitute categories score relatively highly compared with other schemes (e.g., milk, cheese, and tofu)	• Separate categories for milk, cheese, and oils, with lower thresholds for fats• Saturated fat content is balanced with protein content
		• Discretionary foods, ultraprocessed foods, and foods not permitted in other schemes can score relatively highly	• Level of processing is not considered• “Risk” nutrients are balanced against “positive” nutrients• Processed versions of beneficial components or risk nutrients are not considered, e.g., the addition of protein isolates and processed fibers, and the substitution of sugar with nonnutritive sweeteners
Nutrient- and food-based	WHO-Euro NPM	• Moderate agreement with NOVA compared with other nutrient-based schemes	• Classification is based on food category and delineation of categories is sometimes based on level of processing• Does not permit (for marketing) any products in certain categories considered discretionary/ultraprocessed (e.g., sweet biscuits, confectionery, cereal bars, energy drinks)• Permits (for marketing) all unprocessed fresh and frozen meat, fruit, and vegetables
		• Does not permit (for marketing) most ultraprocessed foods that other nutrient-based schemes score relatively highly (e.g., ultraprocessed meat substitutes)	• Only considers “risk” nutrients using a threshold approach (i.e., does not balance with positive components), and nutrients considered depend on the food category
		• Basic recommended dairy and meat substitutes (e.g., tofu and soy milk) often exceed fat thresholds• Some breakfast cereals containing nuts and seeds exceed the fat thresholds	• Does not distinguish intrinsic fats from added fats• Does not consider source of nutrients
		• Many basic recommended breads exceed sodium thresholds	• Sodium thresholds are relatively low for bread products
	PAHO NPM	• Almost all ultraprocessed and discretionary foods are identified as “unhealthy”	• Criteria combine level of processing and nutrient profiling
		• Able to identify more processed versions of recommended foods relative to other schemes (e.g., processed cheeses, sweetened yogurts, plant-based burgers.• Unprocessed, minimally processed foods, and processed culinary ingredients identified as “healthy”	• All foods classified as group 3 and group 4 in NOVA are assessed for content of “risk” nutrients• Foods classified as group 1 and group 2 in NOVA are not assessed for “risk” nutrients
		• Foods can exceed total fat thresholds when basic recommended ingredients are present, such as salmon and olive oil• Basic recommended meat and dairy substitutes (e.g., tofu, tempeh, and soy milk) can exceed total fat thresholds	• Does not distinguish intrinsic fats from added fats• Does not consider source of nutrients
		• Basic recommended breads and canned beans often identified as “unhealthy”	• Sodium thresholds relatively low and based on the energy ratio
Food-based	NOVA	• Identifies all ultraprocessed foods as “unhealthy”• All unprocessed and minimally processed foods are considered “healthy”	• Criteria assess level of processing only
		• Many ADG-recommended foods are identified as ultraprocessed (e.g., strawberry milk, cheddar cheese, vegetable soup)• Some foods considered discretionary or high in sugar or sodium are non-ultraprocessed (e.g., beef jerky, bacon, kombuchas, sauces, condiments, cakes, and slices)	• Classification of group 4 foods is based on the presence of ≥1 cosmetic additive or industrial substance in the ingredients list• Nutrient content and food type are not considered in the criteria
Diet-based	ADGs	• Many ultraprocessed foods are classified as “five food group” foods (e.g., margarine, coconut yogurt, breakfast cereal)	• Does not consider level of processing
		• Recommended foods identified as “healthy” regardless of processing or “risk” nutrients• Certain categories of foods are automatically identified as discretionary (e.g., confectionery, biscuits, cereal bars, and soft drinks)	• Foods classified based on their contribution to an overall healthy diet and nutritional adequacy

¹References to recommended and discretionary foods are based on the Australian Dietary Guidelines (31). ADGs, Australian Dietary Guidelines; HSR, Health Star Rating; NCS, nutrition classification scheme; NPM, nutrient profile model; PAHO, Pan American Health Organization; WHO-Euro, World Health Organization European Division.

Table 6 presents illustrative examples of the observations made in Table 5. Two products (original baked cheesecake and four bean mix) are examples of unanimous agreement between all the NCSs and can be clearly identified as healthy and unhealthy. Other examples show how the nutrient-only NCSs diverge from the other NCSs, allocating the ultraprocessed breakfast cereal and protein bar high ratings on their respective scales. Bread, cheese, and milk are examples of products classified as unhealthy by the stricter food-based schemes but classified as recommended FFG foods by the ADGs.

TABLE 6.

Examples of food and beverage products, their ingredients, and classification by each nutrition classification scheme¹

Product	Crunchy honey cereal	Mild chicken and rice curry	Raspberry cheesecake protein bar	Shredded cheddar cheese	High-protein full-cream milk	Salmon with Sunblush tomato dressing
Ingredients	Cereals (77%) [wholegrain wheat (68%), rice flour], raw sugar, honey (2%), coconut (1%), salt, barley malt extract, flavor, vitamins (niacin, vitamin E, thiamin, riboflavin, folate), mineral (iron)	Coconut milk, rice, chicken (13%), onion, chicken stock, carrot, pumpkin, zucchini, apple, raisins, garlic, cumin, turmeric, salt, pepper, preservative (202), citric acid	Protein blend (whey protein isolate, calcium caseinate, whey protein concentrate, milk protein concentrate, hydrolyzed collagen), white chocolate (18%) [sweetener (maltitol), cocoa solids (33%), soy protein isolate, milk solids (10%), emulsifiers (soy lecithin, 476), food acid (citric acid)], polydextrose, *15 remaining ingredients listed in footnote	Full-cream milk, salt, purified cellulose, culture, kosher nonanimal rennet	Filtered milk	Salmon (89%), dressing (11%) {tomato, red wine vinegar, water, green olives [green olives, water, salt, lactic acid], olive oil, tomato purée, Parmigiano reggiano cheese, Sunblush tomatade [Sunblush tomatoes (2%), tomato concentrate, sunflower oil, olive oil, salt, oregano, sugar, garlic, citric acid], lemon juice, fire-roasted tomatoes, garlic purée, basil, salt, black pepper}
ADGs	Five food group	Five food group	Discretionary	Five food group	Five food group	Five food group
NOVA	Ultra-processed	Ultra-processed	Ultra-processed	Ultra-processed	Minimally processed	Processed
PAHO NPM	Unhealthy—excess free sugars	Unhealthy—excess total fat and saturated fat	Unhealthy—excess saturated fat	Unhealthy—excess total fat and sodium	Healthy	Unhealthy—excess total fat, excess saturated fat, and excess sodium
WHO-Euro NPM	Unhealthy—excess sugars	Healthy	Unhealthy—energy bar category not permitted	Unhealthy—excess total fat	Unhealthy—excess total fat	Healthy
HSR	3.5 stars	4 stars	4.5 stars	4.5 stars	4.5 stars	4 stars
Nutri-Score	C rating	B rating	C rating	D rating	A rating	B rating
Chilean NPM	Unhealthy—high in sugars	Healthy	Healthy	Unhealthy—high in sodium	Healthy	Healthy
Product	Original baked cheesecake	Four bean mix	Stone-baked white batard	Classic sea salt grass-fed beef jerky	Organic apple kombucha	Fava beans supersnack
Ingredients	Neufchatel cheese (31%) [milk, cream, salt, vegetable gum (410), lactic acid, starter culture], cream, biscuit crumb {wheat flour [minerals (iron, zinc), vitamins (thiamin, riboflavin, folic acid)], sugar, vegetable oil, wheat bran, high-fructose syrup, salt, raising agents (341, sodium bicarbonate), emulsifier (soy lecithin), cinnamon}, **15 remaining ingredients listed in footnote	Chickpeas (15%), red kidney beans (15%), butter beans (15%), cannellini beans (15%), water, salt, antioxidant (ascorbic acid)	Wheat flour (thiamin, folic acid), water, iodized salt, soy flour, yeast	Beef, coconut tree sap, apple cider vinegar, sea salt, spices	Live kombucha culture (water, organic Tasmanian Gala apple juice, organic cane sugar, certified organic white tea)	Fava beans (91%), sunflower oil, lightly salted seasoning [maltodextrin (from corn), salt (0.05%), mineral salt (potassium chloride), natural flavor]
ADGs	Discretionary	Five food group	Five food group	Discretionary	Discretionary	Five food group
NOVA	Ultra-processed	Processed	Processed	Processed	Processed	Ultra-processed
PAHO NPM	Unhealthy	Healthy	Unhealthy—excess sodium	Unhealthy—excess free sugars and excess sodium	Unhealthy—excess free sugars	Unhealthy—excess transfat
WHO-Euro NPM	Unhealthy	Healthy	Healthy	Unhealthy—excess sodium	Unhealthy—excess free sugars	Unhealthy—excess sodium
HSR	1 star	5 stars	3.5 stars	0.5 star	1.5 stars	5 stars
Nutri-Score	D rating	A rating	B rating	D rating	D rating	A rating
Chilean criteria	Unhealthy	Healthy	Healthy	Unhealthy—high in sodium	Unhealthy—high in sugars	Healthy

¹ADGs, Australian Dietary Guidelines; HSR, Health Star Rating; NPM, nutrient profile model; PAHO, Pan American Health Organization; WHO-Euro, World Health Organization European Division.

^*Sweetener (maltitol), humectant (glycerol), sweetener (sorbitol), berry bites (4.5%), soy protein nuggets, sunflower oil, natural flavor, maltodextrin, raspberry powder (0.4%), emulsifier (soy lecithin), flavor enhancer (salt), natural color (carmine), acidity regulator (malic acid), antioxidant (mixed tocopherols), sweetener (steviol glycosides).

^**Sugar, egg, lemon juice (from concentrate), glucose syrup, mineral salts (339, 450, 516), vegetable gums (410, 401), thickener (1412), lactic acid, natural vanilla flavor, emulsifiers [vegetable emulsifier (477, 471, 475)].

Discussion

This study aimed to compare the agreement of nutrient-based, food-based, and dietary-based NCSs in their assessment of a food's health potential within the Australian food supply, and examine the conceptual underpinnings and technical characteristics that explain differences. Wide variation was observed in the classification of “healthiness” between schemes, ranging from 22.1% (PAHO NPM) to 63% (HSR) of the sample being classified as healthy. The PAHO NPM, NOVA, WHO-Euro NPM, and the Chilean NPM were relatively stricter, classifying over half the sample as unhealthy, compared with the ADGs, HSR, and Nutri-Score, which classified less than half of the sample as unhealthy. Wide variation between schemes was also observed for the pairwise comparison of food products, with agreement ranging from slight agreement (according to the Cohen κ statistic) for the PAHO NPM and the HSR (κ = 0.1698), to substantial agreement between the HSR and Nutri-Score (κ = 0.7809). Considering the differing conceptual underpinnings and technical aspects of each NCS, some variance was expected. However, the extent of variance is surprising considering that all NCSs attempt to provide a metric for healthiness.

Generally, NCSs informed by the same nutritional exposure, whether nutrients, foods, or dietary patterns, were closely aligned in their classifications, albeit with some exceptions. Two of the nutrient-only–based schemes, the HSR and Nutri-Score, resulted in the highest pairwise agreement, a somewhat unsurprising finding because they were both adapted from the FSA-UK NPM (38). The third nutrient-only–based scheme, the Chilean NPM, resulted in only fair and moderate agreement with the HSR and Nutri-Score, respectively. This is likely due to technical differences—the HSR and Nutri-Score use an algorithm approach (that balances beneficial and risk nutrients), and the Chilean NPM uses a single risk nutrient threshold approach. The Chilean NPM aligned most closely with the WHO-Euro NPM, which also uses a nutrient threshold approach, although the WHO-Euro NPM applies different nutrient thresholds depending on the food category.

Food-based schemes were generally aligned—the PAHO NPM had moderate agreement with both NOVA and the WHO-Euro NPM. However, NOVA and the WHO-Euro NPM resulted in only fair agreement. This finding is likely explained by their divergence in approach: NOVA does not profile nutrients, and the WHO-Euro NPM does not directly assess level of processing. The ADGs, the only dietary-based scheme included in the analysis, had moderate agreement with all NCSs, except those that included level of processing, where only fair agreement was observed. This is not surprising considering the ADGs do not incorporate level of processing, rather they consider nutritional adequacy and the overall dietary pattern based on nutrients and food types. The lowest agreement was seen between the food-based NCSs, PAHO NPM, NOVA, and WHO-Euro NPM, and the nutrient-based NCSs, HSR and Nutri-Score. This result is likely due to a number of differences; the HSR and Nutri-Score cannot distinguish level of processing like NOVA and PAHO, and include considerably fewer food categories compared with the WHO-Euro NPM.

Other studies comparing NCSs have not included the same selection of schemes; however, all 7 have been compared with ≥1 previously. Comparison with the WHO-Euro NPM found the HSR would permit more food products for marketing (with an HSR cutoff at 3.5 stars): 36% compared with 29%, respectively (37). Similar to our findings, the HSR classified a number of foods, such as high-sugar breakfast cereals, fruit juices, and ready meals, as healthy. Broad alignment was previously found between the HSR and the Chilean NPM, with only 8% of products misaligned, compared with 33% in this study, although the threshold for healthy was considered to be ≥3.5 stars, and for unhealthy <2 stars (39). Classifications of Nutri-Score have been compared with NOVA in Spain, with 26% of food products receiving an A rating being classified as ultraprocessed (40). Two studies have compared the HSR, NOVA, and the ADGs in Australia (19, 41), and similar to the current study, NOVA classified more products as unhealthy compared with the ADGs, although comparisons with the HSR were again mixed due to differences in cutoffs (19, 41). When applied to the Mexican, Colombian, and Brazilian food supplies, PAHO NPM classified more food products as unhealthy compared with the Chilean NPM (42, 43). The PAHO NPM also classified the highest proportion of food products as unhealthy in a sample of the Canadian food supply, at 84% compared with 70% for the WHO-Euro NPM and 64% for Nutri-Score (44).

In addition to the conceptual underpinning of each NCS, the differences observed in the resulting classifications can also be attributed to key technical characteristics. These characteristics can be divided into the contrasting broader approaches to classification and the technical differences in assessment of nutrients.

The broader approaches to classification identified are:

1. category-specific criteria, compared with applying the same criteria to all food types;

2. the identification of food categories as always “healthy” or “unhealthy,” as opposed to categories requiring further nutrient assessment, and;

3. the consideration of level of food processing.

The technical differences in the assessment of nutrients identified are:

1. the assessment of nutrients only when added, compared with assessing nutrients both added and intrinsic;

2. the balancing of risk nutrients and beneficial components using an algorithm approach, compared with assessment of risk nutrients using a threshold approach;

3. differences in the upper thresholds used for each nutrient;

4. the choice of nutrients included: assessment of saturated fat or total fat; assessment of free sugars, added sugars, or total sugars; inclusion of energy content, protein, or fiber, and;

5. the inclusion of specific ingredients in the criteria to prevent substitution effects, e.g., nonnutritive sweeteners or protein isolates.

The WHO-Euro NPM, HSR, and Nutri-Score all apply category-specific nutrient criteria. The WHO-Euro NPM assigns the highest number of categories, allowing the criteria to be highly specific for each food type. For example, the “savory snacks” category, which includes nuts and seeds, is only assessed for free sugars and salt, therefore the fats intrinsic to nuts and seeds are not assessed. The upper thresholds also differ based on the food category. For example, there is a greater allowance for saturated fat content in cheese at 20 g/100 g,  compared with ready meals at 4 g/100 g, accounting for the intrinsic fats and the overall health benefits associated with cheese. However, the number of categories and specific inclusion criteria in the WHO-Euro NPM can make the classification process more difficult and time consuming. In comparison, the HSR and Nutri-Score have fewer categories, mainly delineating beverages, dairy, and oils from all other types of foods. Similar to the WHO-Euro NPM this is a positive characteristic, because the intrinsic fats in basic dairy products do not substantially decrease the resulting ratings.

All schemes identify certain food categories that are automatically classified as healthy or unhealthy. The findings indicate this is generally a positive characteristic of NCSs. For example, the WHO-Euro NPM automatically permits unprocessed fruits and vegetables, and automatically does not permit categories such as energy drinks and confectionery. In contrast, the HSR automatically allocates minimally processed fruit and vegetables and water a 5-star rating, but allows all other products to undergo nutrient assessment, often resulting in misleading ratings for ultraprocessed and discretionary products. In part this is also a result of neither scheme assessing level of processing. NCSs that assess level of processing classified more foods as unhealthy, which NOVA and the PAHO do directly, the WHO-Euro NPM does for some categories, and the Chilean NPM does indirectly by only assessing added salt, sugar, and fats, because this feature correlates with processing.

Findings show the Nutri-Score and HSR score products high in added sugar or salt relatively highly compared with other nutrient-based NCSs, likely due to the algorithm approach that balances risk nutrients with beneficial nutrients. This balancing allows manufacturers to manipulate the composition of foods to gain a more favorable rating, a characteristic that might not be solved by simply increasing the penalty for risk nutrients. In contrast, PAHO NPM, WHO NPM, and the Chilean NPM all use a threshold approach for risk nutrients only, and therefore classified more products as unhealthy. However, even among threshold schemes there were marked differences in the nutrients included and the upper thresholds used. For example, the Chilean NPM does not penalize fats to the same degree as the PAHO NPM and WHO-Euro NPM, only evaluating added saturated fats, not total fats. Conversely, the Chilean NPM does not account for substitution effects with nonnutritive sweeteners, an element included in both the PAHO NPM and WHO-Euro NPM. The PAHO NPM has lower thresholds for risk nutrients compared with all the other schemes and uses an energy density approach (per 100 kcal) rather than a volume approach (per 100 g/mL),  meaning foods low in energy can easily be classified as being excessive in risk nutrients. Combined with the other factors, this resulted in the PAHO NPM being the strictest of the 7 NCSs evaluated.

The binary version of the ADGs used in this study is not strictly a dietary-based scheme, because it encompasses only the balance dimension of dietary patterns, and not quantity or variety. In contrast to the other NCSs, the ADGs evaluate foods based on their inclusion in a healthy diet and nutritional adequacy. However, when this is reduced to the food-based level the ADGs lack the nuance required to separate unhealthier versions of FFG foods, particularly when these foods are ultraprocessed or high in added sugar. On the other end of the spectrum the reductionist nature of nutrient-only–based schemes fails to adequately evaluate all elements that synergistically contribute to a food's health potential.

The aim of this study was not to say irrefutably which type of NCS was the best objective measure of the health potential of a food; instead the aim was to assess agreement and explore differences in performance. Nevertheless, a food-based approach to nutrition classification, such as NOVA, is logically aligned with nutrition classification at the individual food level, and therefore could reasonably form the basis for evaluating a food's health potential. NOVA includes 2 of the positive broader approaches to classification identified: consideration of level of processing, and the classification of certain types of foods as always healthy or unhealthy. Therefore, NOVA could be the benchmark against which the level of agreement among NCS pairings is measured.

However, NOVA could need some small technical adjustments to strengthen its applicability to policy actions targeting individual foods. For example, foods high in added sugar or salt, such as sweetened beverages or beef jerky, classified as unhealthy by other NCSs, can be classified as healthy by NOVA (Table 5). Furthermore, some foods classified as healthy by the ADGs, such as cheese, yogurts, and breads, can be classified as ultraprocessed by NOVA. However, these recommended healthy foods could potentially be reformulated by removing the cosmetic additives and industrial ingredients (markers of ultraprocessing) that transform the textural and sensory properties of foods, and this could be encouraged with regulation. This has been described as “wholefood reformulation,” whereby food innovations result in a range of new minimally processed (NOVA group 3) products (45).

The limitations of current schemes have led to the development of novel NCSs combining or expanding established approaches. The Spanish smart phone app El Coco, developed as a counter to Nutri-Score, combines aspects of the WHO-Euro NP, NOVA, and the Chilean NPM (46). However, no description or evaluation of the El Coco method has been published to date. The Siga scheme, also developed for a smart phone app, expands the NOVA scheme into 8 subgroups, adding technical criteria on risk nutrients and “markers of ultra-processing” (47). Currently this system remains propriety limited, leaving its potential use in policy actions uncertain. The more complex “Food Compass” assesses multiple characteristics across “health-relevant” domains, including nutrient ratios, level of processing (according to NOVA), and phytochemicals, resulting in a score from 1 to 100 (48). Yet the increased complexity and broader assessment of characteristics does not always result in logical classifications. For example, minimally processed foods such as boiled eggs and whole milk receive a score of 51 and 49, respectively. Unprocessed fruits and vegetables can also be ranked and compared using the scoring system, which is not a useful characteristic considering all fruits and vegetables should be encouraged.

The Food Compass example suggests that it is not sufficient to only incorporate the concept of level of processing in an NCS, but that an NCS should instead have this concept as the foundation of its design. Fardet and Rock (49) argue that a top-down holistic approach, starting from the food structure and then assessing individual components, is required for nutrition recommendations. A top-down “hybrid” approach to food classification could be informed by the results of this study. For example, the food-based NOVA could be strengthened by overlaying the strongest elements of an NCS identified: having categories that are always healthy and unhealthy regardless of nutrient content; appropriate thresholds for risk nutrients that do not penalize intrinsic nutrients in whole foods; and no allowance for the substitution of ingredients.

Some stakeholder groups are advocating for the Nutri-Score to be the recommended FOPL in the European Union (50). This study indicates Nutri-Score and the HSR perform poorly compared with other schemes, allocating a larger proportion of ultraprocessed and discretionary foods a “pass” mark on their respective scales, and do not represent suitable proxies for the more logical food-based approach. Furthermore, the positively skewed visual application of Nutri-Score and the HSR allow many ultraprocessed foods to receive a “health halo” in the form of high ratings. Indeed, an experimental purchasing study found the HSR encouraged purchasing of foods with positive nutritional attributes but did not perform as well at discouraging foods with nutrients of public health concern (51). Arguably, policy actions targeting diet-related noncommunicable diseases should, as a key objective, discourage the purchase and consumption of unhealthy foods (52), a requirement that Nutri-Score and the HSR fail to meet >50% of the time (when unhealthy is defined as ultraprocessed). The setting of international FOPL guidelines will potentially legitimize the nutrition classification approach for the recommended FOPL, influencing the development of future nutrition policy actions. Therefore, the findings of this study are relevant for informing these discussions.

Considering that global food supplies are becoming increasingly homogenized and physiological responses to unhealthy foods and dietary patterns do not generally differ between populations, a single globalized approach to healthiness classification could be beneficial. However, any plan to harmonize NCSs first needs to be informed by a rigorous evaluation of the potential for any unintended consequences to occur from the scheme's application to policy. Furthermore, certain criteria of any NCS might also need to be modified based on the national context, because public health priorities and food supply composition can differ between nations.

To the best of our knowledge, this is the first study to apply and compare a large and diverse number of NCSs to the Australian food supply, using a novel approach combining a national food composition database and a new product database. The food and beverage types included cover the range of possible products both packaged and unpackaged that are available in the marketplace and potentially subject to regulation. Although results are based on an analysis of the Australian food supply, the NCSs are universal/globally available and the observed differences in classification by each NCS would be applicable across a diversity of countries.

Nevertheless, a limitation of this study is that the dataset does not factor in the market share of the food products nor the usual purchasing and consumption habits of the Australian population, although these are factors that would have little effect on the findings, and it was more appropriate for this study to have a wide cross-section of food products to test the various NCSs. The presence of some nutrients and ingredients had to be estimated when not available in the dataset, including fruit and vegetable content, fiber, free sugars, and whether risk nutrients had been added (for the Chilean criteria in AUSNUT). However, robust and logical methods for estimation have been developed and are detailed in the Methods. Furthermore, the method to classify the binary ADGs NCS, devised by the ABS, lacks the detail necessary to accurately classify the range of novel packaged food products in the marketplace, and further steps needed to be applied to enable more accurate classification. A different choice of healthy/unhealthy cutoff points for the HSR and Nutri-Score would have resulted in differing levels of agreement with other schemes. However, the cutoff points chosen were logical, and differing cutoffs would not have changed findings on the technical characteristics of these schemes.

The AUSNUT 2011–2013 food composition database contains approximations for the nutrient content of foods, and although nutrient profiles were derived from mostly analytical data, some data were borrowed from overseas food composition tables, taken from food labels, estimated from similar foods, or calculated using a recipe approach. Thus, the nutrient composition of AUSNUT items will not be 100% accurate. However, any variability will not affect the findings relating to the comparisons of NCSs.

There are inherent limitations and challenges in attempting to assign individual foods to classifications of healthy and unhealthy. It is acknowledged that the health potential of a food depends on the context of an individual's overall dietary patterns and health circumstances; however, certain foods are clearly unhealthy and provide no nutritional value to diets. For the purposes of certain nutrition policy actions an unambiguous method of identifying foods to be discouraged is required. It is critical that any proposed NCS be rigorously evaluated to ensure effective policy responses to the most problematic foods in the food supply.

In conclusion, a wide variation in agreement was observed when rating the health potential of individual foods among the NCSs analyzed in this study. This level of disagreement is explained by differences in conceptual and technical characteristics among the NCSs. Conceptually, dietary-based and nutrient-only–based NCSs cannot logically assess foods at the individual food-based level. Technical characteristics that accounted for differences would complement a food-based approach; these included: the inclusion of categories that are always healthy and unhealthy; thresholds for risk nutrients that do not penalize whole foods; and no allowance for the substitution of ingredients. With current efforts toward harmonization of nutrition policy actions internationally, it is important to determine the most effective approach to classifying a food's health potential to successfully reduce the burden of diet-related disease.

Notes

This research is supported by an Australian Government Research Training Program (RTP) Scholarship, and an Australian Research Council Discovery Project (DP190101323: “Reforming evidence synthesis and translation for food and nutrition policy”).

The authors declare no external funding for the preparation of this manuscript.

Author disclosures: The authors report no conflicts of interest.

Supplemental Files 1–3 and Supplemental Tables 1 and 2 are available from the “Supplementary data” link in the online posting of the article and from the same link in the online table of contents at https://academic.oup.com/cdn/.

Abbreviations used: ABS, Australian Bureau of Statistics; ADGs, Australian Dietary Guidelines; AUSNUT, Australian food composition database; FFG, “five food group” food; FOPL, front-of-pack labeling; FSA-UK, Food Standards Agency United Kingdom; FVNL, fruit, vegetable, nut, and legume; HSR, Health Star Rating; NCS, nutrition classification scheme; NPM, nutrient profiling model; PAHO, Pan American Health Organization; UPF, ultraprocessed food; WHO-Euro, World Health Organization European Region.

Data Availability

Restrictions apply to the availability of the data described in this manuscript, which were used under license from Mintel, and so are not publicly available. Data are, however, available from the authors upon reasonable request and with permission of Mintel. The analytic code will be made available upon request pending application and approval.