Using artificial intelligence and computer vision to detect and monitor unhealthy child-directed food and beverage marketing: a data-driven study

Summary

Background

Childhood obesity and diet-related noncommunicable diseases are urgent global public health challenges, driven in part by children's continued exposure to persuasive marketing of unhealthy foods and beverages. Traditional methods that manually evaluate child-targeted food marketing practices are resource-intensive and inefficient. This study aims to address these limitations by leveraging artificial intelligence (AI), specifically computer vision and deep learning, to identify child-targeted food products and classify marketing elements on food packaging using a comprehensive database labeled with child-directed marketing features. Additionally, this study explores the practical application of AI within a specific food category to examine the relationships between child-directed marketing features, nutrition quality, and price, to estimate the potential impact of proposed Canadian marketing-to-children regulations.

Methods

Image classification algorithms were trained on 8283 manually labeled food package images, annotated and validated between 2021 and 2024 using a validated child-appealing packaging (CAP) coding tool. This labeled dataset served as the ground truth for model training and evaluation. Three machine learning algorithms, k-nearest neighbors (kNN), support vector machines (SVM), and convolutional neural networks (CNN) were used to classify food package images targeted at children. In addition, latest image object detection model, YOLOv12 (released February 2025), were fine-tuned to identify specific child-targeted marketing features on food and beverage packaging. Model performance was evaluated using accuracy, precision, recall, F1 score, AUC, and confusion matrix. We applied this AI strategy to breakfast cereals (n = 1765) to assess the proportion of food products displaying child-directed marketing features and that would be restricted under Canadian proposed marketing-to-children regulations, and conducted regression analysis to investigate the relationship between food marketing features and food price.

Findings

A total of 22 distinct child-directed marketing techniques featured on food and beverage packages were identified and annotated. The CNN-based image classification model outperformed kNN and SVM, achieving 0.90 accuracy and 0.96 AUC in identifying food marketing targeted at children against manually coded labels. Fine-tuned YOLOv12 object detection model demonstrated varying performance levels across child-targeted marketing features, reflecting the complexity and diversity of marketing strategies on food packaging. By applying this AI strategy, 39.2% of breakfast cereals were found to display child-directed marketing features, of which 89.5% would be considered as unhealthy and restricted under Canadian proposed marketing-to-children regulations. Products with child-directed marketing features showed a negative, but not statistically significant, association with price (coefficient = −0.25, p = 0.072).

Interpretation

This study introduces the first AI-driven approach to effectively identify and categorize child-directed marketing on food and beverage packaging. These methods offer a scalable and efficient AI-driven solution for monitoring compliance with marketing-to-children policies and evaluating associations with nutrition quality and price, and can be extended to other data sources such as social media and online video content for broader applications. This study supports evidence-based policy development to protect children from unhealthy food marketing practices and provides critical insights into its potential impacts on children's health outcomes.

Funding

This work was funded by the Data Sciences Institute catalyst grants, Health Canada research contract on M2K research, and the CIHR project grants.

Research in context.

Evidence before this study

We conducted a comprehensive review using PubMed to identify studies published up to Nov 30, 2024, using the following terms: (“child∗” OR “youth” OR “adolescen∗”) AND (“food marketing” OR “advertising” OR “promotion”) AND (“regulation” OR “policy” OR “compliance” OR “evaluation” OR “monitor”) AND (“nutrition” OR “obesity” OR “diet∗” OR “health∗”) AND (“AI” OR “artificial intelligence” OR “ML” or “machine learning” OR “computer vision”). We found no other relevant papers reported on the performance of AI and advanced computer vision model for detecting and monitoring child-directed marketing in food and beverage package. Existing methods for assessing child-directed marketing features rely heavily on manual coding, which is time-consuming, costly, and infeasible for large-scale and dynamic evaluations.

Added value of this study

Using a comprehensive annotated food composition and image database, we fine-tuned AI models to automate the detection of marketing features with a high accuracy of 0.90. These features were linked to nutritional quality, price, and proposed Canada child-directed marketing regulations. As a practical demonstration, we identified that 39.2% of breakfast cereals contained child-directed marketing, of which 89.5% would be considered as unhealthy and restricted under proposed Canada marketing-to-children regulations.

Implications of all the available evidence

Our study offers a scalable and efficient AI-enhanced solution for monitoring child-directed food marketing practices, significantly reducing reliance on manual methods. Our findings highlight the transformative potential of AI-driven methodologies for effectively monitoring child-directed marketing in food packages, provide scalable, cost-effective, and efficient strategies to assess compliance with marketing-to-children policies and evaluate their associations with nutritional quality and price.

Introduction

Childhood obesity and diet-related noncommunicable diseases are of urgent global public health concern.¹ On-going exposure to persuasive marketing of unhealthy foods and beverages influences children's dietary habits and negatively impacts children's health.1, 2, 3, 4, 5, 6, 7 Because the majority of foods and beverages marketed to children are highly processed, energy-dense, and nutrient-poor,8, 9, 10, 11 child-directed marketing on food and beverage is problematic.^9,12 Prevention-focused policy actions, such as restricting the marketing of unhealthy foods and beverages to children, have been recommended to provide a better food environment for children.^1,13 Several countries have already implemented voluntary or mandatory policies to restrict unhealthy child-directed marketing, however, many are not comprehensive and further action is needed.^14,15 For example, in Canada, mandatory restrictions were implemented as early as 1980 in the province of Quebec and an industry led voluntary Children's Food and Beverage Advertising Initiative (CAI) among participating companies was updated in 2017 to restrict unhealthy food marketing to children in the rest of the country.¹⁶ Recently, the Canadian federal government proposed mandatory national-level restrictions (Bill C-252) and an updated policy was released in 2023 on restricting food and beverage advertising directed at children, focusing on digital marketing to children,^10,17 alongside voluntary industry standards to ensure responsible marketing practices targeting children.¹⁶ In the United States, a voluntary self-regulatory program, the Children's Food and Beverage Advertising Initiative (CFBAI), was implemented by industry in 2007 to improve children's food advertising.¹⁸ The UK government implemented a 9pm TV watershed and restricted advertising for less healthy food and drink in 2022 to reduce childhood obesity.¹⁹ Chile also implemented a comprehensive marketing policy in 2016 to restrict child-directed marketing of foods high in energy, total sugars, sodium or saturated fat.^20,21 Most recently, the World Health Organization (WHO) released new 2023 guidelines to shape policies to protect children from the harmful impacts of food marketing. The WHO recommended that countries should introduce mandatory policies to regulate such marketing.¹² Despite these efforts, there is a timely need for data-driven evidence to inform the development of mandatory policies, as well as advanced methodologies to efficiently monitor food marketing for the purposes of policy evaluation.

Marketing on food and beverage packaging is a key source of children's exposure to unhealthy food marketing, characterized by targeted strategies such as the display of visual appeals (e.g., fonts, colors, shapes), characters (e.g., spokes-characters, licensed characters, cartoons), activities (e.g., games, toys, give-ways), and other elements designed to attract children.^10,22 Previous literature found that food marketing affects children's attitudes, preferences and consumption behaviors.³ However, measuring child-directed marketing is time and labor-intensive, requires extensive training and validation, and is often subjective. Specifically, to classify child-directed food products and identify the presence and type of various marketing features on food packages and advertising is currently being done manually.^21,23,24 However, this manual approach is increasingly unfeasible and costly as the size of food databases, such as the Food Label Information Program (FLIP) in Canada, expands to ∼200,000 products per collection. Most importantly, the maintenance of such a database that classifies products with child-directed marketing features, links them to the levels of nutrients of public health concern, and to the nutrient profile model used for restricting food marketing to children, is essential for monitoring the food environment. Therefore, there is an urgent need for improved methodologies to a) efficiently and accurately analyze child-directed marketing, b) provide critical and timely evidence of the current marketing to children landscape, demonstrates the need for such policies, and c) to assess compliance with national regulations and WHO policies and recommendations.

In recent years, the application of artificial intelligence (AI), particularly deep learning and computer vision technologies, has emerged as a promising solution to address the challenges of monitoring and regulating unhealthy food marketing to children.²⁵ Currently, machine learning and deep learning models are being extensively used in the diverse domain of computer vision, such as image classification, video processing, object detection and segmentation.²⁶ Traditional algorithms (such as HOG, SIFT, and VJ detector) extract features from images for classification tasks.27, 28, 29 Convolutional neural network (CNN) provides a more robust architecture, enabling more expressive and accurate feature representations.³⁰ Furthermore, advanced deep learning-based methods (such as R-CNN, Faster R-CNN, Mask RCNN, SSD, and YOLO) that learn feature representations directly from data have significantly improved image detection performance.31, 32, 33, 34, 35 These AI-driven image recognition tools can be employed to automatically identify and classify marketing strategies on food packaging that target children, such as the use of cartoons, vibrant colors, or promotional characters. Despite the potential of these advanced AI technologies, there is no study that has investigates the application of AI for analyzing and monitoring child-targeted food marketing on food and beverage packaging.

This research aims to leverage state-of-the-art image classification and object detection algorithms to accurately identify child-targeted food products and classify marketing elements targeted at children using a comprehensive, labeled database of child-directed marketing features on food and beverage packaging in Canada. Additionally, this study explores the practical application of these AI methods within a specific food category to examine the relationships between child-directed marketing features, nutrition quality, and price, to estimate the potential impact of potential marketing-to-children regulations. Our study advances existing methodologies for monitoring child-directed marketing practices, provides a valuable tool to guide the development, implementation, and evaluation of national and global policies aimed at protecting children from harmful industry practices.

Methods

Food Label Information and Price Database (FLIP)

This study uses the University of Toronto Food Label Information and Price (FLIP) database, which contains label information for a large cross section of foods and beverages available from Canadian grocery retailers. FLIP was first developed in 2010 and has been updated every 3–4 years with subsequent collections in 2013, 2017, 2020, and 2023.36, 37, 38, 39 Information captured in FLIP includes the product name, brand, Nutrition Facts table (NFt), ingredients list, retailer of collection, price and package photos. Data collection prior to 2020 was done in-person at major Canadian grocery retailers by trained research staff either manually (2010) or using the FLIP data collection smartphone application (2013 and 2017) and data processing was performed manually by researchers.36, 37, 38, 39 For the 2020 collection, due to the COVID-19 pandemic, it was neither feasible nor safe to collect data in-person so automated methods, web-scraping and OCR, were developed to collect data for all foods and beverages available on the websites of top grocery retailers in Canada. Data processing for a comprehensive food database can be expensive and labor-intensive and web-scraping increased the number of retailors and foods and beverages collected substantially (n = 74,445 in 2020) from previous manual collections (n = 10,487 in 2010, n = 15,342 in 2013, and n = 19,721 in 2017). As a result, state-of-the-art artificial intelligence (AI) and machine learning (ML) methods were developed using the 2020 data to automate data processing steps including food categorization,⁴⁰ NOVA classification,⁴¹ and application of nutrient profiling models.⁴⁰ The 2023 collection was automated similar to the 2020 collection, but with an expanded number of retailers, and greater regional variation was added, which greatly increased the number of foods and beverages collected (n > 156,000).

This study used products in the 2017 and 2020 collections of the FLIP database. Products in FLIP2017 and FLIP2020 were categorized into the Table of Reference Amounts (TRA) major food categories (e.g., bakery products) and subcategories (e.g., cookies, toaster pastries).⁴² A total of 8283 products were selected and annotated in this study, focusing on prioritized child-relevant TRA subcategories previously identified in the FLIP database as containing the most child-appealing marketing.^11,23,43 While this sample does not represent the full Canadian food supply, it is representative of the product categories most relevant to children's marketing exposure.

Manually label and annotate child-directed marketing features in images

Manual labeling

The validated child-appealing packaging (CAP) coding tool was used to identify child-appealing marketing techniques^23,44 on the front of food packages in the 2017 and 2020 collections of the FLIP database. The CAP coding tool identifies 12 core marketing techniques, which independently make a package appealing to children, and 10 broad marketing techniques, which are not child-appealing on their own, but increase the persuasiveness of the marketed message.^23,44 Examples of core marketing techniques include games or activities or the presence of celebrities on food packaging. Examples of broad marketing techniques are interesting fonts or lettering, and health or nutrition claims on the package. Each marketing technique in the CAP coding tool was manually identified for all products in the FLIP2017 dataset. A researcher marked the presence of each technique as either Yes (present) or No (not present). A second researcher independently validated these assessments to ensure accuracy. Both researchers were subject matter experts on marketing to children and developed the CAP coding tool. Specifically, they independently coded a random 5% of the sample, achieving 93.2% raw agreement on child-appealing classifications. After coding the full dataset, the second researcher reviewed all uncertain cases and a random 2.5% of the sample, yielding 99.6% raw agreement and a Cohen's Kappa of 0.98, indicating near-perfect reliability. Additional information on the process of identifying and validating child-appealing marketing in FLIP2017 are also available in our earlier study.²³ Building on this work, additional annotations of marketing techniques were conducted to train AI models.

Manual features annotation

Researchers were trained on the marketing techniques in the CAP coding tool using published definitions of each technique and examples of each technique on product packages from the FLIP2017 database (where techniques were previously manually identified and validated).²³ Images of the front of package for foods were uploaded to LabelStudio, an online, open-source platform for labeling data, including images. For each marketing technique a unique “tag” was created and added to Label Studio. A trained researcher then evaluated each image (one researcher per image) and annotated it by drawing bounding boxes around the identified marketing techniques, assigning the appropriate tags to determine their precise location within the image. All annotations were validated independently by a second student researcher, who did not do the original annotation. Any disagreements were discussed and resolved with consensus between researchers. Compared to the manual identification of marketing techniques used on the FLIP2017 data, annotating in LabelStudio enabled the precise determination of the location and type of marketing techniques on an image.

Ethics

This study involved secondary analysis of food package images and did not involve human participants, identifiable private information. As such, ethical approval was not required. No informed consent was necessary as the research did not involve human participants or their data.

Image processing

All images were uniformly resized to a width of 1024 pixels and a height of 1024 to standardize the input across the dataset. We used data augmentation techniques such as random cropping, flipping, and color jitter to enhance the robustness of model. The dataset was then randomly split into 70% for training, 10% for development, and 20% for testing. We extracted and normalized image features using the Histogram of Oriented Gradients (HOG) method, and added additional dominant color features to capture more intricate details from the images. Each image in the training set was manually labeled as either not child-directed (“0”) or child-directed (“1”) (Fig. 1A). In addition, we manually annotated each image with one or more pre-determined child-directed marketing features, and marked the location of each featured object using rectangular boundaries to indicate its position on the food packaging image (Fig. 1B).

Fig. 1.

Examples of front-of-package images for a mock product illustrating (A) the present and absent of child-direct marketing classification, and (B) the identification and annotation of child-directed marketing features.

The manually annotated and processed images were used to develop machine learning and deep learning models for image classification and image object detection, specifically targeting child-directed marketing features. Specifically, this study explored the use of k-nearest neighbors (kNN), support vector machines (SVM), and convolutional neural networks (CNN) with a flattening layer and a global average pooling 2D layer for image classification. Additionally, we fine-tuned a pre-trained YOLOv12 object detection model to divide the images into regions and predict bounding boxes and probabilities for each category of child-directed marketing features. The list of explanations of technical terms in this paper is shown in Supplementary Table S2.

Image classification algorithm

K-nearest neighbors (kNN)

kNN is a non-parametric classifier to classify images based on their feature vectors. The kNN model calculates the distance between the feature vector of an input image and the training images, assigning the input image the class of its k-nearest neighbors, where k is an integer parameter. To implement the kNN model, we used the KNeighborsClassifier from the sklearn library.⁴⁵ The kNN model was initialized with the hyperparameter k, fitted to the training set, and subsequently used to make predictions on the validation set.⁴⁶ Classification is based on the k-nearest neighbors, meaning k many data points who are closest to the input image in terms of the default sklearn metric (Minkowski distance).⁴⁷ Our kNN classifier compared the Minkowski distance between the input image and each of the training images. We fine-tuned our model by adjusting the k parameter to identify the optimal number of closest neighbors.

Support vector machines (SVM)

SVM is a supervised machine learning algorithm that identifies the optimal hyperplane that separates different classes in the feature space, which is a well-known method in image classification.^46,48 SVM separates a set of training images into two different classes and builds optimal separating hyperplanes based on a kernel function. We used extracted image features and labels to find the optimal hyperplane that separates the images of different classes with maximum margin. A polynomial kernel function was used to handle data that are not linearly separable.

Convolutional neural network (CNN)

Unlike kNN and SVM that do not have a learning phase, CNN is a type of a feed-forward neural network that autonomously learns spatial features and patterns from its convolutional layers.⁴⁹ The CNN model was constructed for binary classification.^30,50 We implemented CNN with ResNet18, an architecture that uses a residual learning framework consisting of basic blocks and bottleneck blocks. ResNet18 consists of 18 layers, including convolutional layers, batch normalization, binary cross entropy loss function, and a Rectified Linear Unit (ReLU) activation function. This framework involves introducing skip connections or shortcut connections that bypass one or more layers, which significantly improves the training of deep neural networks and enabled better performance. ResNet18 is good at extracting features from input images and classifying them into predefined categories.^51,52

Image object detection algorithm

You only look once (YOLOv12)

A state-of-the-art object detection algorithm released on February 2025 that operates on attention-centric YOLO framework⁵³ was used to identify and classify child-directed marketing features on the packaging of food and beverage products. This includes detecting one or more core child-directed marketing elements (e.g., branded characters, unconventional shape and color, child-appealing activities, etc.) and other features commonly used to attract children's attention. YOLOv12 processes the entire image and performs object detection in a single pass through the neural network with a compelling balance of speed and accuracy, making it particularly suitable for real-time applications. Using a regression-based approach, YOLOv12 simultaneously predicts class probabilities and bounding box coordinates for multiple detected objects within an image. The model outputs a set of bounding boxes for the detected objects to represent the regions of detected objects, and a probability of the most likely class for each bounding box. We used manually annotated images with defined categories of child-directed marketing features (features with n > 30 annotated examples) to train the model. We fine-tuned the YOLOv12 model by adjusting the learning rate and other parameters to optimize its performance for our specific child-directed marketing image object detection task.

Assessing nutritional quality, food price, and compliance with Canada's proposed child-directed advertising restrictions

To assess the nutritional quality of each product, we used Health Canada's proposed nutrient profile model (HC M2K, healthy or less healthy) from most recent policy update on restricting food advertising primarily directed at children to assess whether or not each product would be permitted to have child-directed marketing features.^17,23 Products that did not have a Nutrient Fact table, TRA reference amount, serving size, or with implausible nutrient information were excluded. In the HC M2K, the thresholds for sugar, sodium and saturated fat are 5% of DV (5 g), 6% of DV (140 mg) and 10% of DV (2 g) per serving respectively.¹⁷ If a product exceeded any one of these thresholds, it was classified as being restricted from carrying child-appealing marking (less healthy). If not, the product is permitted to carry child-appealing marketing (healthy). To assess whether there are potential price differences based on marketing restrictions and display of marketing to kids features, the price per 100 g or mL were calculated. The image features extracted from the food and beverage package images using algorithms were linked to the FLIP food composition database with nutrient and price information by unique product IDs. This linkage enabled further analysis of the relationship between marketing features, nutrition quality relate to the marketing to children threshold, and price.

Performance evaluation and statistics

Model performance was evaluated using accuracy, recall, precision, the F1 score, receiver operating characteristic (ROC) and the area under the ROC curve (AUC). Performance metrics were calculated using true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN), and using tf.keras.metrics.AUC. Specifically, TP refers to the number of images our constructed classifier model correctly identified as being child-directed marketing. FP refers to the number of images our model incorrectly identified as being child-directed marketing compared to our manually coded images. TN refers to the number of images our model correctly identified as not being child-directed marketing. FN refers to the number of images our model incorrectly identified as not being child-directed marketing. Accuracy is the proportion of correctly predicted instances (TP + TN) out of the total number of instances in test set. Precision evaluates the proportion of TP predictions out of TP + FP. Recall is the proportion of TP instances out of TP + FN. F1 is harmonic mean of the precision and recall. In this study, the kNN, SVM, and CNN models were evaluated on accuracy, precision, recall, and F1 score, and AUC. Additionally, we conducted DeLong tests to statistically compare the AUCs between models.⁵⁴

The proportion of products with or without child-directed marketing features that comply with Health Canada's proposed child-directed advertising restrictions using HC M2K thresholds were calculated. An ordinary least squares regression model examined the relationship between product price per 100 g or mL and whether the product would be permitted for child-directed marketing, as well as whether displays AI-predicted child-directed marketing features, controlled for store, container size, and brand type.⁵⁵ All analysis were conducted in Python version 3.8.10.

Role of funding source

The funders had no role in study design, data collection, analyses of the data, interpretation, writing of the manuscript or the decision to submit.

Results

Development of an annotated child-directed food and beverage package marketing images database

A total of 8283 food and beverage products with packaging images were annotated, identifying 22 features related to child-directed marketing on food and beverage packaging. These included two general food package features, 12 child-directed marketing core features, and 10 broad marketing features, all of which were annotated for each product's front-of-pack image (Supplementary Table S1).

Among these features, we annotated and segmented branded logos (n = 8608) and images of the food (n = 5207), indicating a strong emphasis by food manufactures on visual branding and product depiction. Notably, for child-directed marketing features, we identified 755 branded characters, 320 licensed characters. In contrast, celebrity endorsements were relatively rare, with only 47 occurrences. Visual design elements such as fun or cool motifs (n = 164) and creative use of shapes and colors (n = 309) also featured prominently. Interestingly, we annotated relatively fewer occurrences of explicit incentives such as gifts, prizes, and giveaways (n = 62) or tie-ins with movies, sports, or TV shows (n = 15), or games (n = 1). Furthermore, broad marketing features related to perceived benefits such as value or easy to prep (n = 256), taste and texture (n = 437), convenience (n = 36), and health-related claims (n = 99) were annotated.

High accuracy of image classification algorithms in detecting presence or absence of child-directed marketing

After fine-tuning the kNN classifier, k = 2 was found to be the optimal model. This choice of k indicates that classifications are based on the two nearest neighbors from the training set images, measured using the Minkowski distance (Fig. 2A). The plateauing of performance metrics past k = 2 and increasing the k value did not significantly enhance the model's performance, suggesting that the model's effectiveness was maximized at lower k values. With k set to 2, the image classifier achieved its best performance metrics, an accuracy of 0.87, a precision of 0.88, a recall greater than 0.87, and an F1 score of 0.87. These scores imply that our kNN model accurately recognized over 87% of the child-directed marketing images within the validation set. In addition, 88% of the features the model identified as child-directed marketing were indeed geared towards children, indicating strong precision in classification. Although the performance of kNN is strong, with an F1 score of 0.87, there was still room for improvement in balancing false positives and false negatives.

Fig. 2.

Performance of machine learning and deep learning algorithms in classifying the presence or absence of child-directed marketing on food and beverage packaging (A) KNN. (B) SVM. (C) CNN. (D) ROC curve comparison. Panels A, B, and C show model performance across training conditions for KNN (k values), SVM (C values), and CNN (training epochs), respectively. Panel D presents the ROC curves for both KNN (green line), SVM (orange line), and CNN (blue line) models. Each panel includes common classification metrics: accuracy (overall correct predictions), precision (correct positive predictions out of all predicted positives), recall (correct positive predictions out of all actual positives), and F1 score (balance between precision and recall). The ROC curve visualizes the trade-off between true positive rate and false positive rate, with AUC summarizing overall classification performance.

The SVM classifier demonstrated slightly better performance compared to the kNN classifier. By evaluating different values of the regularization parameter c, we found that c = 10 provided the best results (Fig. 2B). Specifically, the SVM model with c = 10 achieved an accuracy of 0.89, a precision of 0.89, a recall of 0.89, and an F1 score of 0.89. These metrics indicate that the fine-tuned SVM model correctly classified approximately 89% of the validation set images, and was effective at minimizing false positives while correctly identify positive instances.

The CNN model achieved a high level of accuracy early in the training process, with the accuracy stabilizing around 0.92 across the epochs (Fig. 2C). This suggests that the model was able to effectively learn from the training data, maintaining consistent performance throughout the training period. The training and validation loss values after the final epoch were 0.014 and 0.325, respectively. The low training loss indicates that the model fit the training data very well, while the slightly higher validation loss suggests some level of overfitting, though not excessive given the strong overall accuracy.

The ROC curves further underscores performance of the KNN, SVM, and CNN models (Fig. 2D). The AUC of CNN model reached 0.96, significantly outperforming KNN and SVM models (0.91 and 0.93, respectively) (Fig. 2D and Supplementary Table S3), indicating excellent discrimination between the positive and negative classes, confirming the model's robustness in image classification. The SVM model showed slightly higher performance for classifying food and beverage package images into child-directed marketing categories than the kNN model (Supplementary Table S3). Overall, these results demonstrate that the CNN model was highly effective, achieving both a high accuracy and an impressive AUC, suggesting its strong potential for the given classification task.

Varied performance of image object detection algorithm in identifying type of child-directed marketing features

The confusion matrix in Fig. 3 presents the classification performance of the YOLOv12 object detection model in identifying the different types of child-directed marketing features on food and beverage packaging. The diagonal values represent correctly classified instances for each feature, while off-diagonal values indicate misclassifications or unable to identify features between different categories. The model demonstrated high performance in identifying certain categories (see Supplementary Table S1 for definitions), with notable high accuracy for features like Pic-Logo-brand (82% correctly classified), M2K-Char-licensed (75% correctly classified). Additionally, the classifier showed moderate high accuracy for M2K-Char-branded (73% correctly classified), indicating that the model effectively differentiates branding-related imagery. However, some categories exhibited challenges in classification. For instance, M2K-Design-visual/shape/color and M2K-Design-fun/cool demonstrated relatively low accuracy, likely due to the subtle and overlapping visual characteristics inherent in these design features.

Fig. 3.

Performance of the fine-tuned YOLOv12 computer vision model in detecting and classifying child-directed marketing features. This normalized confusion matrix shows the classification performance of the fine-tuned YOLOv12 model across eight child-directed marketing categories. Each row corresponds to the model's predicted label, and each column corresponds to the true (human-annotated) label. Values represent the proportion of predictions for each class (row-wise), with darker blue indicating higher accuracy. High values along the diagonal reflect correct classifications, while off-diagonal values indicate misclassifications. M2K, marketing to kids.

Demonstration of AI application in evaluating compliance with proposed child-directed advertising restrictions in a specific food category

To assess the practical application of our AI-based strategy for measuring and monitoring marketing-to-children policies, we applied the algorithm to classify the presence of child-directed marketing features in food and beverage products, with a particular focusing on breakfast cereals, a category heavily marketed to children through colorful packaging, cartoon characters, and other appealing visual elements.56, 57, 58 According to a study, ready-to-eat cereals were consumed by 37.6% of children aged 2–12 years in Canada, constituting one of the most commonly consumed breakfast foods for children.⁵⁹ As these products are often high in sugar and sodium,⁶⁰ they represent an ideal case study for examining child-directed marketing practices and their public health concern of increased risk of childhood obesity and diet-related noncommunicable diseases. Our results showed that 39.2% of the 1765 breakfast cereals products in FLIP2020 database displayed child-directed marketing features identified by our AI algorithms (Table 1).

Table 1.

Estimated proportions of Canadian breakfast cereals products with or without child-directed marketing features that would be subject to proposed child-directed advertising restrictions.

			Proposed marketing to children advertising restrictions
			PERMITTED	RESTRICTED
			101 (14.2%)	608 (85.8%)
Displayed child-directing marketing	NO	431 (60.8%)	72 (10.1%)	359 (50.7%)
	YES	278 (39.2%)	29 (4.1%)	249 (35.1%)

Presence or absence of child-directed marketing feature in food and beverage front of package (YES or NO) using CNN image classification algorithm. Each food assessed against Health Canada's proposed nutrient profile model thresholds to determine if the food product could be advertised to children (PERMITTED or RESTRICTED). n = 709 from FLIP2020 database after removing products with missing nutrition information.

Among all breakfast cereals products that displayed child-directed marketing features, 89.5% would be restricted under a Canada's proposed marketing-to-children regulations if the regulations are applied to food packaging. Those products were considered unhealthy as they exceed the Health Canada's nutrient thresholds for advertising restrictions. Only 10.5% breakfast cereals products currently displaying child-directed marketing features would be permitted (i.e., considered healthier according to Health Canada's nutrient thresholds for advertising restrictions).

Table 2 presents the relationship between child-directed marketing features, nutrition quality under proposed child-directed advertising restrictions, and food price. Breakfast cereal products displaying child-directed marketing features on food packaging were negatively associated with food prices but not significant, with a coefficient of −0.25 (p = 0.072). Products considered as unhealthy according to the proposed marketing to children advertising restrictions had a positive, marginal association with food prices, with a coefficient of 0.168 (p = 0.056), controlling for store, container size, brand type.

Table 2.

Relationship between child-directed marketing features, nutrition quality under proposed child-directed advertising restrictions, and the price of breakfast cereals.

	Coefficient	Standard Error	p	CI [0.025, 0.975]
Not displayed M2K–Restricted	0.168	0.088	0.056	−0.004, 0.340
Displayed M2K–Permitted	−0.254	0.141	0.072	−0.532, 0.023
Displayed M2K–Restricted	−0.050	0.092	0.589	−0.231, 0.131

Presence or absence of child-directed marketing feature in food and beverage front of package (YES-displayed M2K or NO-not displayed M2K). Each food assessed against Health Canada's proposed nutrient profile model thresholds to determine if the food product could be advertised to children (PERMITTED or RESTRICTED). Adjusted for store, container size, brand type, and year. n = 686 from FLIP2020 database after removing products with missing nutrition information. Not Displayed M2K—Permitted was set as the reference group.

Discussion

This study presents the first AI-driven approach to identify and monitor unhealthy child-directed marketing on food and beverage packaging. Leveraging advanced image classification and object detection AI techniques, such as CNN with ResNet18 and YOLOv12, we demonstrated the feasibility and effectiveness of machine learning and deep learning algorithms in automating the identification and classification of child-appealing marketing features on food and beverage packaging, which achieved a high accuracy of 0.90. The automated strategies significantly accelerate previously manual processes, enabling timely evaluation of large-scale food datasets, tracking evolving food marketing trends, and supporting real-time compliance monitoring of marketing-to-children policies.

The strategies developed in this study further enhances the utility of FLIP, the only comprehensive branded food and beverage database based in Canada with product images, making it more useful for analyzing the Canadian food market. Most importantly, these methods are adaptable to other multidimensional databases containing food product images, marketing media (e.g., social media, online video content, television and streaming advertisements), and child-related settings (e.g., schools and sports venues) where food marketing images are available. This adaptability offers broad applicability across diverse data sources on a global scale. These advancements provide a robust foundation for generating data-driven evidence to inform ongoing policy development aimed at restricting marketing to children, as well as for enabling efficient monitoring and evaluation of child-directed food marketing policies.

This study demonstrated the importance of selecting and fine-tuning appropriate machine learning models to classifying food and beverage packaging images into the presence or absence of child-directed marketing. The optimized kNN classifier, with k = 2, achieved good performance with an accuracy of 0.87, precision of 0.88, recall of 0.87, and an F1 score of 0.87. These results highlight the effectiveness of the kNN model in identifying core and broad child-directed marketing features. However, despite its strong overall performance, the model's F1 score suggests some remaining challenges to balance false positives and false negatives. This indicates that future improvements in feature extraction and data augmentation can further refine the kNN classifier's performance in identifying a broader range of marketing strategies. In comparison, the SVM model, with a fine-tuned regularization parameter c = 10, outperformed the kNN model across all key metrics, especially higher F1 score of 0.89. These improvements reflect the SVM's stronger capability in minimizing misclassifications and correctly identifying positive instances of child-directed marketing features. This suggests that the SVM model is currently the most effective traditional machine learning approach to classify these marketing features. The CNN model demonstrated the highest overall accuracy of 90%, with an AUC of 0.96, indicating excellent discrimination between child-directed and non-child-directed marketing features. However, a higher validation loss compared to the training loss suggests potential overfitting, which needs to be addressed through further regularization or augmentation strategies. Despite its robust performance in accuracy and AUC, the CNN model struggled with nuanced features, indicating that more refined architectures or feature-specific training might be required to accurately identify subtle and overlapping elements of child-directed marketing. Further refining the model's feature extraction, increasing the sample's image resolution, fine-tuning the model, or integrating additional labeled image training data could improve the model's ability to recognize a wide range of marketing strategies.

Object detection results further underscore the challenges in identifying specific child-directed marketing features beyond the general presence of child-directed marketing. Overall, our model demonstrated higher accuracy in classifying distinct, visually recognizable features such as branded logos (82%) and licensed characters (75%) displayed on food and beverage packaging. The results suggest that familiar and popular figures are commonly used to appeal to children, along with the prevalence of visually stimulating designs. On the other hand, direct promotions and entertainment tie-ins appear less in the packaging design on the front of these products. However, the model still faced difficulties in accurately identifying more abstract or visually subtle elements, such as visual shapes and fun designs. These challenges likely stem from the inherent complexity and overlap of visual attributes, mixed marketing features such as vibrant colors and playful fonts, and the limited availability of certain features in the training data due to their infrequent appearance on food packaging. These findings underscore the need for future work to refine models for capturing nuanced visual distinctions in packaging designs than the current architectures. Enhancements could include further fine-tuning object detection algorithms, incorporating advanced feature representation techniques, expanding labeled datasets for underrepresented features, or adopting hybrid approaches that leverage both traditional machine learning and the latest deep learning advancements.

The pioneering application of AI to monitor child-directed marketing on breakfast cereal products demonstrates its significant application for public health policy. Using the automated AI method, we identified a high prevalence of child-directed marketing features on breakfast cereal products (39.2%) of which 89.5% would be considered unsuitable for advertising to children under the proposed regulations, highlighting the need for continued monitoring of child-directed food marketing practices and stronger enforcement of guidelines to better protect children's health. Future research is needed to investigate whether this trend is similar across different food categories. The AI-driven methods developed in this study provide a scalable and cost-effective solution for continuous monitoring of child-directed food marketing practices.^12,25 The methods can be applied to datasets from many countries across the globe, including FLIP 2023 in Canada, and FLIP-LAC 2022 Latin America and the Caribbean. This capability is particularly important as governments and health organizations increasingly recognize the need for robust monitoring systems to enforce regulations on unhealthy food marketing to children worldwide. For example, it provides a powerful tool for assessing compliance and evaluating the effectiveness of Canada's proposed Bill C-252 and the WHO's 2023 guidelines for regulating child-directed marketing. By automating the detection and analysis of child-directed marketing, policymakers can obtain timely and accurate data to support evidence-based decision-making and policy evaluation.

This study has several limitations that should be acknowledged. First, the annotated dataset food and beverage products in this study, while substantial and targeted toward child-relevant categories, does not encompass the full food supply and may not fully capture the diversity of packaging designs or marketing strategies across all product categories. As such, interpreting results beyond the selected product categories or to other food environments (e.g., restaurants, convenience stores) should proceed with caution. Second, while the machine learning models achieved high performance overall, the classification of more abstract or subtle marketing features, such as fun fonts or colorful shapes, remains challenging due to their variability, subjectivity, and lower frequency in the training data. Moreover, misclassifications between visually similar categories suggest that distinguishing complex and nuanced design elements remains a key challenge for the model. Third, although this study focused on static packaging images, children are increasingly exposed to marketing across digital and dynamic media, including video advertisements and in-app content, which may require different detection strategies and model architectures. Fourth, the use of data from 2017 to 2020 may not reflect the most current marketing practices, especially post-pandemic; although the methods can be applied to future datasets, real-time implementation would benefit from continual updates and retraining. Finally, the association analyses were cross-sectional and observational in nature, limiting causal inference about relationships between marketing, nutrition quality, and price. Future work should explore longitudinal data and integrate behavioral measures to strengthen causal claims.

The development of AI-enhanced food marketing assessment methodologies enables the integration of food marketing data with existing purchasing data and packaged food composition data (e.g., nutrition information, price, etc.), offering insights into how child-directed food marketing influences affordability, dietary quality, and health inequities.^6,10 For example, unhealthy child-directed food products may disproportionately impact children in low-income households, exacerbating disparities in dietary patterns and childhood obesity. This AI automated approach helps evaluate whether policies effectively reduce these disparities and protect children's health. Additionally, this research provides a useful strategy for assessing other related food marketing techniques that influence consumer food choices (e.g., front-of-package labeling, health claims, nutrition symbols), linking these strategies to product healthfulness and overall health and well-being of the population.

Contributors

G.H. and M.L. designed the studies; G.H. and O.G. wrote the code and analyzed the data; G.H., O.G., and E.Z. wrote the paper, C.M. and D.S. contributed methodology, E.Z., M.D., K.L., and R.N. prepared the data. All authors read and approved the final version of the manuscript. G.H. and M.L. have verified the underlying data.

Data sharing statement

All code, scripts, and data used to produce the results in this article will be available to any researcher provided appropriate ethics approval, inter-institutional data sharing agreements and other regulatory requirements are in place. Additional specific approval from the University of Toronto will also be required. Some approvals are outside the remit of the authors.

Declaration of interests

M.L. reports research funding, travel support, licensing fees, and unpaid leadership or advisory roles. She declares no other competing interests. All other authors declare no competing interests.