Managing Outliers in Adolescent Food Frequency Questionnaire Data

Abstract

Objective:

The goal of this study was to explore the impact of five decision rules for removing outliers from adolescent food frequency questionnaire (FFQ) data.

Design:

This secondary analysis used baseline and three-month data from a weight loss intervention clinical trial.

Participants:

African American adolescents (n = 181) were recruited from outpatient clinics and community health fairs.

Variables Measured:

Data collected included self-reported FFQ and mediators of weight (food addiction, depressive symptoms, relative reinforcing value of food), caregiver-reported executive functioning, and objectively measured weight status (percentage overweight).

Analysis:

Descriptive statistics examined patterns in study variables at baseline and follow up. Correlational analyses explored the relationships between FFQ data and key study variables at baseline data and follow up.

Results:

Compared to not removing outliers, using decision rules reduced the number of cases and restricted the range of data. The magnitude of baseline FFQ-mediator relationships was attenuated under all decision rules but varied (increasing, decreasing, reversing direction) at follow up. Decision rule use increased the magnitude of change in FFQ estimated energy intake and significantly strengthened its relationship with weight change under two fixed range decision rules.

Conclusions and Implications:

Results suggest careful evaluation of outliers and testing and reporting the effects of different outlier decision rules through sensitivity analyses.

INTRODUCTION

Childhood obesity is a serious public health problem in the US. The National Health and Nutrition Examination Survey (NHANES) indicates a steady increase in the prevalence of childhood overweight and obesity over the past 2 decades.¹ Multiple interventions have been developed in response to this alarming trend. Studies evaluating the effects of these interventions have used a variety of different measures to assess key variables, primarily food intake.

The assessment of food intake has emerged as a major challenge in the study of childhood obesity interventions.² Collecting data on children’s food intake is particularly difficult due to their limited capacity to accurately recall food intake and estimate portion sizes,^3,4 variability in daily intake,⁵ and biased reporting.⁶ However, memory and portion estimation skills increase with age,⁶ making recall measures like food frequency questionnaires (FFQ) useful assessment tools when working with adolescents. Relative to other common recall-based measures, e.g., food diaries and 24-hour recalls, FFQs have been criticized for collecting less detailed information regarding foods consumed, cooking methods, and portion sizes, thereby sacrificing accuracy; however, they are recognized as a more practical option given respondents can complete them independently, require a single administration, and are relatively inexpensive.⁷ The Block Kids 2004 Food Frequency Questionnaire is a FFQ designed for children and adolescents ages 2-17 with demonstrated high validity among adolescents and making it a preferred assessment tool in adolescent food intake studies.⁶

Studies using the Block FFQ highlight an important issue in analysis of FFQ data: how to identify outliers. Outliers are extreme values in a dataset, numerically distant from the remainder of the data. In FFQ data, outliers can be the result of reporting or coding errors, or they may reflect accurate self-report of non-normative food consumption, i.e., very high or low intake. Several recommendations for addressing energy intake outlier data in adult samples have been described. Recommendations often involve establishing a priori high and low cut points (sometimes gender-specific) for energy intake or setting cut points based on the distribution of reported intake estimates of each dietary variable.^8-10 No such recommendations have been presented or tested for addressing outliers in child or adolescent FFQ data. Consequently, a variety of unvetted approaches, including using cut points from the adult FFQ literature,¹¹ not searching for outliers, or at least not reporting on the strategy that was implemented,¹² have been used.

In the absence of guidance for how to manage FFQ outliers, it is unclear how different approaches may alter the relationship between energy intake and weight outcomes. Different outlier approaches may also affect relationships with weight-related mediators. Among adolescents, depression¹³ and executive functioning¹⁴ are well-established mediators of the association between energy intake and weight status. More recently, food addiction and the value individuals place on different types of food have been identified as key mediators among adolescents.^{15, 16} The present study used Block FFQ data from a completed adolescent weight loss intervention trial to explore the impact of various approaches to identify and remove outliers on number of cases excluded, average energy intake across 2 time points, and the relationship among energy intake and weight loss mediators and outcomes. Three approaches were evaluated: capping energy intake based on a fixed allowable range of values, distance outside the interquartile range, and distance from the mean in standard deviation units.

METHODS

The present study was a secondary analysis of data from a broader behavioral intervention study to promote weight loss among African American adolescents with primary obesity (NHLBI U01HL097889). Full details of trial methodology and enrollment/recruitment are reported elsewhere.¹⁷ In brief, the parent study was a sequential multiple assignment randomized trial (a SMART trial¹⁸) to develop an adaptive intervention for adolescent weight loss. Adolescents were first randomized to either home- or office-based weight loss skills training. After 3 months of treatment, adolescents’ weight was re-assessed, and youth who had lost at least 3% of their original weight continued their original treatment modality, with emphasis shifting to relapse prevention. Youth with <3% weight loss were re-randomized to 3 months of either home-based contingency management or continued skills training. At the end of the 6-month program, youth across treatment conditions had decreased their percentage overweight by 2.96% [95% CI: 1.64%, 4.27%]; however, there were no significant between group differences.¹⁷ The parent study is registered with clinicaltrials.gov (accessed 06/26/2020; identifier: NCT01350531).

Participants

Adolescents were recruited using convenience sampling procedures from primary care, endocrine, cardiology, and asthma clinics located within a large tertiary children’s hospital and from community health fairs. Inclusion criteria were (a) English-speaking families self-identifying as African American and living <30 miles of the medical center and (b) adolescents aged 12-16 with BMI > 95th percentile (obese). Exclusion criteria were (a) obesity secondary to medication or chronic condition, (b) medical conditions preventing participation in normal exercise, and (c) pregnancy.

Procedure

The university’s institutional review board approved the study via a full review process. Hospital recruits were identified by medical staff during clinical encounters and via data extracted from the hospital electronic medical record data prior to study launch. Medical staff approached eligible patients during outpatient clinic visits, briefly describing the study to families and obtaining a release of contact information from interested caregivers. Caregivers of recruits whose electronic medical record indicated they met the BMI and age criteria were mailed a letter of introduction describing the study and providing a contact number to opt out of further contact. Caregivers recruited from community health fairs signed a release of contact information. Study staff followed up with caregivers recruited using both strategies by phone to further assess eligibility and interest in participation. Study staff obtained written parental informed consent and adolescent assent from interested, eligible families during the first home-based data collection visit.

Data were collected in participants’ homes at baseline and 7 months by research assistants blinded to treatment condition. Adolescents’ weight was assessed at 3 months for re-randomization purposes. Families received a $50 incentive for completing the baseline and 7-month data collections and $10 for the 3-month weight assessment. Transportation or parking passes were also provided.

Measures

Adolescents’ self-reported energy intake was assessed using the Block Kids 2004 FFQ.² Adolescents reported the number of days in the past week they consumed 63 types of food and 10 categories of beverages. Adolescents were asked to report the quantity of food usually eaten, with pictures depicting various serving sizes provided for reference. The publisher, NutritionQuest (www.nutritionquest.com, accessed 06/26/2020), provided a comprehensive nutrient analysis including a dietary analysis (e.g., total energy in kilocalories [kcal] and total fat, carbohydrates, and sugar in grams), food intake summaries, MyPyramid food group servings, and raw response data. The test-retest reliability for energy intake (kcal, the variable of interest in this research), as assessed by the intraclass correlation, was .63.² The Block was moderately correlated with 2 alternative methods of dietary assessment, the 24-hour recall (r = .38) and the Youth/Adolescent Questionnaire (r = .50).² The Block has also demonstrated a moderate correlation (r = .56) with food diaries.¹²

Consistent with previous adolescent research in primary obesity, adolescents’ weight status was assessed using percentage overweight.^19-22 Percentage overweight is the percentage over the Centers for Disease Control’s median age- and gender-normed body mass index (BMI, kg/m²). Weight was assessed with the Seca 869 scale, and height was assessed using the Seca 213 Stadiometer (Seca, Hanover, MD). Weight was calculated from the average of 2 measurements collected between 1 and 9 days apart (M = 4.42, SD=2.10). Body Weight Index (BMI) was computed using Epi Info software version 3.5.1 (CDC, Atlanta, GA).

Food addiction, or engaging in addictive eating patterns in response to certain foods, has been associated with greater food intake among adolescents with obesity, making it an important correlate of weight management.¹⁵ Food addiction was assessed with the Yale Food Addiction Scale for Children (YFAS-C).²³ The YFAS-C is a 25-item questionnaire that assesses signs of addictive eating based on the DSM-IV²⁴ criteria for substance dependence. Summary scales include the number of food addiction symptoms reported and a binary classification of whether clinical diagnostic criteria were met. In its development study, the YFAS-C demonstrated internal consistency as well as convergent and incremental validity.²³

Because of the strong relationship between depressive symptoms overeating behavior among adolescents,²⁵ depression is a frequently examined correlate in weight loss studies. Adolescents’ depressive symptoms were assessed with the 8-item Patient-Reported Outcome Measurement Information System (PROMIS) Depressive Symptoms - Pediatric Short Form v1.0.²⁶ The PROMIS measures negative mood, decreased positive affect, and negative views of the self in the past week. Adolescents indicated how often they felt each symptom on a 1 (never) to 5 (almost always) scale. The PROMIS was normed with a sample of 1,529 youth recruited from public schools, hospital-based outpatient clinics, and specialty pediatrics clinics, and 21% of the item calibration sample were African American youth.²⁶

Executive functioning refers to self-regulatory cognitive processes involved in inhibition, attention, set-shifting, planning, and organizing. These cognitive processes have direct implications for adherence to weight management strategies making them an important correlate of weight loss.²⁷ Adolescents’ executive functioning was assessed with the 86-item Behavior Rating Inventory of Executive Function - Parent Report (BRIEF).²⁸ Caregivers rated the frequency of their child’s problems over the past 6 months with inhibition, shifting situations, modulating emotions, working memory, planning/organizing, and task- and self-monitoring using a 1 (never) to 3 (often) scale. Items were summed to generate a total score, the Global Executive Composite. In a standardization sample with more than 1,400 participants matched to US population for gender and ethnicity (13.5% African American, 38.0% lower middle or lower class socioeconomic status), the overall Cronbach’s alpha was .93.²⁸

The Relative Reinforcing Value of Food (RRV) assesses the extent to which an individual is responsive to food cues, a characteristic that is more dominant in children who are overweight.¹⁶ Using a questionnaire approach developed and validated by Epstein and colleagues,²⁹ adolescents were asked to report how many portions of a highly palatable, preferred snack food (they selected Oreos, M&Ms, or Doritos corn chips) they would consume at 19 different cost tiers ranging from $0.00 (free) to $1120. Their responses were summarized into 4 different RRV scales. The total scale represents the total number of responses for food (portions selected) across all 19 cost tiers. The breakpoint is the first reinforcement schedule for which the adolescent’s response was 0 (would not purchase the snack food at that cost). The maximum expenditure is the highest cost tier endorsed (highest price willing to pay). The maximum price is the cost tier with the greatest expenditure (tier where the most money was spent).

Statistical Analyses

Variables were created for energy intake under various decision rules for removal of outliers. Selection of the decision rules began with a review of the published literature on outliers in dietary intake measures,^30,31 including a more focused review of outlier approaches in studies that involved use or evaluation of the Block Kids 2004 FFQ. The review also incorporated a document created by NutritionQuest describing recommend data exclusion criteria; however, this document was not specific to the child version of the measure, as no such guidelines have been developed by the company.

Because the purpose of the present study was to assess the impact of broad strategies for identification and removal of outliers, decision rules that involved removing outliers on an analysis-by-analysis basis (e.g., based on residuals in a regression analysis) were excluded. The literature suggested 3 broad ways that energy intake outliers are commonly identified, none of which were specific to the adolescent population: fixed allowable range of values (based on expected intake from a human metabolism standpoint, by far the most common approach), distance outside the interquartile range (IQR, the middle 50% of values), and distance from the mean in standard deviation units. Additionally, the literature described 2 different cutoffs for the fixed allowable range of values and the IQR approaches. ^31,31 Consequently, analyses for this study were conducted using a total of 6 conditions, 1 with no outliers identified and removed and the other 5 with outliers removed using the decision rules from the review of relevant literature and documentation: 2 based on fixed ranges of values (500-3500 kcals [narrower] and 500-5000 kcals [broader]), 2 based on the IQR (multiplying by 1.5 or 3), and 1 based on standard deviations from the mean (Table 1).

Table 1.

Decision Rules for Energy Intake Outlier Removal

Decision Rule Name	Decision Rule Description
Narrower fixed range 500-3500 kcals	Removes values below 500 or above 3500 kcals per day
Broader fixed range 500-5000 kcals	Removes values below 500 or above 5000 kcals per day
IQR 1.5	Removes values below the first quartile minus 1.5 times the IQR or above the third quartile plus 1.5 times the IQR
IQR 3	Removes values below the first quartile minus 3 times the IQR or above the third quartile plus 3 times the IQR
SD >2	Removes values more than two standards deviations away from (above or below) the mean

Note. IQR = interquartile range. kcals = kilocalories. SD = standard deviation.

First, descriptive statistics were used to describe participants’ demographic characteristics and energy intake under all 6 conditions (i.e., with no outliers removed and under the 5 decision rules) at baseline and at follow-up were generated. Next, change in energy intake was calculated for all 6 conditions and the relationships between changes in mean energy intake and change in weight over the same time period were examined using Pearson correlation. Finally, Pearson correlations were used to explore the relationships between energy intake for all 6 conditions and key study variables including symptoms of depression, executive functioning, symptoms of food addiction, meeting criteria for food addiction, and the reinforcing value of food. Correlations were calculated for both the baseline data and the follow-up data. Analyses were conducted using SPSS version 24 with the alpha set to .05.

RESULTS

Participants were 181 African American adolescents (Table 2). Summaries of energy intake at both baseline and follow-up including outliers and under the 5 decision rules are presented in Table 3. Compared to no removal of outliers, the number of cases removed ranged from 7 (3.9%) to 22 (12.2%) at baseline and 5 (3.2%) to 23 (14.6%) at follow-up. As expected, in all instances, the standard deviation was reduced and the range of values narrowed when outlier decision rules were applied (compared to no removal of outliers), albeit to varying degrees. At both times, the narrower fixed range (500-3500 kcals) decision rule resulted in the most cases removed. The standard deviation-based decision and IQR 3 rule resulted in the fewest cases removed at baseline and follow-up respectively.

Table 2.

Demographic Characteristics of African American Adolescents Enrolled in the Weight Loss Intervention Trial (N = 181)

Variable	Mean	SD	Range
Age	14.26	1.45	12 – 17
BMI	38.15	7.45	25.70 – 60.50
Percent Overweight	96.81	37.59	35.38 – 218.47
Variable	N	%
Gender
Female	122	67.40
Male	59	32.60
Ethnicitya,b
Hispanic	3	1.70
Non-Hispanic	176	97.80
Household
One-Parent	115	63.50
Two-Parent	66	36.50
Parent Incomea
< $25,000	126	69.61
≥ $25,000	52	28.73

Note. BMI = body mass index. N = number of participants. SD = standard deviation.

^aNumbers do not sum to 181 (the total sample size)/100% due to missing data

^bRace is not reported because all study participants self-identified as Black/African American

Table 3.

Energy Intake Measured by Food Frequency Questionnaire at Baseline and Follow-up

	Baseline			Three-Month Follow Up
Condition	Number of Cases	Energy Intake Mean (SD)	Range	Number of Cases	Energy Intake Mean (SD)	Range
No removal of outliers	181	1325.0(854.9)	107-5547	157	1215.3 (1233.2)	66-12589
500-3500 kcals	159	1312.0(605.2)	532-3458	134	1125.4(547.9)	512-3147
500-5000 kcals	163	1373.6(713.7)	532-3977	138	1216.2(759.9)	512-4765
IQR 1.5	167	1137.4(525.1)	107-2577	145	957.8 (444.3)	66-2137
IQR 3	176	1235.3 (666.4)	107-3584	151	1034.0(578.1)	66-3147
SD >2	174	1209.1 (623.0)	107-2999	152	1050.3 (610.2)	66-3511

Note. SD = standard deviation. IQR = interquartile range. Kcals = kilocalories.

Table 4 presents the results of analyses involving energy intake change and the relationships between changes in energy intake and weight. Mean change in energy intake ranged from a low of 107 (i.e., a reduction of 107 kcals per day) when no outliers were removed to a high of 199 under the narrower fixed range (500-3500 kcals) decision rule, which also resulted in the largest number of cases removed. When parametric statistics were used, energy intake change was associated with weight change under the 2 fixed range decision rules (500-3500 kcals and 500-5000 kcals) but not in any of the other conditions. When nonparametric statistics were used, the strength of the association was reduced and energy intake change was not associated with weight change under any condition.

Table 4.

Mean energy intake change from baseline to follow-up and relationship between energy intake change and weight change.

Condition	Number of Cases (% of total cases)	Energy Intake Change Meana (SD)	Correlation with Weight Change: Pearson	Correlation with Weight Change: Spearman
No removal of outliers	157(100%)	107.40(1254.82)	.08	.10
Calories 500-3500	124 (79%)	199.38(734.03)	.18*	.16
Calories 500-5000	128 (82%)	165.88 (840.42)	.20*	.17
IQR 1.5	135 (86%)	180.44 (540.71)	.01	−.01
IQR 3	148 (94%)	183.14(703.04)	.14	.09
SD >2	148 (94%)	183.14(703.04)	.14	.09

^*p ≤ .05. IQR = interquartile range. SD = standard deviation

^aPositive values indicate a reduction in daily energy consumption from baseline to follow-up

Table 5 presents the results of correlational analyses exploring the relationships between energy intake for all 6 conditions and key study variables at baseline (above the slash) and at follow-up (below the slash). Although the sizes of the correlations varied slightly by condition, energy intake was not significantly correlated with executive functioning for any condition at either time. Relationships between energy intake and symptoms of depression varied somewhat across conditions: Energy intake was positively and significantly related to symptoms of depression at both baseline and follow-up in the broader fixed range (500-5000 kcals) decision rule condition but only at baseline for all other conditions. The magnitude of the positive correlation between energy intake and symptoms of depression was attenuated under all decision rules. At follow-up, application of decision rules had a variety of effects, increasing or decreasing the correlation and even reversing its direction in the narrower fixed range (500-3500 kcals) decision rule condition.

Table 5.

Correlations Between Energy Intake Measured by Food Frequency Questionnaire and Weight Loss Mediators

Conditi on	Executive Functioni ng T1/T2	Depressi on T1/T2	Sympto m Count T1/T2	YFAS Diagnos is T1/T2	RRV Tota 1 Tl/T 2	RRV Breakpoi nt T1/T2	RRV Maximu m Expenditu re T1/T2	RRV Maximu m Price T1/T2
No removal of outliers T1/T2	.11/−.03	.30***/.10	.24***/.14	.24***/.04	.27*/.13	.01/.01	.35***/.63***	.26***/.70***
500-3500 kcals T1/T2	.03/.00	.21**/.07	.20*/-.06	.16*/-.09	.17/−.13	−.05/−.03	.14/.01	.08/.04
500-5000 kcals T1/T2	.14/−.01	.23**/.22**	.22**/.14	.17*/.07	.15/−.05	−.02/−.02	.19*/.00	.15/.02
IQR 1.5 T1/T2	.13/.01	.27***/−.01	.20**/−.02	.23**/−.03	.13/−.06	−.03/.01	.02/.09	.01/.13
IQR 3 T1/T2	.03/.01	.25***/.04	.20**/−.06	.17*/−.05	.18/−.06	−.02/.00	.14/.05	.09/.07
SD >2 T1/T2	.04/.06	25***/.10	.18*/−.01	.20**/−.05	.07/−.05	−.02/−.01	−.01/.04	−.02/.06

^*p ≤ .05

^**p ≤ .01

^***p ≤ .001

Note. Values above the slash represent baseline correlations and below the slash are follow-up. IQR = interquartile range. kcals = kilocalories. T1 = baseline. T2 = follow-up. YFAS = Yale Food Addiction Scale. RRV = Relative Reinforcing Value of Food. SD = standard deviation.

For food addiction, energy intake was positively and significantly correlated with both addiction variables at baseline but not at follow-up for all conditions. As with depression, these correlations were attenuated when decision rules were used at baseline. At follow-up, the correlations between energy intake and both food addiction variables reversed direction under all decision rules except for the broader fixed range (500-5000 kcals) decision rule condition.

For the RRV, breakpoint was unrelated to energy intake for all conditions at both time points. Total RRV was significantly positively correlated with energy intake at baseline with no outliers removed, but this relationship was non-significant in all other instances. Other RRV variables were significantly positively correlated with energy intake at both time points with no outliers removed. Application of any of the decision rules reduced the magnitude of the correlations to nonsignificance in all but 1 instance, and the magnitude of change was extreme in several instances. For example, for maximum expenditure at follow-up, the correlation was .63 with no outliers removed and reduced to .00 under the broader fixed range (500-5000 kcals) decision rule.

DISCUSSION

Guidance for identifying outliers and the effect of outlier removal in child and adolescent FFQ data has received little attention in the literature. In this study, application of outlier decision rules culled from the extant literature on outliers in adult FFQ data had notable effects (compared to no removal of outliers) on the results and the conclusions drawn from an adolescent sample. Use of outlier decision rules altered the number of cases available for analysis but also change in energy intake over time, relationship between change in energy intake and change in weight status, and relationships between energy intake and relevant psychosocial mediators. Overall, findings supported using multiple approaches to examine data, and transparent sensitivity analyses under less and more restrictive outlier decision rules be reported in future studies.

Of the 6 conditions in this study (i.e., no outliers removed and 5 decision rules: 2 based on fixed intake ranges, 2 based on the IQR, and 1 based on the standard deviation from the mean), the narrower fixed range decision rule (500-3500 kcals) was the most stringent: This rule resulted in the most cases removed from analyses and substantially reduced the variability in the data, potentially artificially restricting the data (i.e., in adolescent weight loss trials, some participants may eat more than 3500 kcals per day). Furthermore, reduced sample size due to outlier deletion may result in less power to detect notable differences between treatment and control groups in weight loss trials, which tend to have small sample sizes even before removing cases. While some studies report the number of cases lost to outlier deletion (e.g., 4% of original sample removed),¹¹ others do not,³² making it difficult to gauge the extent to which loss of power is a concern. Removing potentially valid cases may be particularly problematic in adolescent weight loss intervention studies where it is especially difficult to recruit³³ and retain participants.^{34, 35} Although it is possible to replace outlier values using a variety of imputation methods, no instances of this practice were found in literature reporting adolescent FFQ data.

The current analyses demonstrated the effect of outlier decision rules on change in energy intake, a primary weight loss outcome. Change in energy intake increased under all decision rules compared to the no outliers removed condition. Mean change in energy intake ranged from a low of 107 kcals when no outliers were removed to a high of 199 kcals under the narrower fixed range (500-3500 kcals) decision rule. Application of decision rules strengthened the relationship between change in energy intake and change in weight when traditional parametric tests were used. This relationship was significant under the 2 fixed range decision rules, the decision rules under which sample size was reduced the most, but not for any of the other conditions. The use of nonparametric statistical tests, a conservative alternative less sensitive to extreme values, demonstrated similar findings. The relationship between change in energy intake and weight change was strengthened in the same 2 fixed range decision rules but not significant. These results illustrate how the use (or lack thereof) of outlier decision rules may be substantially impacting the literature in this area, highlighting the critical need to discuss how outliers are being addressed.

In this study, removing outliers using 5 outlier decision rules attenuated relationships between energy intake and study variables (compared to no removal of outliers) in most cases. Specifically, 38 of 40 (95%) baseline correlations were weakened and 32 of 40 (80%) follow-up correlations were weakened. The magnitude of change was as large as −.36 in the baseline data and −.66 at follow-up. In 15% of baseline and 50% of follow-up correlations, removing outliers reversed the direction of the correlation. These findings illustrate the potential problems presented when different strategies for removing outliers are used across the literature.

Given that there is no gold standard method for removing FFQ outliers and the potential problems introduced when different approaches are used across studies, the following approach to addressing energy intake outliers in FFQ data from adolescent samples is suggested. Sensitivity analyses under less and more restrictive outlier decision rules to assess the consistency of findings across different conditions are recommended. In some datasets the use of non-parametric statistical methods such as the Mann Whitney U test or the Kruskal-Wallis test, which utilize the median value over the mean, might be a more suitable approach to outlier management.³⁰ Lastly, it would be remiss to not recommend transparency in the reporting of the strategies used to identify and address outliers.

A large sample and valid and reliable measures of weight, food intake, food addiction, depression, executive functioning, and the reinforcing value of food are strengths of this study. Limitations include a relatively homogeneous sample of African American adolescents recruited primarily from academic medical center clinics for an intervention study. Replication efforts with more heterogeneous samples would strengthen the results reported here. As a secondary analysis, this study is limited to the measures selected by the parent study. Specifically, it would have been useful to examine the presence of outliers across multiple FFQs and compared to a more objective measure of dietary intake. Finally, studies that continue to explore the effects of alternative methods of handling outliers on mediational relationships using analytic approaches like structural equation modeling are important future directions.

IMPLICATIONS FOR RESEARCH AND PRACTICE

This study suggests a process for the identification and removal of outliers. These results suggest investigators examine different decision rules to understand how the removal of outliers affects study findings. Currently, there is insufficient information to suggest a single optimal outlier removal strategy. More research is needed to understand how removal of dietary intake outliers affect weight and psychosocial outcomes. In the meantime, a good strategy is to carefully evaluate potential outliers, test and report the effects of different outlier decision rules through sensitivity analyses, and consider statistical methods that are less sensitive to outliers.

Abbreviations

BMI

Body Mass Index

BRIEF

Behavior Rating Inventory of Executive Function – Parent Report

FFQ

Food Frequency Questionnaires

IQR

Interquartile Range

NHANES

National Health and Nutrition Examination Survey

NHLBI

National Heart, Lung, and Blood Institute

PROMIS

Patient-Reported Outcome Measurement Information System

RRV

Relative Reinforcing Value of Food

SMART

Sequential Multiple Assignment Randomized Trial

YFAS-C

Yale Food Addiction Scale for Children