The validity and reliability of the Dietary Index for a Child's Eating in 2–8‐year old children living in New Zealand

Abstract

This study aimed to examine the validity and reliability of the Dietary Index for a Child's Eating (DICE) in children living in New Zealand (NZ). Caregivers of healthy children aged 2–8 years completed a 4‐day estimated food record (4DFR) for their child and completed the DICE online on two separate occasions, 8 weeks apart. Relative validity was assessed by comparing the DICE and 4DFR total score and component subscores using the Wilcoxon test, Spearman rank correlation coefficients, cross‐classification, and weighted kappa (ĸ) statistic. For evaluating construct validity, the DICE total score was compared with energy and nutrient intake from the 4DFR using linear contrast analysis. Intraclass correlation coefficients (Cronbach's α) and Bland–Altman plots were used to assess the reliability of DICE. From a possible score of 100, the mean ± standard deviation of DICE was 78.2 ± 11.5 and from the 4DFR, was 73.8 ± 10.8, with a positive correlation (r = 0.72; p < 0.001) and moderate agreement (ĸ = 0.49). Cross‐classification showed 61.9% were correctly categorised into the same tertile group from DICE and the 4DFR. Participants in the highest tertile of DICE had higher intakes of fibre, vitamin C, vitamin A, vitamin D, folate, and calcium. Good agreement (α = 0.87) was found for reliability. DICE is a valid and reliable tool for the assessment of children's adherence to a healthy diet, as recommended by the NZ Ministry of Health Food and Nutrition guidelines.

Key messages.

• In New Zealand (NZ), there is currently no valid and reliable dietary index for children, which can assess adherence to the New Zealand Ministry of Health Food and Nutrition guidelines.

• The Dietary Index of a Child's Eating (DICE) was found to be a valid and reliable tool to assess adherence to dietary guidelines in NZ.

• DICE will be a useful tool to assess diet quality in young children aged 2–8 years old living in NZ.

1. INTRODUCTION

The early years of life are important for growth and development and also for shaping future health behaviours. Adopting an optimal diet during childhood may have a protective effect against the development of negative health outcomes such as obesity. According to the New Zealand (NZ) Health Survey in 2015/2016, 11% of children and adolescents (aged 2–14 years) living in NZ were obese (Ministry of Health, 2016).

In the past, the main aim of nutritional studies has been to determine adequate intakes of single foods and nutrients (Waijers, Feskens, & Ocke, 2007); however, there is increasing recognition of the importance of overall diet quality and its association with general health (Moeller et al., 2007). The evaluation of diet quality using dietary indices in children provides an opportunity to follow up food habits and behaviours during childhood in order to predict the later risk of diet‐related diseases (Marshall, Burrows, & Collins, 2014). Dietary indices are typically based on country‐specific dietary guidelines (Golley, Hendrie, & McNaughton, 2011) or a particular dietary pattern (e.g., Mediterranean) (Serra‐Majem et al., 2004). They usually provide a summary score, indicating how closely an individual adheres to these dietary guidelines or patterns. Diet quality indices can be developed from traditional dietary assessment tools such as food frequency questionnaires, or can be developed in a simple questionnaire format based on national dietary guidelines and recommendations. Using the latter approach, diet quality indices can be relatively simple, and provide information about diet quality quickly with minimal participant burden (Bell, Golley, & Magarey, 2013).

At least 13 children's diet quality indices have been developed across different countries (Andrade, Previdelli, Marchioni, & Fisberg, 2013; Burrows et al., 2014; Cheng, Duan, Kranz, Libuda, & Zhang, 2016; Golley et al., 2011; Huybrechts et al., 2010; Kyttala et al., 2014; Manios et al., 2009; Marshall, Watson, Burrows, Guest, & Collins, 2012; Serra‐Majem et al., 2004; Sluik, Streppel, van Lee, Geelen, & Feskens, 2015; Sotos‐Prieto et al., 2015; Vyncke et al., 2013; Wong, Parnell, Howe, Black, & Skidmore, 2013). Three of these indices were developed in Australia (Burrows et al., 2014; Golley et al., 2011; Marshall et al., 2012). Golley et al. (2011) developed and validated the “Dietary Guideline Index for Children and Adolescents.” They used only two 24‐hour recalls to validate the Dietary Guideline Index for Children and Adolescents, which may not reflect habitual dietary intake for dietary data collection and validation. The “Australian Recommended Food Scores for Preschoolers (ARFS‐P)” was developed by Burrows et al. (2014). Although they recruited enough participants to validate the tool, the sample population was from only rural locations, which decreases the generalisability of the results to the urban population. Marshall et al. (2012) developed and validated the “Australian Child and Adolescent Recommended Food Score” in 691 children. Their findings suggested that the Australian Child and Adolescent Recommended Food Score has adequate accuracy for assessing food‐based diet quality in children and adolescent. However, same as previous studies (Burrows et al., 2014; Golley et al., 2011), there is a chance of overestimation of child's diet quality and parental bias in dietary information.

Wong et al. (2013) designed the “NZ Adolescent‐specific Diet Quality Index (NZDQI‐A)” and investigated its validity in 41 adolescents. However, to the best of our knowledge, there is no published validation of a dietary index for children living in NZ.

Moreover, there are several issues associated with assessing dietary intake in children. It has been shown that children younger than approximately 8 years old are not able to recall foods or estimate portion size accurately and do not have the ability to conceptualise the time frame and frequency of food consumption (Livingstone & Robson, 2000). Therefore, it is necessary to involve their parents or caregivers in their dietary evaluation.

In NZ, a Dietary Index for a Child's Eating (DICE) was developed based on the NZ Ministry of Health (MoH) Food and Nutrition Guidelines for healthy children 2 to 18 years (Ministry of Health, 2012). The aim of the current study was to examine the validity and reliability of the DICE, as a measure of adherence to the guidelines, in children aged 2 to 8 years of age.

2. METHODS

A convenience sample of participants was recruited from around NZ using existing contacts, social, and print media. Caregivers were considered as proxy reporters of their children's dietary intake, and caregivers of healthy children aged 2–8 years old with no chronic diseases and living in NZ were eligible to participate. Participants received a study pack containing an information sheet, consent form, a 4‐day estimated food record (4DFR), and an online SurveyMonkey link to the DICE questionnaire. Participants completed the online DICE questionnaire and returned their consent and 4DFR booklet in the post by a prepaid envelope. Eight weeks later, participants were asked to complete the DICE a second time (to assess reliability). When determining reliability, it is recommended to have 4 to 8 weeks between the first and second administration of dietary assessment method. This period of time is long enough so that the respondent is not simply remembering his/her answers, but is short enough to minimise real dietary changes in the interval (Block & Hartman, 1989). To assess validity, data from the first administration of DICE was compared with the 4DFR. The study protocol was approved by the Human Ethics Committee of Massey University (Southern A; approval no. MUHECN14/023), and all caregivers provided written informed consent prior to participating.

2.1. Dietary assessment tools

2.1.1. Dietary Index for a Child's Eating

The DICE consists of 13 questions. The DICE components, scoring, and cut‐off points are based on the NZ MoH Food and Nutrition Guidelines for 2–18 year olds (Ministry of Health, 2012). These guidelines are evidence‐based and specific for the NZ population, therefore providing a standard reference for evaluating the quality of children's diets. The DICE was scored from 0 to 100, with a higher score representing greater adherence to these guidelines. The daily servings of main food groups including fruits, vegetables, milk products, meat/meat‐alternatives, bread, and cereals were scored from 0 (not meeting the guidelines) to 10 (meeting the guidelines) signifying the importance of these food groups (See Figure 1).

Figure 1.

Dietary Index for a Child's Eating questionnaire

The MoH Food and Nutrition Guidelines categorise the colours of fruits and vegetables into five groups. The variety of fruits and vegetables were scored from 0 to 5. Caregivers were asked to record how often they chose wholegrain products. Examples provided by the MoH Food and Nutrition Guidelines include whole‐wheat, whole‐wheat flour, wheat flakes, bulgur wheat, whole and rolled oats, oatmeal, oat flakes, brown rice, whole rye and rye flour, whole barley, and popcorn (Ministry of Health, 2012). The wholegrain products consumption was scored from 0 to 5. The guidelines recommend children consume three meals and two to three small snacks every day (Ministry of Health, 2012). A score of 5 was given when caregivers reported their child eating three or more meals or snacks most days or every day. A score of 0 was assigned when the child ate less than three meals or snacks per day, most days, or every day.

Fluid consumption was scored from 0 to 15. Participants could score a maximum of 15 points if only water and/or milk were consumed. If participants consumed only one serving of fruit juice or flavoured milk alongside water or milk every day, they received 7.5 points, as consumption of these drinks in children should be limited (Ministry of Health, 2012). If participants consumed any other drinks such as cordial, fizzy drinks, sport drinks, tea, or coffee (or more than one serving of fruit juice or flavoured milk per day), they scored 0 points.

2.2. Four‐day estimated food record

Each participant completed a 4DFR over four consecutive days including three weekdays and one weekend day. At the beginning of the study, written instructions were given to the caregivers on how to record the amount, brand names, type, recipe (ingredients of mixed dishes), and preparation methods for foods and beverages, which their children consumed during the recording days. In order to simplify estimates of portion size, caregivers were asked to use household measures (e.g., cups, and teaspoons), weights marked on food packages, and comparisons (e.g., a scoop of ice cream equal to the size of a hen's egg). The importance of recording precise information and following normal dietary habits during the recording period were emphasised.

A trained nutritionist entered the 4DFRs into FoodWorks Professional diet analysis programme (version 8, 2016, Xyris Software), which uses the New Zealand Food Composition Database and FOODfiles (New Zealand Institute for Plant and Food Research Limited, 2010), and then converted into the same components (e.g., the number of serves of fruit, variety of vegetables, and servings of wholegrain products) as included in the DICE so the scoring of the 4DFRs could be aligned to the DICE. The average daily amount of foods and beverages recorded by participants over 4 days were used for calculating the number of servings of the five main components (fruits, vegetables, bread/cereals, milk/milk products, and meat/alternatives).

The number of different colours of fruits and vegetables consumed over the 4 days was used for determining the variety of fruits and vegetables consumed, respectively. If participants consumed wholegrain products on 2 days out of the 4 days assessed (i.e., some days), then they scored 2.5 points. If participants consumed wholegrain products on only 1 day (rarely) or they did not consume (never) from this food group, they scored 0 points. If participants consumed wholegrain products 3 (most days) or 4 (always) days, they scored 5 points. All beverages over the 4 days were considered for determining the fluid component. Fifteen points were given where only water and/or milk was consumed, and 7.5 points if one serving of fruit juice or flavoured milk alongside water or milk per day was consumed. Zero points were given if any other drinks such as cordial, fizzy drinks, sport drinks, tea or coffee were consumed (or if more than one serving of fruit juice or flavoured milk per day was consumed).

The amount of sugar, fat, and salt was identified from the Foodworks analysis of the 4DFR in order to calculate the score for low fat/sugar/salt foods/snacks/drinks components. The NZ MoH Food and Nutrition Guidelines recommend that <10% of total energy should be provided from saturated fatty acids (SFAs) (Ministry of Health, 2012). Therefore, children who consumed <10% of their total energy from SFAs received 5 points and those who consumed ≥10% of total energy from SFAs scored 0 points. Total sugar was calculated using glucose, sucrose, and maltose. Fructose and lactose were excluded from this calculation. The World Health Organisation recommends limiting sugar to <10% of total energy intake (World Health Organisation, 2003). In this study, participants who consumed <10% of their total energy from sugar received 5 points, and 0 points if they consumed ≥10% of total energy from sugar. According to the guidelines, the adequate intake of sodium for children 2–3 years old is 200–400 mg and for 4–8 years old is 300–600 mg (Ministry of Health, 2012). If participants met these recommendations, they scored 5 points otherwise they received 0 points.

2.3. Statistical analysis

The statistical software SPSS version 22 (IBM corporation, New York) was used to analyse the data. Variables were tested for normality using the Kolmogorov–Smirnov test and Shapiro–Wilk tests; and for homogeneity using, the Levene's test. Descriptive statistics were applied for total scores and subscores from DICE and 4DFR data. Total scores were normally distributed and were assessed using parametric tests. Subscores were not normally distributed and non‐parametric tests were applied. A p‐value of less than 0.05 was considered to be statistically significant.

The relative validity of the first administration of DICE total score with the 4DFR total score was assessed using Pearson correlation coefficients, cross‐classification, and the weighted kappa (ĸ) statistic. Cross‐classification was used to determine whether participants were correctly categorised into the same tertile or grossly misclassified into opposite tertiles. Participants were correctly classified when more than 50% were allocated to the same tertile; when >10% were allocated into opposite tertiles, then misclassification was deemed to have occurred (Masson et al., 2003). In order to overcome the effect of chance, the two dietary assessment methods were compared using the weighted ĸ statistic. A weighting of one was used for participants classified into the same tertile, 0.5 for adjacent tertiles, and zero for opposite tertiles. The following categories were used to describe the results of the weighted ĸ statistic; very good agreement (>0.80); good agreement (0.61–0.80); moderate agreement (0.41–0.60); fair agreement (0.21–0.40), and poor agreement (<0.20) (Landis & Koch, 1977).

The Wilcoxon test and Spearman rank correlation coefficients were used to compare the first administration of DICE subscores for each component against the same component subscores from the 4DFR.

Construct validity was assessed by linear contrast analysis in order to investigate the relationship between the tertile of DICE total score distribution and energy and nutrient intakes calculated from 4DFRs. Total energy, protein, carbohydrates, sugar, total fat, saturated fat, dietary fibre, vitamin C, vitamin A, vitamin D, folate, sodium, calcium, iron, and iodine were included in the analysis. Polynomial contrast for nutrient intakes was used in order to calculate the p‐value for the linear trend.

Participants completed the DICE twice, with an interval of 8 weeks to evaluate reliability. Intraclass correlation coefficients (ICC; Cronbach's α) were used to calculate the reliability of DICE. The ICC value of 0.8 or above was considered a good reliability result (Field, 2009). Agreement between the two DICE administrations was assessed using a Bland–Altman plot. Mean differences were calculated and the limits of agreement (mean difference ± 1.96 standard deviations) were determined (Bland & Altman, 1986). Linear regression analysis was undertaken to investigate bias (difference in intake—the dependent variable, and mean intake—the independent variable).

3. RESULTS

Sixty‐five participants (29 boys and 36 girls) were recruited. All caregivers completed the first administration of DICE, 63 completed a 4DFR, and 49 completed DICE a second time. The mean and standard deviation (mean ± SD) of age in boys was 4.1 ± 1.8 years and in girls was 4.3 ± 1.7 years.

3.1. Relative validity of DICE

The mean ± SD DICE score was 78.2 ± 11.5 (range = 47–100) and the 4DFR was 73.8 ± 10.8 (range = 47.5–89). Over 75% of participants scored more than 70 points according to both DICE and the 4DFR. A significant positive correlation was observed between the total scores for DICE and the 4DFR (r = 0.72; p < 0.001). Results from cross‐classification showed 61.9% of participants were correctly categorised into the same tertile from DICE and the 4DFR, and 6.3% were misclassified into opposite tertiles. The weighted ĸ‐statistic demonstrated moderate agreement (ĸ = 0.49) between DICE and the 4DFR.

Subcomponent scores from DICE and the 4DFR are shown in Table 1 and Figure 2. Spearman's correlation coefficients showed significant (p < 0.001) positive correlations between the DICE and 4DFR for servings of fruit (r = 0.91), servings of vegetables (r = 0.96), variety of vegetables (r = 0.44), servings of bread and cereals (r = 0.85), consumption of wholegrain products (r = 0.71), servings of milk and milk products (r = 0.88), servings of meat and its alternatives (r = 0.60), number of meals and snacks (r = 1.0), and fluid consumption (r = 0.61). A significant and inverse correlation was found for low fat foods/snacks/drinks (r = −0.30) consumption (p < 0.05; Table 1). The variety of fruits, low salt and low sugar foods/snacks/drinks components were not significantly correlated with the same components scores from the 4DFR.

Table 1.

Comparison and agreement between sub‐scores for each component from the first administration of DICE and the 4DFR (n = 63)

DICE components	Mean intake (serves/d) (DICE)a	Mean intake (serves/d) (4DFR)a	Mean difference (serves/d) (DICE & 4DFR)	DICE scoresa	4DFR scoresa	Differences between DICE scores and 4DFR scores	Wilcoxon test (p‐value)	Spearman's correlation coefficients (r)	Correlation (p‐value)
Serves of fruit	3.6 ± 0.7	2.0 ± 0.6	1.6	9.0 ± 1.8	9.0 ± 1.8	0	0.655	0.913	<0.001
Variety of fruit	‐	‐	‐	4.7 ± 0.8	4.1 ± 0.9	0.6	<0.001	0.224	0.078
Serves of vegetables	3.0 ± 1.0	1.5 ± 0.6	1.5	7.6 ± 2.7	7.6 ± 2.7	0	0.655	0.965	<0.001
Variety of vegetables	‐	‐		4.2 ± 0.9	3.8 ± 0.8	0.4	0.004	0.440	<0.001
Serves of bread and cereals	3.2 ± 0.8	3.1 ± 0.8	0.1	8.0 ± 2.0	7.6 ± 2.1	0.4	0.002	0.853	<0.001
Serves of wholegrain products	‐	‐	‐	3.9 ± 1.6	3.8 ± 1.6	0.1	0.796	0.712	<0.001
Serves of milk and milk products	4.0 ± 1.7	1.8 ± 0.7	2.2	6.6 ± 2.5	6.4 ± 2.5	0.2	0.206	0.889	<0.001
Serves of meat and its alternatives	2.6 ± 1.0	1.3 ± 0.5	1.3	9.0 ± 1.9	9.6 ± 1.3	−0.6	0.008	0.605	<0.001
Number of meals and snacks	‐	‐	‐	5.0 ± 0.0	5.0 ± 0.0	0	0.677	1	<0.001
Frequency of consuming low fat foods, snacks, and drinks	‐	‐	‐	2.0 ± 2.1	3.8 ± 2.0	−1.8	<0.001	−0.300	0.017
Frequency of consuming low salt foods, snacks, and drinks	‐	‐	‐	3.3 ± 2.1	1.0 ± 1.9	2.3	<0.001	0.191	0.134
Frequency of consuming low sugar foods, snacks, and drinks	‐	‐	‐	4.1 ± 1.5	0.6 ± 1.5	3.5	0.001	−0.025	0.846
Fluid consumption	‐	‐	‐	10.1 ± 5.8	11.3 ± 5.3	−1.2	0.047	0.611	<0.001

Note. DICE: Dietary Index for a Child's Eating; 4DFR: 4‐day estimated food records.

^aValues represent mean ± SD.

Figure 2.

Subcomponent scores (as a percentage) from the first administration of the Dietary Index for a Child's Eating (DICE) and estimated food record (4DFR). For example, for breads and cereals the mean score was 8/10 (80%) for DICE and 7.6/10 (76%) for the 4DFR

3.2. Construct validity

Linear contrast analysis showed that higher intake of fibre, vitamin C, vitamin A, vitamin D, folate (p < 0.05), and calcium (p < 0.001) were associated with increasing tertiles of the DICE total score (Table 2).

Table 2.

4DFR dietary intakes categorised by tertiles of the DICE scores (n = 63)

Nutrients	Tertiles of DICE scoresa
	1 (n = 21)	2 (n = 23)	3 (n = 19)b	P for trend
Energy (kcal)	1572 ± 297	1497 ± 240	1651 ± 258	0.186
Protein (g)	57.8 ± 7.0	58.4 ± 6.0	57.7 ± 7.7	0.934
Protein (%kJ)	15 ± 2.50	15.8 ± 2.10	14.2 ± 2.10	0.634
Carbohydrate (g)	252 ± 21	256 ± 16	260 ± 19	0.442
Carbohydrate (%kJ)	65.7 ± 9.6	69.9 ± 10.6	64.4 ± 10.6	0.390
Sugar (g)	91.7 ± 25.7	88.9 ± 18.8	94.0 ± 26.5	0.783
Total fat (g)	58.9 ± 15.0	56.0 ± 12.7	61.2 ± 11.5	0.446
Total fat (%kJ)	33.3 ± 2.0	33.6 ± 4.7	33.4 ± 3.1	0.364
Saturated fat (g)	29.8 ± 15.5	25.9 ± 11.5	33.2 ± 12.8	0212
Dietary fibre (g)	16.7 ± 3.0	18.9 ± 2.4	18.2 ± 2.8	0.039*
Vitamin C (mg)	30.4 ± 7.6	37.3 ± 14.8	39.1 ± 6.9	0.029*
Vitamin A (μg)	487 ± 116	527 ± 91	574 ± 53	0.015*
Vitamin D (μg)	2.1 ± 1.1	2.3 ± 1.2	3.2 ± 1.2	0.011*
Folate (μg)	185 ± 38	204 ± 40	214 ± 35	0.052*
Sodium (mg)	866 ± 304	919 ± 359	861 ± 320	0.813
Calcium (mg)	569 ± 91	611 ± 86	687 ± 102	0.001**
Iron (mg)	9.7 ± 1.3	9.6 ± 1.2	10.2 ± 1.3	0.299
Iodine (μg)	77.8 ± 10.1	76.7 ± 10.6	76.6 ± 8.2	0.920

Note. DICE: Dietary Index for a Child's Eating; 4DFR: 4‐day food record.

^* Significantly different from tertile 1. p < 0.05 level (two‐tailed).

^** Significantly different from tertile 1. p < 0.01 level (two‐tailed).

^a Values represent mean ± SD.

^b DICE total scores were divided into tertiles based on the natural break (T1 = 21, T2 = 23, T3 = 19 participants).

3.3. Reliability

The mean ± SD for the first and second administration of DICE (n = 49) was 77.9 ± 11.3 and 80.0 ± 12.5, respectively. The ICC was 0.87 (p < 0.001). Figure 3 shows the Bland–Altman plot of the agreement between the first and the second DICE administration. Results from linear regression showed that there was no bias between the two methods; that is the difference in intake between the DICE and 4DFR did not alter across the mean intake (Figure 3).

Figure 3.

Bland–Altman plot for reliability: Agreement between the first and the second Dietary Index for a Child's Eating (DICE) administration. A solid line represents the mean difference between two DICE total scores and dashed lines representing the limits of agreement (mean difference ± 1.96 standard deviations)

4. DISCUSSION

To our knowledge, there is currently no valid and/or reliable dietary index that can evaluate the adherence of children to the NZ MoH Food and Nutrition Guidelines. Therefore, the DICE was developed for healthy children aged 2–8 years old.

The main objective of this study was to examine the relative and construct validity and reliability of the DICE. Our results demonstrated that the DICE has moderate relative validity with 4DFR total scores and good reliability. Construct validity results showed that participants in the highest tertile of DICE had higher intakes of fibre, vitamin C, vitamin A, vitamin D, folate, and calcium. These results suggest that the DICE is a valid and reliable tool that may be utilised in research studies requiring assessment of diet quality in healthy children living in NZ.

4.1. Description of DICE scores

The maximum score of 100 for DICE would represent meeting all the recommendations for children from the NZ MoH Food and Nutrition Guidelines. Therefore, the higher the score on DICE, the greater adherence is to current guidelines. The range of DICE total scores was between 47 and 100, with a mean total score of 78.2, and over 75% of participants scored more than 70 points, suggesting adherence of participants to the NZ MoH Food and Nutrition Guidelines was moderate to high. The wide range in scores reflects the ability of the tool to determine a range of adherence to current guidelines.

The dietary intake of the children in this study was comparable with the “2008/09 National Survey of Children and Young People's Physical Activity and Dietary Behaviours in New Zealand” results (Clinical Trials Research Unit, 2010). For instance, 39.7% and 68.6% of 5–19‐year olds met the recommendations for vegetables and fruits, respectively (Clinical Trials Research Unit, 2010), compared with 50.8% and 75.4% of participants in the current study. In the national survey, half of participants (50.3%) ate brown/whole meal/wholegrain bread most of the time (Clinical Trials Research Unit, 2010) and more than half of our participants (67.7%) consumed wholegrain products most of the time. It appears that perhaps our population group had a “better diet” than “average,” and thus one might hypothesise that the caregivers in the current study may be more nutritionally aware, probably due to the self‐selected nature of recruitment. It should also be noted that there was a difference in the maximum age of the two study populations (8 years vs. 19 years).

4.2. Relative validity (DICE vs. 4DFR)

Typically, relative (or criterion) validity is used in order to investigate how well a newly developed index and its scores (“test” dietary assessment method) agree with a standard and independent dietary assessment method (“reference” dietary assessment method), which has a greater degree of validity (Waern et al., 2015). It is suggested that relative validity is examined by using a range of statistical tests in order to ensure the robustness of the validity process (Cade, Thompson, Burley, & Warm, 2002). Furthermore, correlation coefficients can be misleading, as they measure the relation between two methods rather than the agreement between them (Bland & Altman, 1986). Therefore, relative validity was assessed using different statistical analysis tests (Wilcoxon test, Spearman rank correlation coefficients, cross‐classification, and weighted ĸ statistic). The DICE total score showed a high correlation (r = 0.72; p < 0.001) and moderate agreement (ĸ = 0.49) with the 4DFR. In addition, over half of participants (61.9%) were correctly categorised into the same tertile.

In the present study, significant, positive, moderate to high correlations (r = 0.44–1) were found for DICE subscale components including servings of fruits and vegetables, bread and cereals, milk and milk products, meat and its alternatives, variety of vegetables, consumption of wholegrain products, number of meals and snacks, and fluid consumption. The average correlation for food groups was 0.66 (range − 0.30 for low fat foods/snacks/drinks to 1 for number of meals and snacks). This compares well with similar studies in NZ and Australia (Burrows et al., 2014; Wong et al., 2013). In 41 NZ adolescents, Wong et al. (2013) found an average correlation of 0.39 for food groups between NZDQI‐A and a 4DFR, with correlations ranging from 0.21 for vegetables to 0.57 for cereals. Burrows et al. (2014) found subscale scores for the ARFS‐P were significantly positively correlated ranging from 0.27 to 0.51 (average correlation 0.35 for food groups) with dietary intake from an FFQ for vegetables, fruits, meat, and meat alternatives. The ARFS‐P, showed a positive relationship for recommended food items (e.g., fruits and vegetables) and a negative relationship for nonrecommended food items (e.g., sugar sweetened drinks and packaged snacks; Burrows et al., 2014).

In our study, a significant and inverse correlation was found for low fat foods/snacks/drinks consumption between the 4DFR and the DICE (r = −0.30), and three subscores (variety of fruits, low salt foods/snacks/drinks, and low sugar foods/snacks/drinks components) were not significantly correlated with the same component scores from 4DFR. This lack of agreement may be associated with some discrepancies in the scoring systems between DICE and the 4DFR results. For example, for low fat/sugar/salt foods/snacks/drinks, participants could either score 0 or 5 points based on whether they met Nutrient Reference Values (Ministry of Health, 2005) according to the 4DFR. This was compared with DICE responses ranging from “never to always” and a different scoring allocation of either 0, 2.5, and 5 points.

Furthermore, there is a chance of misreporting in DICE (e.g., the tendency to over‐estimate the frequency of consuming low salt and sugar foods/snacks/drinks). Another explanation for these findings may be due to the design of the related questions, meaning that caregivers could not estimate children's low salt and low sugar foods/snacks/drinks consumption accurately. There is a chance that caregivers are not aware which foods/snacks/drinks are low in salt and sugar. Moreover, the way that the guidelines are phrased suggests an assumption of a high intake of processed foods for all children. However, if our participants were not eating processed foods/snacks, choosing low fat/sugar/salt snacks would not be possible. This inconsistency highlights issues not only with the wording of the DICE, but also this section of the guidelines requires more explanation. It is also recommended that the related questions should clarify intake of processed and nonprocessed foods/snacks/drinks. For example, if no processed foods are consumed, choosing low salt foods/snacks may not be applicable for some participants.

4.3. Construct validity of DICE

When selecting a dietary assessment tool, construct validity should be considered. Construct validity can be assessed by observing the correlation between the index score and nutrient intakes derived from the reference method (Wong et al., 2013). A positive association was observed between higher DICE total scores and intakes of fibre, vitamin C, vitamin A, vitamin D, folate, and calcium. However, our results showed that DICE was unable to discriminate between macronutrients and this limitation needs to be recognised when using DICE.

Among studies that have developed indices for children and adolescents (Burrows et al., 2014; Cheng et al., 2016; Kyttala et al., 2014; Marshall et al., 2012; Wong et al., 2013), only a few have examined the construct validity of the index by dividing the total score into quartiles (Kyttala et al., 2014) or tertiles (Wong et al., 2013) to indicate upper and lower ranges of diet quality. As previous studies used different divisions (e.g., quartiles or tertiles), it is difficult to reasonably compare results with those of other studies. Kyttala et al. (2014) found a positive relationship between higher index scores and intake of more favourable nutrients (e.g., vitamin D and vitamin E) and an inverse correlation with less favourable nutrients (e.g., SFA and sugars). Wong et al., (2013) using tertiles, found a significant correlation between higher NZDQI‐A scores and higher intake of iron and lower consumption of total fat, SFAs, and monounsaturated fatty acids.

4.4. Reliability of DICE

Reliability was tested using ICC and a Bland–Altman plot between the first and the second administrations of DICE. The ICC is considered the most common and useful method for evaluating the reliability and determining the agreement between repeated questionnaires (Bountziouka & Panagiotakos, 2010). Although there was a short time interval (8 weeks) between the first and the second DICE administration, only 75% of our participants completed the DICE for the second time. The ICC for the DICE total score was r = 0.87, indicating that DICE is a reliable tool for assessing the diet quality of young NZ children. The Bland–Altman plot demonstrated good agreement between the two methods across the range of mean intakes from both the DICE administrations.

Only a few diet indices developed for children and adolescents have been examined for their reliability (Huybrechts et al., 2010; Wong et al., 2013). Differences in the design of previous studies such as the time interval between the first and the second administration, participants' age group, and sample size make it difficult to compare our reliability results with other studies. Huybrechts et al. (2010) found similar reliability of the “Diet Quality Index for Preschool Children” (r = 0.88). Their sample size was similar (n = 58) as was the age group (2.5–6.5 years old). However, they only had a 5‐week interval between the first and the second administration, compared with 8 weeks in our study. Dietary intakes and habits may change during childhood making it difficult to determine the ideal time interval for assessing reproducibility.

4.5. Methodological issues

Similar to previous studies (Waern et al., 2015; Wong, Parnell, Black, & Skidmore, 2012), we used the estimated food record rather than the weighed food record. Although it is more susceptible to errors (i.e., less than ideal), it is likely to have improved the response rate due to less participant burden (Waern et al., 2015).

It is suggested that it is ideal if the “test” and “reference” cover the same period of time (Wong et al., 2012). In our study, participants completed the DICE first and then the 4DFR, which meant that the assessment did not cover the same period of time. This order of administration was chosen to reduce any influence that the reference method (4DFR) may have on the test method (DICE) as recommended by Gibson (2005).

Our results were not adjusted for energy intake since the DICE design is based on the NZ MoH Food and Nutrition Guidelines and it is not able to measure total energy intake.

There were some discrepancies in the scoring system between the DICE and the 4DFR. For example, for low fat/sugar/salt foods/snacks/drinks, 4DFR data were categorised as binomial (0 or 5 points), although these same components were scored 0, 2.5, and 5 points in DICE, which may increase the risk of misclassification.

4.6. Strengths and limitations

One of the limitations of this study is that our results are not necessarily transferable to other populations. We recruited a convenience sample size of 65 participants. Other experts have suggested that sample sizes of at least 50 participants for validation studies could be enough (Cade, Burley, Warm, Thompson, & Margetts, 2004). Our participants were children under 8 years old whose caregivers completed the questionnaires on their behalf (proxy reporters), which can increase the chance of caregiver bias. Also, there is a chance of misreporting because caregivers may not have been with their children at all times (e.g., daycare), and this was not investigated. Some dietary indices for children have been adjusted for energy intake (Kyttala et al., 2014; Marshall et al., 2012) but not all (Burrows et al., 2014; Cheng et al., 2016; Huybrechts et al., 2010). An advantage of the DICE is its design as a simple and short diet index, which makes it convenient for health professionals, researchers, and caregivers to use when assessing diet quality or adherence to the NZ MoH Food and Nutrition Guidelines.

5. CONCLUSION

The current study evaluates a novel, stand‐alone index of children's diet quality, the DICE, in NZ. Results from this study demonstrated that DICE is a valid and reliable tool for the assessment of children's adherence to a healthy diet, as recommended in the NZ MoH Food and Nutrition Guidelines. Traditional methods such as the FFQ, 24‐hour food recall, or food records, which have been used in dietary surveys, demand considerable efforts from participants as well as researchers. In some situations, such as evaluating public health interventions, the overall diet quality is more important than the consumption of a single nutrient. Further research should be undertaken to refine the content and apply the DICE questionnaire to larger and different population groups.

CONFLICTS OF INTEREST

The authors declare that they have no competing interests.

CONTRIBUTIONS

CAC, KLB, PRvH, and OM were responsible for study design, data collection, acquisition of funding, and obtaining ethical approval. MD checked, entered and analysed the data and drafted the manuscript. KLB advised on the statistical analysis. All authors were involved in data interpretation, critical revisions of the paper, and read and approved the final manuscript. The study was funded by Massey University.

Delshad M, Beck KL, von Hurst PR, Mugridge O, Conlon CA. The validity and reliability of the Dietary Index for a Child's Eating in 2–8‐year old children living in New Zealand. Matern Child Nutr. 2019;15:e12758 10.1111/mcn.12758