Psychometric testing of the healthy eating and physical activity self‐efficacy questionnaire and the healthy eating and physical activity behavior recall questionnaire for children

Abstract

Purpose

The purpose of this study was to develop and assess the psychometric properties for two related questionnaires: the Healthy Eating and Physical Activity Self‐Efficacy Questionnaire for Children (HEPASEQ‐C) and the Healthy Eating and Physical Activity Behavior Recall Questionnaire for Children (HEPABRQ‐C).

Design and Methods

HEPASEQ‐C and HEPABRQ‐C were administered to 517 participating children with 492 completing. Data were analyzed to evaluate for reliability and validity of the questionnaires.

Results

Content validity was established through a 10‐person expert panel. For the HEPASEQ‐C, item content validity index (CVI) ranged from 0.80 to 1.00. The CVI for the total questionnaire was 1.0. All HEPASEQ‐C items loaded on a single factor. Cronbach's alpha was deemed acceptable (.749). For the HEPABRQ‐C, item CVI ranged from 0.88 to 1.00. CVI for the total questionnaire was 1.0. Pearson product moment correlation between HEPASEQ‐C and HEPABRQ‐C scores was significant (r = .501, p = .000).

Practice Implications

The HEPASEQ‐C and HEPABRQ‐C are easily administered and provide helpful insights into children's self‐efficacy and behavior recall. They are easy to use and applicable for upper elementary school settings, in clinical settings for individual patients, and in health promotion settings.

1. INTRODUCTION

Recently a major focus of school‐based health promotion efforts has been on preventing childhood obesity. This focus is justified given that 17% of children between the ages 2 and 19 in the United States are obese (Centers for Disease Control and Prevention, 2017), and physical activity and dietary patterns are imprinted during their school‐age years (Ickes, McMullen, Haider, & Sharma, 2014). However, knowledge of healthy lifestyles, on its own, does not readily translate into healthy behavior patterns. Self‐efficacy is a psychosocial determinant of health, which helps translate knowledge into healthy behavior patterns (Bandelli, Lee Gray, Paul, Contento, & Koch, 2016; Elmore, & Sharma, 2014). Self‐efficacy is the confidence or belief in one's own ability to engage in a desired behavior and overcome barriers to that behavior (Bandura, 1997). Consistent with Bandura's (1998) social cognitive theory, some school‐based interventions aim to improve children's knowledge as well as their self‐efficacy related to healthy eating and physical activity. By doing so, they anticipate improved patterns of healthy behaviors (Bandelli et al., 2016; Elmore & Sharma, 2014). Researchers hypothesize that improved behaviors ultimately will translate into a reduced prevalence of childhood obesity.

Bandura's (1998) social cognitive theory provided the framework for our school‐based intervention. We hoped to improve children's self‐efficacy though hands‐on learning activities, mastery experiences, and social modeling. However, we faced a hurdle when we were unable to locate a tool designed with and for children to measure their self‐efficacy related to healthy eating and physical activity. We also wanted an assessment that could be completed in a short amount of time, preferably in about 10 min to minimize participant burden.

We were keenly aware of the National Institutes of Health's initiative, No More Hand‐Me‐Down Research, which primarily focuses on improving the clinical care of children by developing tools, treatments, medications, and devices specific to children through engagement of children in research (National Institutes of Health, 1998; 2015; National Institutes of Health, National Heart, Lung, and Blood Institute, 2015). With this awareness, we devised a research plan to directly engage children in the development of a self‐efficacy questionnaire and a corresponding behavior recall questionnaire. Through five focus groups with different children, we developed and refined the Healthy Eating and Physical Activity Self‐Efficacy Questionnaire for Children (HEPASEQ‐C) and the corresponding Healthy Eating and Physical Activity Behavior Recall Questionnaire for Children (HEPABRQ‐C). The purpose of this study was to further develop and assess the psychometric properties of the HEPASEQ‐C and HEPABRQ‐C.

2. METHODS

2.1. Procedure

2.1.1. Ethical considerations

Our university's institutional review board approved this study. To protect participants’ identities, we did not collect identifying information. Participants were not compensated for their time.

2.1.2. Recruitment, consent, and assent

Any child in fourth, fifth, or sixth‐grade participating in a 7‐week, school‐based intervention called “Anatomy Academy” was eligible to participate in this study. Children in grades four through six have had basic education about what constitutes healthy eating and activity; thus, we did not provide a definition of “healthy” foods in the questionnaires. Anatomy Academy was designed to add to children's basic knowledge of how anatomy and physiology principles relate to their patterns of eating and physical activity along with how to establish healthy patterns of eating and physical activity. Several days prior to the beginning of Anatomy Academy, a packet of information about the study was sent home. The packet included a cover letter with the research team's contact information and an explanation of the study, two copies of the “Parental Permission for a Minor,” and two copies of the “Child Assent.” Children who returned a completed and signed set of consent and assent forms were eligible to participate. Completed forms were securely stored in a research team member's office.

Expert opinions vary over the sample size needed for factor analysis studies (Pett, Lackey, & Sullivan, 2003). A minimum recommended number is generally 10 participants per initial instrument item with a larger number of participants considered more desirable (Pett et al., 2003). The large number of participants who completed the questionnaires is a strength of this study.

2.1.3. Data collection

The two questionnaires with clearly written instructions were administered electronically through Qualtrics (2013). Each child was assigned a code using a random number generator. On the first day of Anatomy Academy, participants entered their code numbers and then most completed the questionnaires in 4–12 min. Research assistants were on hand to provide instruction and help with technical difficulties.

2.1.4. Setting

Data were collected in computer laboratories and classrooms from 30 cohorts of fourth, fifth, and sixth‐graders in 12 schools in California and Utah. The schools represented a diversity of ethnicities and socioeconomic backgrounds, including a charter school primarily for Pacific Islander and Hispanic children and Title‐1 schools, which receive federal funding for supplemental services due to their students’/neighborhoods’ high levels of poverty (Salt Lake City School District, 2017).

2.1.5. Sample description

Descriptive statistics for demographic variables are reported in Table 1. Five hundred seventeen children participated, and 492 of them completed data collection. The average age for the 517 participating children was 10.6 (SD = 0.86) years. About half of them reported being White 256 (49.5%).

Table 1.

Demographic characteristics

	Characteristic	N (%)
Age	Mean (SD)	10.6 (0.86)
	9 years	54 (10.4)
	10 years	129 (25.0)
	11 years	226 (43.7)
	12 years	59 (11.4)
	13 years	2 (0.4)
	Missing	47 (9.1)
	Total	517
Race
	White	256 (49.5)
	Latino	84 (16.2)
	Native American	24 (4.6)
	Pacific Islander	22 (4.3)
	Asian	10 (1.9)
	African American	7 (1.4)
	Other	51 (9.9)
	Missing	63 (12.2)
	Total	517

2.2. Questionnaires

2.2.1. Item development

The earliest version of our items were adapted with permission from Decker's (2012) questionnaire evaluating parents’ self‐efficacy related to enacting healthy eating and activity behaviors in their children. Decker's (2012) original 35 items were based on USDA guidelines for healthy eating and physical activity. After simplifying the original language to ensure it was appropriate for school age children, five focus groups were conducted with children, as our future “end users,” to further refine the items and establish face validity and understandability, as well as provide additional input and clarification. Based on the children's input, we changed the item format from questions to statements because the children indicated statements were easier to understand (Lassetter, Ray, Driessnack, & Williams, 2015).

Then collaborating with an expert in the development of research questionnaires for children (author #4), we recognized a need to reduce the number of items from Decker's (2012) original 35 as well as change the response options from a 10‐point Likert‐type scale to three options in the HEPASEQ‐C. Although we reduced the number of items from 35 to 19, the original content domains were retained. While the reduction in response options diminished the ability to discriminate, having three response options was age appropriate and easier for the children to understand. The final versions of the questionnaires had Flesch–Kincaid grade levels of 1.8 for the HEPASEQ‐C and 4.0 for the HEPABRQ‐C (Lassetter et al., 2015).

Theoretically, it was important to assess whether the participants’ self‐efficacy was correlated with the children's self‐reported behaviors. Originally the self‐efficacy and behavior recall items were together in one questionnaire; however, the children told us it was confusing to have the self‐efficacy and behavior recall items together. They suggested separating them to make them easier for children to follow and complete (Lassetter et al., 2015). Finally, we worried that children might respond to the behavior questions the way they thought we wanted them to rather than with an honest recall of their behavior. When we asked our focus groups about this concern, the children's answer was simply to ask participants right upfront to “Please tell the truth.” Then, they told us, children will be honest (Lassetter et al., 2015).

2.2.2. Healthy eating and physical activity self‐efficacy questionnaire for children

HEPASEQ‐C consists of nine items with response options on a 3‐point Likert‐type scale (1 = There is no way I can do this, 2 = This could be hard for me, 3 = I believe I can do this). Seven items focus on self‐efficacy related to healthy eating. For example, one item says, “I will eat healthy food even when my friends eat food that is not healthy.” See Figure 1 for the complete HEPASEQ‐C.

Figure 1.

Healthy eating and physical activity self‐efficacy questionnaire for children

2.2.3. Healthy eating and physical activity behavior recall questionnaire for children

The HEPABRQ‐C consists of 10 items, two of which are in an open‐response format allowing children to write the actual foods they ate. For example, one of these two items states, “The last time I ate a snack at a friend's house, I ate __________.” The other eight items include two yes/no items, and the remaining six items had response options that were presented using an ordinal scale. See Figure 2 for the complete HEPABRQ‐C.

Figure 2.

Healthy eating and physical activity behavior recall questionnaire for children

2.3. Data analysis

Data were entered into Qualtrics (2013) online survey software and then exported to SPSS version 22 for analysis (IBM Corporation, 2013). Accuracy of the data was assessed by checking for outliers and anomalous values using appropriate descriptive statistics and graphs before further analysis.

Content validity assessment was based on Lynn's (1986) process. Our panel of 10 content experts included four dieticians who teach at a university, three dieticians who practice at a large children's hospital, and three pediatric nurse educators who teach at a university. Their educational level included five experts with Ph.D.s and five master's‐prepared experts. Their years of experience ranged from 9 to 37 years (M = 17 years), and years in their current role ranged from 3 to 19 years (M = 7 years). The experts rated all items in both questionnaires using a 4‐point rating scale (4 = very relevant and succinct, 3 = relevant but needs minor revision, 2 = unable to assess relevance, 1 = not relevant). They were asked to provide suggestions for revision and to identify any areas omitted from the questionnaire that should be included.

The index of content validity (CVI) was determined for each item as well as both questionnaires. The CVI for each item was determined by the proportion of experts who rated it as content valid (a rating of 3 or 4), and the CVI for the entire questionnaire is the proportion of total items judged content valid. According to Lynn (1986), the minimal acceptable CVI for items is based on the number of experts. In this study, 10 experts were used. Lynn (1986) indicates that 8 of the 10 experts’ endorsements are required to establish content validity beyond the .05 level of significance.

Construct validity of the HEPASEQ‐C was assessed by exploratory factor analysis using steps described by Pett and colleagues (2003). This was appropriate because attitudes and beliefs, such as those associated with self‐efficacy, conform to the theoretical underpinnings of exploratory factor analysis (Bollen & Bauldry, 2011). Reliability of the HEPASEQ‐C was assessed by calculating Cronbach's alpha.

The HEPABRQ‐C was assessed by concurrent validity. HEPABRQ‐C includes clinically relevant behavior patterns that may not be closely related. In such cases, exploratory factor analysis and internal consistency are inappropriate (Bollen & Bauldry, 2011; Fayers & Hand, 1997, 2002; Juniper, Guyatt, Streiner, & King, 1997; Streiner, 2003a, 2003b). Thus, neither exploratory factor analysis nor calculation of Cronbach's alpha was done for the HEPABRQ‐C.

3. RESULTS

Five hundred seventeen children started the electronic questionnaires, and 492 (95.2%) responded to all of the questions. Over half (53.4%) of the participants completed the questionnaires in 4 min or less, with 95% of participants completing them in 12 min or less. About half of the participating children reported being white, and the other half reported diverse racial backgrounds.

3.1. Content validity for HEPASEQ‐C and HEPABRQ‐C

For the HEPASEQ‐C, the item CVI ranged from 0.80 to 1.00, indicating all items were judged content valid. The CVI for the total questionnaire was 1.00. For the HEPABRQ‐C, item CVI ranged from 0.88 to 1.00. The CVI for the total questionnaire was 1.0. Comments of the experts were reviewed for both questionnaires, and no changes on the items were needed.

3.2. Construct validity and reliability for HEPASEQ‐C

All self‐efficacy item means and standard deviations were at the upper range of the scale, indicating greater self‐efficacy (Table 2). Interitem correlations were reviewed to assess for extremely high or low values. No correlations between items were deemed too high (> 0.8); however, correlations between item six (about dairy consumption) and other items were somewhat low (≤0.3), indicating potential issues. Bartlett's Test of Sphericity was significant (χ² = 735.6, df = 36, p = .000), indicating the correlation was not an identity matrix and, accordingly, was able to be factored. The Kaiser–Meyer–Olkin test, which tests overall sampling adequacy, was deemed adequate (KMO = .809) to proceed with factor analysis (Pett et al., 2003). Individual measures of sampling adequacy (MSAs), reported in Table 2, ranged from .76 to .84 also indicating adequacy to proceed.

Table 2.

Self‐efficacy item descriptive statistics

Item	Mean (SD)	MSAa
1. I will say no when my friends offer me junk food or food that is not healthy.	2.4 (.68)	.81
2. I will eat healthy food even when my friends eat food that is not healthy.	2.5 (.62)	.83
3. I will eat healthy foods when I eat out with my family.	2.5 (.66)	.79
4. I will eat at least 4 servings of vegetables every day.	2.4 (.70)	.84
5. I will eat at least 3 servings of fruits every day.	2.8 (.49)	.82
6. I will eat or drink 3 servings of milk, cheese, yogurt, or cottage cheese every day.	2.7 (.59)	.80
7. I will not drink soda pop more than one time a week.	2.5 (.71)	.83
8. I will be physically active even when my friends choose to sit still and hang out.	2.6 (.61)	.76
9. I will be physically active for 60 minutes every day.	2.7 (.58)	.79

SD, standard deviation.

^aIndividual measures of sampling adequacy.

Principal Axis Factoring (PAF) was used for factor extraction because it uses shared variance and emphasizes the latent construct over total variance (Pett et al., 2003). Initial factor extraction using an eigenvalue of greater than 1 as a cutoff indicated a two‐factor solution. Different rotations were then examined to clarify factor loadings and theoretical utility. Both orthogonal and oblique factor rotations were attempted; however, oblique rotation solutions resulted in high correlations (>0.3) between factors and did little to help items load strongly on a single factor. Pett and colleagues (2003) noted, “Factors with correlations that are too high should be rejected because they are undoubtedly measuring the same construct” (p. 164). A quartermax rotation was then performed to assess factor loadings with orthogonal rotation, with the assumption a general factor may be present, as evidenced by high factor correlations with oblique rotations. Item loadings under the two‐factor solution are reported in Table 3. All items loaded the highest on factor 1, so factor loadings were then recalculated using a single factor solution. All items, except item six, loaded above a cutoff value (.4). While item six loaded highest on factor 1, the loading was still considered very low (.305). Due to low factor loadings, we considered dropping item six from the questionnaire; however, we opted to retain the item in interest of theoretical completeness. Final factor loadings for the one‐factor solution with PAF are also reported in Table 3.

Table 3.

Factor loadings with principal axis factoring and quartermax rotation

	Two factor solution		One factor solution
Item	1	2	1
1	.564*	.403	.568
2	.542*	.248	.560
3	.503*	–.007	.506
4	.557*	.052	.570
5	.482*	–.010	.487
6	.305*	–.088	.297
7	.486*	.220	.503
8	.573*	–.301	.512
9	.526*	–.188	.496

^*Highest loading factor.

Internal consistency, or reliability, was assessed for HEPASEQ‐C. Cronbach's alpha was deemed acceptable with a value of .749. Item‐total correlations then were reviewed. All items, except item six, were moderately correlated with the total (r = .422—.481). As expected, item six demonstrated a low item‐total correlation (r = .257). Retaining item six did not significantly lower the Cronbach's alpha, so the final decision was made to retain item six as an important component of healthy eating. A summary score (M = 23.1, SD = 3.3) for the questionnaire was calculated by summing the nine individual items. Total scores ranged from a low of 9 to a high of 27 with the distribution for the overall score negatively skewed (more students reporting higher self‐efficacy).

3.3. Concurrent validity for HEPABRQ‐C

Descriptive statistics for the quantitative recall items are reported in Table 4. Response format for these items vary, with open‐response format for items two and four, allowing researchers and clinicians to assess children's understanding of healthy eating in multiple ways. The summary score was calculated by summing the seven healthy eating and activity items and the reverse‐coded “soda pop” item (#9). The summary score (M = 12.7, SD = 4.3) ranged from 1 to 21. Recall questionnaire items corresponded with self‐efficacy questionnaire items and allowed for comparison between self‐efficacy and self‐reported behaviors. The Pearson product moment correlation between self‐efficacy and recall was significant (r = .501, p = .000) with children with higher self‐efficacy scores reporting more healthy behaviors. The nonparametric Spearman's rho yielded a similar result (ρ = .453, p = .000).

Table 4.

Recall item descriptive statistics

Item	Response format	Mean (SD)
1. The last time I ate a snack at a friend's house, it was a healthy snack.	0 = “no”, 1 = “yes”	.55 (.50)
2. The last time I ate a snack at a friend's house, I ate __________.	Open ended	N/A
3. The last time I ate out with my family, I chose a healthy option.	0 = “no,” 1 = “yes”	.52 (.50)
4. The last time I ate out with my family, I ate __________.	Open ended	N/A
5. Yesterday I ate __ servings of vegetables.	0 = “0” 1 = “1,” 2 = “2,” 3 = “3,” 4 = “4 or more”	1.9 (1.3)
6. Yesterday I ate ___ colors of vegetables.	0 = “0,”, 1 = “1,” 2 = “2,” 3 = “3 or more”	1.6 (1.0)
7. Yesterday I ate __ servings of fruit.	0 = “0,” 1 = “1,” 2 = “2,” 3 = “3 or more”	1.8 (1.0)
8. Yesterday I ate or drank ___ servings of milk, cheese, yogurt, or cottage cheese.	0 = “0,” 1 = “1,” 2 = “2,” 3 = “3 or more”	1.7 (.96)
9. In the last 3 days I drank ___ soda pop.	0 = “0,” 1 = “1,” 2 = “2,” 3 = “3 or more” (reverse code for scoring)	2.2 (1.0)
10. Yesterday I was physically active for ___ minutes.	0 = “0,” 1 = “15,” 2 = “30,” 3 = “60 or more”	2.3 (.91)
Total	Minimum = 0, maximum = 21	12.7 (4.3)

SD, standard deviation.

4. DISCUSSION

Development of the HEPASEQ‐C and HEPABRQ‐C fill an important gap in child health and behavior research by providing researchers and clinicians with tools that demonstrate acceptable face, content, construct, and concurrent validity, as well as acceptable internal consistency. The sequential steps taken in developing the HEPASEQ‐C and HEPABRQ‐C were essential for establishing reliable and valid questionnaires for children. Perhaps most important was to involve children during questionnaire development and testing. Failure to do this may result in a tool that is inappropriate to use with children due to the potential for them to misunderstand the concepts, words, instructions, and/or format, which will then produce flawed results.

Validity is a key concern when considering questionnaires for use in health care and healthcare research (American Educational Research Association [AERA], American Psychological Association [APA], & National Council on Measurement in Education [NCME], 2014; Streiner & Norman, 2003). One aspect of validity is the adequacy of the test content in representing the domain of interest (AERA et al., 2014). Additionally, our sample is racially diverse, strengthening its reliability and validity in other populations. Content validity for both questionnaires was assessed and deemed adequate by content experts. Other validity evaluations performed in this study were driven by the theoretical underpinnings of our two questionnaires.

When developing a tool to measure a psychometric construct, such as self‐efficacy, it is important to report the theoretical underpinning for selection of various methods to assess reliability and validity. For example, when establishing validity from a proposed unified construct, it is important to assess the internal structure and relationships between items proposed to measure that construct (AERA et al., 2014). Exploratory factor analysis was used to examine construct validity of HEPASEQ‐C. Bollen and Bauldry (2011) suggest that attitudes and beliefs, such as those associated with the construct of self‐efficacy, tend to conform to theoretical underpinnings of exploratory factor analysis. The HEPASEQ‐C items were found to belong to a single factor and demonstrated internal consistency above 0.7, which is considered acceptable.

Exploratory factor analysis and metrics, like internal consistency, may not be appropriate or necessary for evaluating all collections of measurement items, particularly when items may not be closely related but are considered clinically useful (Bollen & Bauldry, 2011; Fayers & Hand, 1997, 2002; Juniper et al., 1997; Streiner, 2003a, 2003b). Such is the case with HEPABRQ‐C, which includes clinically relevant behavior patterns.

Therefore, the validity of the scores derived from the HEPABRQ‐C was not evaluated using exploratory factor analysis and internal consistency; however, other validity assessments were performed. Content validity scores from HEPABRQ‐C were assessed and determined to be good. Validity evidence for such test scores may also be provided by examining the relationships of test scores with other variables (AERA et al., 2014). Convergent evidence examines the relationship between test scores and scores from other variables intended to measure similar constructs (AERA et al., 2014). We examined the HEPABRQ‐C score evaluating healthy behaviors and the HEPASEQ‐C score for healthy eating and activity self‐efficacy and found a positive relationship that was also statistically significant, providing convergent evidence between self‐efficacy and behavior.

Although HEPASEQ‐C and HEPABRQ‐C show acceptable reliability and validity, additional psychometric testing is needed. Cross validation studies are needed representing more racial and ethnic groups, varying socioeconomic levels, and related educational factors. For example, using the questionnaires in a school setting with a significant portion African American children would be helpful in further assessing reliability and validity. Further investigation is needed to determine the relationship of healthy eating and physical activity self‐efficacy and behavior recall. Additionally, longitudinal studies should be conducted to determine the predictive validity of the two questionnaires in determining the rates of obesity. Finally, controlled intervention studies should be conducted to assess the questionnaires’ sensitivity to change in self‐efficacy and behavior recall based on health promotion programs focusing on healthy eating and physical activity. Many children had high self‐efficacy scores. This may indicate that many children feel capable of making good eating choices. However, it leaves little room to measure improvements in self‐efficacy. During instrument development, we reduced the response options from 10 to 3 on the HEPASEQ‐C, which limits the potential variability as well as sensitivity to change; however, this compromise was intentional to tailor this instrument for usability with children. Ceiling effects may be a limitation of the current version of the instrument.

5. HOW MIGHT THIS INFORMATION AFFECT NURSING PRACTICE?

Nearly one in five children in the United States is obese (Centers for Disease Control and Prevention, 2017), causing some to refer to childhood obesity as an epidemic. While interventions aimed at reversing this unhealthy trend can be developed, they are not able to be refined and/or evaluated without reliable and valid questionnaires to assess children's self‐efficacy and/or recall of healthy eating and physical activity. This study shares the initial development and psychometric evaluation of two questionnaires that measure these concepts. Children's input during questionnaire development helped establish reliability and validity by assuring wording and formatting were understandable to children. Our experience reiterates the need to involve children in developing questionnaires, rather than simply adapting questionnaires developed with and for adults. The HEPASEQ‐C and HEPABRQ‐C are easily administered and provide helpful insights into children's self‐efficacy and behavior recall. They are easy to use and applicable for upper elementary school settings, in clinical settings for individual patients, and in health promotion settings.

CONFLICT OF INTEREST

The authors report no actual or potential conflicts of interests.

Lassetter JH, Macintosh CI, Williams M, Driessnack M, Ray G, Wisco JJ. Psychometric testing of the healthy eating and physical activity self‐efficacy questionnaire and the healthy eating and physical activity behavior recall questionnaire for children. J Spec Pediatr Nurs. 2018;23:e12207 https://doi.org/10.1111/jspn.12207