Identification of measurement needs to prevent childhood obesity in high-risk populations and environments

Abstract

Introduction:

Children at highest risk for obesity include those from certain racial/ethnic groups, from low-income families, with disabilities, or living in high-risk communities. However, a 2013 review of the National Collaborative for Childhood Obesity Research (NCCOR) Measures Registry identified few measures focused on children at highest risk for obesity. Our objective was to 1) Identify individual and environmental measures of diet and physical activity added to the NCCOR Measures Registry since 2013 used among high-risk populations or settings, and 2) Describe methods for their development, adaptation, or validation.

Methods:

We screened references in the NCCOR Measures Registry from January 2013 to September 2017 (n=351) and abstracted information about individual and environmental measures developed for, adapted for, or applied to high-risk populations or settings, including: measure type, study population, adaptation and validation methods and psychometric properties.

Results:

Thirty-eight measures met inclusion criteria. Of these, 30 assessed individual dietary (n=25) and/or physical activity (n=13) behaviors, and 11 assessed the food (n=8) and/or physical activity (n=7) environment. Seventeen measures were developed for, 9 were applied to (i.e., developed in a general population and used without modification), and 12 were adapted (i.e., modified) for high-risk populations. Few measures were used in certain racial/ethnic minorities (i.e., American Indian/Alaska Native, Hawaiian/Pacific Islander, Asian), children with disabilities, and rural (vs. urban) communities.

Conclusions:

Since 2013, 38 measures were added to the Measures Registry that were used in high-risk populations. However, many of the previously identified gaps in population coverage remain. Rigorous, community-engaged methodologic research may help researchers better adapt and validate measures for high-risk populations.

Introduction

In the United States, 18.5% of children ages 2–19 have obesity.¹ Childhood obesity is a broad public health concern and leading health equity issue. Obesity prevalence is higher among Hispanic (25.8%) and non-Hispanic black (22.0%) than white (14.1%) and Asian (11.0%) children.¹ Obesity prevalence is also elevated among children living in families with low household income or where the head of household has low educational attainment.² Additionally, obesity prevalence is higher among children in rural than urban areas.³ While there is less rigorous information, data suggest children with intellectual, developmental, or physical disabilities have a 27–59% higher risk of obesity than those without.⁴

There is a need to develop, adapt, and validate measures children and families at high risk for obesity to accurately assess risk factors and evaluate interventions. Most available measures are developed for general or lower-risk populations and may require modification to be valid among high-risk populations.⁵ Populations at high risk for obesity may differ from lower-risk populations in important and interrelated ways, including historical, environmental, and social contexts, literacy level or spoken language, and cultural and psychosocial perspectives on diet, physical activity, and weight control.^6,7 Additionally, they may have differential access to obesity prevention and treatment interventions.

The National Collaborative on Childhood Obesity Research (NCCOR) Measures Registry was launched in 2011 to improve the quality of research related to dietary and physical activity behaviors and related environments, contribute to standardization across studies, and better inform policies and programs to promote the health of children.^8,9 The Registry is a searchable database of individual and environmental dietary and physical activity measures relevant to childhood obesity research.⁹ In 2013, an Institute of Medicine (IOM) report, “Evaluating Obesity Prevention Efforts: A Plan for Measuring Progress,” reviewed measures in the Registry used among high-risk populations.¹⁰ The report focused on environmental-level measures and identified 174 (of 893) measures used among high-risk populations, but a paucity specifically developed or adapted for high-risk populations.¹⁰

Our objective was to update and expand upon the previous IOM review by identifying and characterizing individual and environmental measures of diet and physical activity used among high-risk populations added to the NCCOR Measures Registry since 2013. To accomplish this goal, we: 1) identified individual and environmental measures of diet and physical activity used among high-risk populations or settings, and 2) abstracted information about their development, adaptation, or validation.

Methods

Definition of High-Risk Populations or Settings

Our definition for “high-risk populations” was similar to the 2013 IOM report¹⁰ and modified based on the input from the NCCOR workgroup. We defined high-risk populations as children (ages 0–18 years) and their families at high risk for obesity or residing in communities where the risk of obesity and related comorbidities may be highest. Factors related to high-risk individuals and communities include race/ethnicity, education/income, urbanicity, region of the country, and individuals with disabilities.

The Role of NCCOR

NCCOR is a partnership of the four leading funders of childhood obesity research: the Centers for Disease Control and Prevention, the National Institutes of Health, the Robert Wood Johnson Foundation, and the U.S. Department of Agriculture. All NCCOR projects are informed by a workgroup of staff at the four agencies. The workgroup for this project met monthly and provided input throughout.

The NCCOR Measures Registry

The NCCOR Measures Registry contains measures relevant to childhood obesity identified from literature searches of English-language articles using approximately 500 search terms. Additional details on the development of the Registry have been published previously.^8,11 The search is updated periodically, most recently to include articles published through September 2017. The Registry currently contains nearly 1,400 measures, organized in four domains: individual dietary behaviors, individual physical activity behaviors, food environment, or physical activity environment. Each measure’s entry contains information on how to use the measure and its validity and reliability. Examples of measures include questionnaires, logs, electronic devices, and methods for direct observation.

Search Strategy and Identification of Measures

We searched articles added to the NCCOR Measures Registry from January 2013 through September 2017 (n=351).⁹ We uploaded all articles into DistillerSR (Evidence Partners) for screening and data abstraction. Two trained investigators (KF, EK) independently performed title and abstract screening. Articles were included for full text review if 1) they reported measures of one or more domains in high-risk populations or settings, or 2) it was unclear and full text review was needed to determine eligibility. We excluded studies that did not include a high-risk population or setting. Additionally, we excluded studies that were: (a) not conducted among children or settings applicable to children (e.g. schools, home), (b) conducted outside the U.S. (it was unclear whether high-risk populations in other countries were generalizable to the U.S.), (c) not in English, (d) published prior to 2013, (e) not original research, or (f) the full text was not available (Figure 1).

Figure 1.

PRISMA Diagram.

Data Abstraction

With guidance from the NCCOR working group, we developed a data abstraction form (Appendix 1) in the DistillerSR database. The form included data elements contained in the NCCOR Measures Registry, the 2013 IOM report,¹⁰ and related to adaptation and validation methods. We abstracted several data elements included in the NCCOR Measures Registry: domain, measure type, study location, participant ages, race/ethnicity, and psychometric properties of the measure. We used similar categories as the IOM report¹⁰ to characterize sociocultural influences and socioeconomic status (SES) of the study population or setting; for example, we abstracted information about whether the study population was described by country of origin, language proficiency, level of education or income. We additionally abstracted whether studies included or focused on lesbian, gay, bisexual, transgender, and questioning (LGBTQ+) populations.

We abstracted information about whether the measure was developed for a high-risk population, applied to a high-risk population (i.e., developed in a general population and used without modification), or adapted (i.e., modified) for use among a high-risk population; to describe what aspects of the instrument were modified; and methods for adaptation and validation. When reported by the authors, we abstracted and summarized methodologic considerations for measurement among high-risk populations or settings. The codebook is available in Appendix 2.

For quality assurance, two trained reviewers (KF, CP) independently reviewed a random sample of articles and compared data abstraction. Additionally, the senior author (WB) reviewed 20% of the articles to assess completeness and accuracy.

Data Synthesis

We summarized the number of measures within each domain, i.e. individual dietary behaviors, individual physical activity behaviors, the food environment, and the physical activity environment. Measures which assessed multiple domains were counted in each relevant domain. We counted the number of measures used among high-risk populations of interest by domain. We summarized the types of measures in each domain.

We counted the number of measures by domain which were developed for, applied to, or adapted for high-risk populations; these categories were mutually exclusive. For measures adapted for high-risk populations, we summarized methods for adaptation and validation, and how the content was modified from the original instrument.

Results

Thirty-eight measures from the NCCOR Measures Registry met inclusion criteria (Appendix 3 and Appendix 4). Thirty measures assessed individual behaviors; 25 assessed individual dietary behaviors, and 13 assessed individual physical activity behaviors (8 assessed both). Eleven measures assessed environmental determinants of obesity; eight assessed the food environment and seven assessed the physical activity environment (4 assessed both) (Table 1).

Table 1.

Summary of the number of National Collaborative on Childhood Obesity Research Measures Registry tools, identified since 2013, targeting obesity prevention efforts for high-risk populations and settings.

	Individual Behavior Measures n=30 measures			Environmental Measures n=11 measures
High-risk population or setting	Dietary behavior (n=25)	Physical activity behavior (n=13)	# by sub-population	Food environment (n=8)	Physical activity environment (n=7)	# by setting
Racial/ethnic group	n (%)	n (%)	n (%)	n (%)	n (%)	n (%)
African American	16 (64%)	6 (46%)	18 (60%)	3 (38%)	3 (43%)	5 (45%)
American Indian/ Alaska Native	5 (20%)	1 (8%)	5 (17%)	2 (25%)	2 (29%)	2 (18%)
Hispanic	19 (76%)	9 (69%)	23 (77%)	4 (50%)	4 (57%)	6 (55%)
Hawaiian/Pacific Islander	0 (0%)	0 (0%)	0 (0%)	0 (0%)	0 (0%)	0 (0%)
Asian	6 (24%)	2 (15%)	6 (20%)	1 (13%)	1 (14%)	1 (9%)
White	11 (44%)	4 (31%)	12 (40%)	3 (38%)	3 (43%)	5 (45%)
Other	10 (40%)	5 (38%)	12 (40%)	4 (50%)	2 (29%)	5 (45%)
Not reported	1 (4%)	0 (0%)	1 (3%)	2 (25%)	2 (29%)	3 (27%)
Disability/special health care needs	1 (4%)	0 (0%)	1 (3%)	0 (0%)	0 (0%)	0 (0%)
Geographic location
Metro/Urban	16 (64%)	9 (69%)	21 (70%)	5 (63%)	5 (71%)	8 (73%)
Small Town/ Rural	4 (16%)	2 (15%)	4 (13%)	4 (50%)	4 (57%)	5 (45%)
Social influences
Low income/SES	15 (60%)	6 (46%)	17 (57%)	6 (75%)	5 (71%)	8 (73%)
Low education	6 (24%)	4 (31%)	8 (27%)	2 (25%)	1 (14%)	2 (18%)
Language proficiency	4 (16%)	3 (23%)	6 (20%)	1 (13%)	0 (0%)	1 (9%)
Acculturation	3 (12%)	2 (15%)	4 (13%)	0 (0%)	0 (0%)	0 (0%)
Foreign born	3 (12%)	2 (15%)	4 (13%)	1 (13%)	0 (0%)	0 (0%)
Living and working conditions	3 (12%)	2 (15%)	4 (13%)	2 (25%)	2 (29%)	2 (18%)
Racial/ethnic composition of community	6 (24%)	2 (15%)	6 (20%)	2 (25%)	4 (57%)	4 (36%)
Measure type
24-hour dietary recall	2 (8%)	N/A	2 (7%)	N/A	N/A	N/A
Food frequency questionnaire	4 (16%)	N/A	4 (13%)	N/A	N/A	N/A
Other questionnaire	16 (64%)	11 (85%)	19 (63%)	5 (63%)	3 (43%)	5 (45%)
Record or log	0 (0%)	1 (8%)	1 (3%)	0 (0%)	0 (0%)	0 (0%)
Electronic monitor	0 (0%)	1 (8%)	1 (3%)	0 (0%)	0 (0%)	0 (0%)
Interview	1 (4%)	0 (0%)	1 (3%)	0 (0%)	0 (0%)	0 (0%)
Behavioral observation	0 (0%)	0 (0%)	0 (0%)	N/A	N/A	N/A
Environmental observation	N/A	N/A	N/A	2 (25%)	3 (43%)	5 (45%)
GIS	N/A	N/A	N/A	1 (13%)	1 (14%)	1 (9%)
Other	2 (12%)	0 (0%)	2 (7%)	0 (0%)	0 (0%)	0 (0%)
Use in high-risk population
Applied to high-risk population	8 (32%)	3 (23%)	9 (30%)	1 (13%)	1 (14%)	2 (18%)
Developed for high-risk population	10 (40%)	6 (46%)	13 (43%)	4 (50%)	3 (43%)	5 (45%)
Adapted for high-risk population	7 (28%)	4 (31%)	8 (27%)	3 (38%)	3 (43%)	4 (36%)

Measures of individual behaviors

Individual behavior measures were most commonly used among Hispanic (n=23, 77%) and African American (n=18, 60%) populations. Six studies included Asian populations (20%), five (17%) included American Indian/Alaska Native populations, and none included Hawaiian/Pacific Islander populations. One measure was used amongchildren with autism or other special health care needs. There were no measures used among LGBTQ+ populations. Measures were more commonly used among populations living in metropolitan or urban areas (n=21, 70%) than small towns or rural areas (n=4, 13%). Seventeen measures assessed individual behaviors among low income or low SES populations (57%). Measures more commonly assessed behaviors among children ages 6–11 (n=16, 53%) and ages 2–5 (n=15, 50%), compared with those ages 12–18 (n=7, 23%) or ages <2 (n=2, 7%). Questionnaires were the most frequently identified measure type (n=19, 63%).

There were 13 measures (43%) developed for high-risk populations, nine (30%) applied to high-risk populations (i.e., developed in a general population and used without modification), and eight (27%) adapted (i.e., modified) for use among high-risk populations. Authors described several considerations for measuring individual dietary and physical activity behaviors among high-risk populations (Table 2). Cultural and linguistic adaptations were the most common forms of adaptation. Most often, researchers modified dietary measures to be more culturally appropriate (n=6). Focus group discussions, other qualitative, and mixed-methods approaches were often used for these purposes.

Table 2.

Considerations for Developing or Adapting Measures for High-Risk Populations as Described in Articles Included in the National Collaborative on Childhood Obesity Research Measures Registry.

Individual behavior measures
Measure type
Food frequency or other questionnaire	• Include culturally-relevant foods.12,23–25 • Availability of foods will vary by region. Consider local food availability and where food is sourced. For example, in one study conducted in Puerto Rico, most foods consumed were imported from the continental United States.26 • Level of acculturation and education may influence the difficulty in responding to a food frequency questionnaire.23 • Describe foods and beverages in ways that are familiar to certain cultural groups to help improve validity.12 • Consider cultural differences in perception of healthfulness of sugar sweetened beverages (e.g., sport drinks) and culturally-relevant sweetened drinks (eg, aguas frescas, which contain sugar, fruit, and water). Misconceptions have been reported among Hispanic youth.12 • Systematic biases (e.g., by personal characteristics such as body weight, social or cultural desirability, acculturation level, or literacy level) may influence the reporting of dietary intake, with a larger variance and reduced correlations with true intake.12,27
High-risk population
Acculturation status	• First-, second-, and third-generation immigrants may have different health beliefs and behaviors. The influence of acculturation may also vary by country of origin.23
Language proficiency	• Respondents who choose to complete a measure in English may systematically differ from those who choose to complete it in another language.13
Food insecure	• Capture the child’s perspective as children may experience food insecurity differently from their parents or caregivers. Measures of child and adult food insecurity may be more appropriate than a single adult or household measure.28
Environmental measures
Measure type
Environmental observation (home)	• Include culturally-relevant foods, particularly for racial/ethnic minorities and recent immigrants.14,24 • Timing of grocery shopping will affect the foods available in the home.14 • Include activities available for families with socioeconomic, racial, and ethnic diversity.24
Population or setting
Rural	• Many rural residents have Post Office Box mailing addresses. This will affect the validity of GIS measures of the food and physical activity environment based on participant address in rural settings.29 • Season and rurality may impact food availability. Certain fruits and vegetables may not be available in very rural areas, nor in specific climates.30
Community safety	• Safety may influence physical activity. In one study, American Indian children living on a reservation reported feeling unsafe when using their community bike path and a lack of resources to engage in physical activity. It is important to consider such barriers when developing measures and interventions.31

For example, one study adapted a validated adult beverage intake questionnaire, the BEVQ-15, for use among Hispanic preschool children ages 3 to 5 (BEVQ-PS).¹² Researchers conducted 20 semi-structured interviews with Hispanic mothers to identify relevant beverages from the original instrument, add suggested beverages, and adapt serving sizes for young children. The adapted instrument was piloted (n=5 mothers) and refined based on feedback on the questions, format, and mode of administration. In a validation/reliability study, 109 mothers completed the BEVQ-PS, which was compared with a 4-day food intake record. Test-retest reliability was assessed over a 6 to 9-day period. The authors found sugar-sweetened beverages, whole milk, and water met validity and reliability criteria, but modifications may be needed to accurately assess total beverage intake.¹²

In another study, researchers developed the Preschooler Physical Activity Parenting Practices (PPAP) instrument¹³ for use among Latino parents. The instrument was developed based on formative qualitative research using the Nominal Group Technique. Latino parents were asked what they do to encourage or discourage physical activity. Responses were ordered and grouped into parenting factors based on the literature. The instrument was translated into Spanish and back-translated into English. Conceptual and cultural, rather than linguistic equivalence was prioritized when there were differences between the original and back-translated survey. Researchers also conducted cognitive interviews with five English-speaking and five Spanish-speaking participants to refine survey items. While the PPAP showed moderate to excellent test-retest reliability and acceptable internal consistency, only certain subscales were significantly correlated with accelerometer-measured child physical activity. The authors noted for some subscales regarding parenting practices that discouraged physical activity, the Cronbach’s alpha was lower for the Spanish- than English-language version of the instrument. Spanish-speaking participants had lower levels of education, which may have confounded this observation.

Environmental measures

Of the 11 environmental-level measures, there were six (55%) used among Hispanic populations, five (45%) in African American populations, two in American Indian/Alaska Native populations, one in an Asian population, and none in Hawaiian/Pacific Islander populations. No environmental-level measures were used among children with disabilities or special health care needs. Measures were more commonly used in metropolitan or urban settings (n=8, 73%) than small towns or rural areas (n=5, 45%). Eight measures were used in low income or low SES populations or settings (73%). Environmental observation (n=5, 45%) and questionnaires (n=5, 45%) were the most common measure types; there was one GIS measure.

Five environmental measures (45%) were developed for high-risk settings, two (18%) were applied to high-risk settings (i.e., developed in general settings and used without modification) and four (36%) were adapted for high-risk settings (i.e., modified). One measure which was adapted was a home food inventory for low-income Spanish- and Somali-speaking families with pre-school aged children.¹⁴ Focus groups were conducted with five Spanish-speaking and five Somali-speaking individuals with English language skills to update an existing home food inventory. The updated inventory was translated into Spanish and Somali. The inventory was validated comparing responses of a trained staff member with those from 15 Spanish-speaking and 15 Somali-speaking parents. All validity indices were in an acceptable range, except for specific items such as “whole wheat bread,” possibly due to language or literacy barriers combined with poor understanding of nutrition labels among the general population. The authors concluded the tool is a valid measure among Spanish and Somali households and should be validated in other populations.

The Texas Childhood Obesity Prevention Policy Evaluation School Environmental Audit Tool was developed to assess the safety and walkability of school environments.¹⁵ The tool, developed from a conceptual framework, includes street, school site, and map audits. It was pre-tested in one urban, suburban, and rural elementary school, refined, and tested again. Two trained auditors then visited 12 elementary schools (four urban, four suburban, and four rural, including two high- and two low-income schools in each area) to assess interrater, test-retest, and peak versus off-peak hour reliability. Test-retest and peak versus off-peak reliability were highest among rural schools. Interrater reliability was highest at urban schools and lowest at rural schools for perceptual qualities (eg, safety, attractiveness), likely due to heterogeneity in rural environments; interrater reliability for objective items was excellent for all settings. The authors concluded, with proper training to reduce interrater differences, this tool can assess school environments reliably across settings for surveillance, research, and policy evaluation.

Discussion

Since the 2013 IOM review of the NCCOR Measures Registry, an increasing number of measures are being developed or adapted for high-risk populations. However, measurement gaps for specific populations and settings were similar to those identified in the previous report. While a large proportion of measures identified in our review were used among African American and Hispanic individuals and communities, fewer were used among Asians, American Indians/Alaska Natives, or Hawaiian/Pacific Islanders, despite the high prevalence of overweight and obesity among American Indian and Hawaiian/Pacific Islander children¹⁶ and substantial, understudied heterogeneity by Asian ethnicity.^17,18 While country of origin and acculturation may influence knowledge, beliefs, and behaviors related to diet and physical activity, these factors are rarely assessed; additional tailoring of measures may be needed. As in the 2013 report, we identified a need for measures for children with disabilities and special health care needs as we found only one relevant measure. Additionally, there is a critical gap in measures used among LGBTQ+ populations, as none were identified in the 2013 report or our study. More measures were applied in urban than rural settings, similar to the previous report. Few studies reported sufficient detail to precisely classify geographic areas. The choice of rural/urban classification scheme may affect the validity of certain measures.¹⁹

We found most measures used in high-risk populations were questionnaires. While there are advantages with respect to participant burden and ease of administration, questionnaires have limitations such as the potential for recall bias or reporting errors. Additionally, language or literacy barriers among certain subgroups could further affect validity. The type of instrument and mode of administration may be particularly important considerations when engaging high-risk populations.

There are several challenges related to measurement in high-risk populations to be addressed. First is to promote the use of best practices for adaptation and standard validation procedures.²⁰ Few studies reported details about adaptation methods used and the quality of such studies varied. Further, a number of studies did not discuss differences in the validity of subscales within a measure or differences across populations (if applicable), nor the limitations on the context in which the measure could provide useful information.²¹ We see a need for rigorous methodologic research and to increase dissemination of adaptation and validation studies, which may not necessarily be published in the literature. A second challenge is to balance the tension between tailoring measures for specific groups and using standardized measures to facilitate comparison across populations.²¹ Researchers will need to consider the tradeoffs and select measures appropriate for the purpose of their work. Third is intersectionality. Disparities in childhood obesity are rarely explained by a single factor.²² Characteristics used to define “high-risk” populations often co-occur and interact. Researchers will need to consider how their intersection may influence measurement and the implications for practice and policy responses. A final challenge is to increase community engagement. Few studies described how community members were engaged in measure development or adaptation, which may ultimately affect validity. Community members’ perspectives are critical to measure what matters and understand how to measure it.²⁰ Researchers conducting community-engaged studies can advance the field by documenting and sharing best practices and lessons learned.

Our study has several limitations. First, we focused on measures included in the NCCOR Measures Registry and may have missed other studies using these measures that were not captured. However, the NCCOR search strategy is rigorous.¹¹ Additionally, individual behavior measures included in the NCCOR Measures Registry are required to be previously validated; there may be other tools used in practice which are not included in the registry. Second, we relied on study descriptions about setting and urbanicity/rurality. Third, we relied on the authors’ descriptions of adaptation and validation methods, and considerations for measures used among specific high-risk groups, which were often only briefly reported.

Recognizing limited progress, NCCOR has taken steps to identify measurement priorities to address gaps related to children in high-risk populations or settings. In September 2019, NCCOR held a two-day workshop, titled “Advancing measurement for high-risk populations and communities related to childhood obesity” with a goal of illustrating current challenges, discussing best practices to adapt and develop measures, and developing recommendations to address gaps in the field. The workshop convened over 20 experts in measurement of high-risk populations. Recommendations from the workshop will be shared on the NCCOR website, www.nccor.org/measurement-workshop-series/.

To reduce disparities in childhood obesity, it is necessary to measure individual behaviors, and environmental factors in the socioeconomic and sociocultural contexts in which they occur. This report provides an overview of the current state of measures available for use in high-risk populations. Though there has been an increase in measures used among high-risk populations since 2013, there are certain populations and settings for which major gaps remain. It is also important to understand whether methodological choices related to development and adaptation of measures for high-risk populations achieves the goal of accurately measuring constructs of interest. These issues may be especially salient among high-risk populations and in disadvantaged neighborhoods, as well as in rural areas. Addressing gaps in the availability of validated tools and measures and improving the quality of measurement can help practitioners understand and address risk factors for obesity among high-risk children and their families.

APPENDIX

Appendix Material

1. Data abstraction form

2. Codebook and guidelines for completion of abstraction form

3. Summary of studies which include individual dietary and physical activity behaviors and environmental factors related to childhood obesity among high-risk populations

4. Excluded studies