Validation of environmental content in the Young Children's Participation and Environment Measure

Abstract

Objective

Evaluate the concurrent validity of the environment content in the newly developed Young Children's Participation and Environment Measure (YC-PEM).

Design

Cross-sectional study.

Setting

Data were collected online.

Participants

Convenience and snowball sampling methods were used to survey caregivers of 381 children (85 children with developmental disabilities and delays, 296 without developmental disabilities and delays) between 0-5 years (mean = 36.49 months, SD = 20.18) and residing in North America.

Interventions

Not applicable.

Main Outcome Measure(s)

The YC-PEM includes an assessment of participation and environmental impact on children's participation for three settings: home, daycare/preschool, and community. Pearson and Spearman correlational analyses were used to examine the concurrent validity of YC-PEM environmental content according to a criterion measure, the Craig Hospital Inventory of Environmental Factors – Child and Parent Version (CHIEF-CP). YC-PEM and CHIEF-CP items were first mapped to the International Classification of Functioning, Disability and Health – Children and Youth Version (ICF-CY) to identify items for pairwise comparison.

Results

We found small to moderate negative associations for 51 out of 66 pairwise comparisons involving CHIEF-CP and YC-PEM environment items (r = -0.13 to -0.39, p < 0.01). Significant associations were found for items in all 5 ICF-CY environmental domains.

Conclusion(s)

Results lend further support for use of the YC-PEM for valid caregiver assessment of the physical, social, attitudinal, and institutional features of environments in terms of their impact on young children's participation within the home, daycare/preschool, and community settings.

Introduction

Participation in activities is important to early childhood health and development.¹ Participation difficulties for young children with developmental disabilities and delays are related to the child's abilities, family factors, and environmental influences.² Instruments are needed to advance research on the specific role that young children's environments play in the presence of child and family factors to optimize decision-making about resource allocation and intervention design towards improving participation outcomes.³

The Young Children's Participation and Environment Measure (YC-PEM)⁴ assesses for participation in home, daycare/preschool, and community activities and environmental qualities that impact participation. Similar to the Craig Hospital Inventory of Environmental Factors for Children – Parent Version (CHIEF-CP), which was adapted from the CHIEF to be suitable for use with children with and without disabilities between 2 and 12 years⁵, the YC-PEM assesses for a broad range of environmental influences (e.g., physical layout, activity demands, attitudes, policies). However, the combined format in the YC-PEM may help to clarify the impact of environmental factors, compared to child and family factors, on children's participation in specific settings.

The YC-PEM underwent field testing to examine its psychometric properties prior to use in large sample pediatric rehabilitation research.⁶ In this paper, we establish the concurrent validity of YC-PEM environmental content to provide further evidence of its research utility. Significant negative item-pair associations are expected when comparing environmental content from the YC-PEM and CHIEF-CP because the CHIEF-CP assesses for environmental barriers and the YC-PEM captures the extent to which the child's environment is perceived to support participation.⁷

Methods

Participants

Using a cross-sectional design, the YC-PEM was field-tested online with 395 caregivers of young children in a three-part study (June-October 2013). Recruitment and sampling methods have been described elsewhere.⁶ This study involves secondary data analysis of Part 1 data collection which included participants completing the demographic questionnaire, YC-PEM, and CHIEF-CP online. Participants 1) could read and write in English; 2) resided in the US or Canada; 3) identified as a parent or legal guardian who is 18 years or older; 4) had a child between 0-5 years old; and 5) had internet access.

Measures

Three questionnaires were administered to participants.

Demographic Questionnaire

Caregivers reported on 1) family factors (education), 2) household factors (income), 3) child factors (age, gender), and 4) their child's functioning in 12 areas related to participation² (no problem [0] vs. little/big problem [1]).

Young Children's Participation and Environment Measure (YC-PEM)⁴

Caregivers were asked to evaluate their young child's participation in broad types of activities in the home (13 items, e.g., mealtime, cleaning up, indoor play), daycare/preschool (3 items, e.g., group learning, socializing with friends, field trips), and community (12 items, e.g., dining out, classes, community attractions) settings. The caregiver was provided with examples.

After completing participation items for a setting, caregivers evaluated the impact of environmental features (e.g., physical layout, activity demands, policies) and resources (e.g., transportation, equipment, money) on the child's participation (13 items for home, 16 items for daycare/preschool, and 17 items for community). For example, parents were asked, ‘does the physical layout help or made it harder for your child to participate in these activities at home’.⁴ Perceived impact of environmental features on participation was assessed on a 3-point scale (no impact/usually helps [3] to usually makes harder [1]). Perceived support of resources for participation was assessed on a 3-point scale (not needed/usually yes [3] to usually no [1]).

Craig Hospital Inventory of Environmental Factors for Children – Parent Version (CHIEF-CP)

The CHIEF-CP⁵ was adapted from the CHIEF, which is based on the ICF framework and has shown discriminant validity in psychometric testing on adults with and without disabilities⁵. The CHIEF-CP has been validated for use in studies involving children with disabilities (e.g., cerebral palsy, autism spectrum disorder (ASD)). The CHIEF-CP contains 10 items pertaining to environmental barriers that impact the child's participation in school and work, community, recreational, social, and civic activities. For example, caregivers were asked ‘how often did your child need someone else's help at preschool, school or work and could not get it easily’.⁵ For each item, caregivers reported on 1) frequency (never [1] to daily [5]), and 2) magnitude of impact (no problem [1] to big problem [3]). Frequency-magnitude product scores (representing overall item impact) were computed by multiplying the frequency and magnitude responses for each item. The CHIEF-CP has adequate internal consistency (α = 0.76 - 0.78) and test-retest reliability (ICC = 0.73).⁵

Data Analysis

Data collected online were saved in a central data repository and exported to IBM SPSS 22.0 for analyses. Data were screened via visual inspection (histogram) and normality statistics (absolute values of > 2 for skewness and > 7 for kurtosis) to reveal 6 CHIEF-CP items and 5 YC-PEM environment items that violated assumptions of normality, resulting in use of nonparametric tests for analyses on items. Fourteen cases with missing data for all CHIEF-CP items were excluded. Most YC-PEM environment items contained random missing data (range = 1-14, < 11% of cases) and were retained with use of pairwise deletion.

Addressing concurrent validity of the YC-PEM environmental content required identifying item-pairs by mapping those items along with CHIEF-CP items to the 5 ICF-CY environmental domains: 1) products and technology; 2) natural environment and human-made changes to the environment; 3) support and relationships; 4) attitudes; and 5) services, systems and policies.⁷ Pearson or Spearman rank correlations were computed on item-pairs that were identified through the ICF-CY mapping, to determine the degree of association between items using these criteria: r = 0.10-0.29 as weak, r = 0.30-0.49 as moderate, and r ≥ 0.50 as strong association.⁸ Internal consistencies of the CHIEF-CP scales were computed for our sample (α = 0.83 for frequency, α = 0.84 for magnitude). Alpha was set to 0.01 to reduce Type 1 error.

Results

Child and Family Characteristics

Participants were 381 caregivers of children between 1 and 71 months old (M = 36.49, SD = 20.18) and residing in the USA (91.0%) and Canada (8.9%). Most respondents were mothers (95.8%), married (90.0%), and had earned an associates, college, or graduate level education (78.8%). Eighty-five children were reported as receiving early intervention or early childhood special education services. The three most common functional issues reported by caregivers were related to managing emotions (34.9%), controlling behavior (31.0%), and paying attention (27.6%).

Concurrent Validity of YC-PEM Environmental Content

Results of 66 bivariate analyses are summarized in Table 2. Small to moderate negative associations were found for 51 item-pair associations (r = -0.13 to -0.39, p < 0.01). Since the CHIEF-CP was validated on children over 2 years of age, we conducted age subgroup analyses that produced similar patterns of associations.

Table 2.

Item-level associations involving corresponding YC-PEM Environment and CHIEF-CP items.

ICF-CY Domains	CHIEF-CP Items		YC-PEM Environment Items
		Home	Daycare/Preschool	Community
CF-2: Natural environment and human-made changes to environment	1. How often has the natural environment-temperature, terrain, climate-made it difficult for what your child wants or needs to do? (e2250, e2100, e225)		3. Outside weather conditions (temperature, climate) (e225, e2250)	8. Outside weather conditions (e225, e2250)
	a.Frequency b.Magnitude c.Product		a.r = .06, p = .57 b.r = .002, p = .98 c.r = −.003, p = .94	a.r = −.03, p = .55 b.r = .03, p = .59 c.r = −.03, p = .59
CF-2: Natural environment and human-made changes to environment	2. How often have other aspects of your child's surroundings-light, noise, crowds, etc.-made it difficult for what your child wants or needs to do? (e240, e250)	2. Sensory qualities (e.g., amount and/or type sound, light, temperature, textures of objects) (e215, e225, e240, e250, e255, e260)	2. Sensory qualities (e.g., noise, crowds, lighting, etc.) (e240, e250)	2. Sensory qualities (e.g., noise, crowds, lighting) (e240, e250)
	a.Frequency b.Magnitude c.Product	a.r = −.25, p < 0.001 b.r = −.21, p < 0.001 c.r = −.31, p < 0.001	a.r = −.29, p < 0.01 b.r = −.32, p < 0.001 c.r = −.30, p < 0.01	a.r = −.25, p < 0.001 b.r = −.22, p < 0.001 c.r = −.30, p < 0.001
CF-1: Products and technology	3. How often has the information your child wanted or needed not been available in a format that he/she can use or understand? (e130)	11. Information (about activities, services, programs) (e130, e560)	14. Information (about activities, services, programs) (e130, e560)	15. Information (about activities, services, programs) (e130, e560)
	a.Frequency b.Magnitude c.Product	a.r = −.23, p < 0.001 b.r = −.22, p < 0.001 c.r = −.28, p < 0.001	a.r = −.27, p < 0.01 b.r = −.26, p < 0.01 c.r = −.35, p < 0.001	a.r = −.19, p < 0.001 b.r = −.20, p < 0.001 c.r = −.25, p < 0.001
CF-5: Services, systems, and policies	4. In the past 6 months, how often has the availability of health care services and medical care been a problem for your child? (e5800)	9. Services in the home (e575, e580, e585)	12. Programs and services at the organized daycare or preschool (e.g., educational assistant, special resources, etc.) (e575, e580, e585)	13. Programs and services in the community (e575, e580, e585)
	a.Frequency b.Magnitude c.Product	a.r = −.29, p < 0.001 b.r = −.28, p < 0.001 c.r = −.28, p < 0.00	a.r = −.18, p = .05 b.r = −.18, p = .05 c.r = −.19, p = .04	a.r = −.25, p < 0.001 b.r = −.24, p < 0.001 c.r = −.25, p < 0.001
CF-3: Support and relationships	5. How often did your child need someone else's help at preschool, school or work and could not get it easily? (e340)
	a.Frequency b.Magnitude c.Product
CF-4: Attitudes	6. How often have other people's attitudes toward your child been a problem in the community? (e445, e425)		8. Attitudes/actions of directors, teachers, therapists, and other staff, who care for your child at daycare or preschool e425, e430)	6. Attitudes/actions of other members of community towards child (e.g., staff at stores and restaurants, instructors, coaches, child care provider, other families) (e425, e430, e455)
	a.Frequency b.Magnitude c.Product		a.r = −.20 b.r = −.19 c.r = −.39, p < 0.001	a.r = −.22, p < 0.001 b.r = −.20, p < 0.001 c.r = −.29, p < 0.001
CF-3: Support and relationships	7. How often has a lack of support and encouragement from others at preschool, school or work been a problem for your child? (e330, e325)		7. Child's relationship with peers (e320, e325)	7. Child's relationship with peers (e320, e325)
	a.Frequency b.Magnitude c.Product		a.r = −.33, p < 0.001 b.r = −.29, p < 0.01 c.r = −.31, p < 0.01	a.r = −.17, p < 0.01 b.r = −.16, p < 0.01 c.r = −.16, p < 0.01
CF-4: Attitudes	8. How often did your child experience prejudice or discrimination? (e445, e425)†	7. Attitudes and actions of babysitters, therapists, and other professionals who care for child at home (e440)	8. Attitudes/actions of directors, teachers, therapists, and other staff, who care for your child at daycare or preschool (e425, e430)	6. Attitudes/actions of other members of community towards child (e.g., staff at stores and restaurants, instructors, coaches, child care provider, other families) (e425, e430, e455)
	a.Frequency b.Magnitude c.Product	a.r = −.31, p < 0.001 b.r = −.29, p < 0.001 c.r = −.30, p < 0.001	a.r = −.32, p < 0.001 b.r = −.35, p < 0.001 c.r = −.34, p < 0.001	a.r = −.27, p < 0.001 b.r = −.27, p < 0.001 c.r = −.27, p < 0.001
CF-5: Services, systems, and policies	9. How often has the lack of programs and services in the community been a challenge? (e5750)			13. Programs and services in the community (e575)
	a.Frequency b.Magnitude c.Product			a.r = −.17, p < 0.01 b.r = −.18, p < 0.01 c.r = −.18, p < 0.001
	10. How often did education and employment programs and policies make it difficult for what your child wants or needs to do? (e585, e590)	8. Policies (residential and workplace policies, such as family leave or working from home, time off, work hours)	9. School-related policies and procedures (e.g., enrollment and attendance policies, check-in procedures and incident reporting to ensure safety, rules for behavior) (e585)	10. Policies (e.g., neighborhood, childcare, employer) (e585)
	a.Frequency b.Magnitude c.Product	a.r = −.15, p < 0.01 b.r = −.13, p < 0.01 c.r = −.14, p < 0.01	rules for behavior) (e585) d.r = −.02, p = .82 e.r = −.01, p = .90 f.r = −.03, p = .78	a.r = −.17, p < 0.01 b.r = −.14, p < 0.01 c.r = −.15, p < 0.01

Note: Values are based on use of Spearman rank correlational analyses for comparisons involving CHIEF-CP items 4, 5, 7-10 and YC-PEM items 9-13 in the daycare/preschool environment section.

Discussion

Validity of the YC-PEM for assessing environmental impact on young children's participation shows significant associations in 77% of cases where YC-PEM and CHIEF-CP were compared. Associations were most consistent when comparing CHIEF-CP and YC-PEM items capturing the influence of physical/structural barriers and attitudes. These findings are congruent with prior research using the CHIEF-CP to examine parent-perceived barriers to participation for children with cerebral palsy and other physical disabilities.^9-10

Items from both the YC-PEM and CHIEF-CP can be mapped to all five ICF-CY environmental domains and therefore capture the broadest range of environmental factors that might help or hinder participation when using the ICF-CY as a standard for relevant environmental content. However, these assessments differ in their level of specificity, which may explain the small to moderate association strengths found. While the YC-PEM affords for greater specificity, stakeholders may find its content to be more or less useful and/or feasible to obtain depending on their decision-making priorities.⁷ For example, a school-based therapist may benefit from use of the YC-PEM if afforded the time and resources to conduct a more comprehensive and detailed assessment of perceived environmental impact on daycare/preschool participation. Future studies are needed to understand the utility of the YC-PEM for meeting information needs of stakeholders when planning contextually focused interventions for individuals¹¹ and groups.⁷ Alternatively, lower association strengths may be due to a mixed sample of young children with and without developmental disabilities and delays as compared to a disability-only sample.⁷

Study limitations include use of a convenience sample with limited generalizability to larger US and Canadian populations based on child race/ethnicity, respondent characteristics (education level and marital status), and family income. Some YC-PEM environmental content is not captured in the CHIEF-CP and thus not validated in this study. Finally, pairwise deletion was used to retain sample size but may have minimized the strength of item-pair associations. Further validation of YC-PEM environmental content with different caregiver perspectives (e.g., father vs. mother, or caregiver vs. teacher) and with more diverse and representative samples are needed. It may also be helpful to examine differences in YC-PEM environmental scores by severity and/or type of impairment.

Conclusion

Findings lend further support for YC-PEM utility in large sample research towards improving knowledge about environmental influences on participation. This knowledge has potential to strengthen research on child versus environment-focused approaches to intervention for young children with disabilities.^7,11

Table 1.

Parent and child characteristics.

			Disability	No Disability
			N=85	N=296
Characteristic	Response		n(%)	n(%)
Respondent
	Mother		81(95.3)	284(4.1)
Employed*
	Yes		45(52.9))	150(50.8)
	No		40(47.1)	145(49.2)
Annual Income
	≤ 30,000		14(16.5)	29(9.8)
	30,001-50,000		13(15.3)	52(17.6)
	50,001-70,000		18(21.2)	46(15.5)
	70,001-100,000		19(22.4)	75(25.3)
	<100,000		21(24.7)	94(31.8)
Marital Status*
	Married		73(85.9)	270(91.2)
	Single, Never Married		5(5.9)	11(3.7)
	Domestic Partner		3(3.5)	11(3.7)
	Separated		2(2.4)	3(1.0)
	Divorced		2(2.4)	1(0.3)
Respondent Education
	Some high school, no diploma		0(0.0)	2(0.7)
	High school graduate		7(8.2)	5(1.7)
	Some college/university/technical training		14(16.5)	53(17.9)
	Associates degree		13(15.3)	12(4.1)
	College/university graduate		26(30.6)	112(37.8)
	Some graduate coursework		4(4.7)	20(6.8)
	Graduate degree		21(24.7)	92(31.1)
Geographic Region‡
	Canada		8(9.4)	26(8.8)
		Ontario	7(8.2)	21(7.1)
		New Brunswick	0(0.0)	3(1.0)
		Nova Scotia	0(0.0)	1(0.3)
		Alberta	0(0.0)	1(0.3)
		British Columbia	1(1.2)	0(0.0)
	United States		77(90.6)	270(91.2)
		West	57(67.1)	167(56.4)
		Midwest	5(5.9)	54(18.2)
		South	10(11.8)	40(13.5)
		Northeast	5(5.9)	9(3.0)
Child Race*
	American Indian/Alaskan Native		0(0.0)	1(0.3)
	Asian		2(2.4)	5(1.7)
	Black or African American		0(0.0)	3(1.0)
	White		72(84.7))	238(81.0)
	Multiracial		7(8.2)	39(13.3)
	Other		4(4.7)	8(2.7)
Child Gender
	Male		59(69.4)	156(52.7)
	Female		26(30.6)	140(47.3)
Childcare†
	Parent		67(78.8)	245(82.8)
	Daycare/Preschool/Kindergarten		31(36.5)	91(30.7)
	In-Home Provider		7(8.2)	14(4.7)
	Family Daycare/Cooperative		5(5.9)	24(8.1)
	Other		7(8.2)	9(3.0)
Service Enrollment†
	No		0(0.0)	296(100.0)
	Yes		85(100.0)	0(0.0)
		Speech and Language	62(72.9)	N/A
		Occupational Therapy	53(62.4)	N/A
		Physical Therapy	25(29.4)	N/A
		Private/Public Special Education Preschool	18(21.2)	N/A
		Other	29(34.1)	N/A
Functional Issues†
	Mobility		42(49.4)	13(4.4)
	Processing information		50(58.8)	18(6.1)
	Seeing		22(25.8)	3(1.0)
	Hearing		16(18.8)	4(1.4)
	Communicating with others		67(78.8)	30(10.1)
	Self-feeding		34(40.0)	17(5.7)
	Bladder and bowel control		36(42.4)	38(12.8)
	Paying attention		52(61.2)	53(17.9)
	Safety awareness		48(56.5)	46(15.5)
	Controlling behavior		51(60.0)	67(22.6)
	Managing emotions		48(56.5)	85(28.7)
	Reacting to sensations		47(55.3)	21(7.1)

^*Variables in which there is one missing value. Estimates are based on complete cases.

^†Respondents could select more than one response option.

^‡Sample distribution by geographic region according to census regions and divisions as reported by 2010 U.S. Census.

1. A new proxy questionnaire combining assessment of young children's participation in home, daycare/preschool, and community activities and perceived environmental impact on participation in each setting has been developed;

2. The questionnaire provides valid estimates of the perceived supportiveness of a young child's environment on participation when compared to a criterion assessment that is currently being used in pediatric rehabilitation.