Scale Development for Perceived School Climate for Girls’ Physical Activity

Abstract

Objectives

To test an original scale assessing perceived school climate for girls’ physical activity in middle school girls.

Methods

Confirmatory factor analysis (CFA) and structural equation modeling (SEM).

Results

CFA retained 5 of 14 original items. A model with 2 correlated factors, perceptions about teachers’ and boys’ behaviors, respectively, fit the data well in both sixth and eighth graders. SEM detected a positive, significant direct association of the teacher factor, but not the boy factor, with girls’ self-reported physical activity.

Conclusions

School climate for girls’ physical activity is a measurable construct, and preliminary evidence suggests a relationship with physical activity.

Despite considerable evidence that physical activity confers substantial health benefits, many adolescents do not engage in sufficient activity to meet minimum recommended levels.^1,2 Physical activity (PA) declines during the transition from childhood to adolescence in both girls and boys.^3–6 Boys generally engage in more PA at each age,⁶ and there is a sharp decline among teenage girls.^7,8 Understanding the gender discrepancy can inform interventions promoting PA among girls. One of the few large-scale physical activity intervention studies in middle schools underscores this concern: investigators found that boys were more active than girls both during physical education (PE) classes⁹ and on school grounds during non-PE times.¹⁰ After 2 years of intervention, significant increases in PA were detected among boys but not girls.¹¹

Because adolescents spend a substantial amount of time at school, this environment is of interest for understanding and modifying PA behaviors.^12,13 The extent to which the school climate supports girls’ being physically active may be important. School climate has been described as the quality of the overall school environment, comprising 4 domains: physical environment, social system, culture, and aggregated characteristics of members of the school community.^14,15 Each of these domains could potentially affect girls’ participation in PA, for example, through adequate PA facilities (physical environment), interpersonal relationships that are respectful of gender differences and social and pubertal development (social system), values and norms that support a physically active lifestyle for all (culture), and the prevalence of PA-related behaviors of other girls and female staff members in the school (aggregated characteristics).

There is some evidence to support the hypothesis that general school climate or functioning affects students’ health behaviors such as use of tobacco, alcohol, or violence.^16–18 Research also suggests that school-level norms specifically concerning alcohol or tobacco use are related to individual students’ substance use.^19,20 To date, there has been less research into the effects of school climate on physical activity. School-level measures such as the recent School Health Index ²¹ will help facilitate investigations into the effects of school policies and physical environments on physical activity and other health behaviors. However, the School Health Index does not have an explicit gender focus, nor does it explicitly evaluate students’ perceptions of the culture regarding PA. A computerized literature search yielded no school climate survey measures specific to girls’ or to general PA. Qualitative research, however, supports the idea that girls perceive school climate regarding PA to be relevant to their PA participation both in PE classes^22–24 and overall.²⁵

The purpose of this study was to establish the factorial validity, factorial invariance, and construct validity of an original scale assessing perceived school climate for girls’ physical activity in a sample of sixth- and eighth-grade girls. The measure was developed as part of the Trial of Activity for Adolescent Girls (TAAG), a multicenter intervention study to promote PA in middle school girls. The candidate items for the school climate measure were developed through a review of qualitative research with adolescent girls including formative assessment conducted as part of the developmental phase of TAAG, as well as resources designed for school-level assessments, such as the CDC’s School Health Index²¹ and the National Association of School Physical Educators’ PE Program Improvement and Self-Study Guide for middle schools.²⁶ After developing the scale items, we used standard confirmatory factor analytic procedures^27,28 to establish the factorial validity and the multigroup and longitudinal factorial invariance of the school climate measure in samples of sixth- and eighth-grade girls. After the factorial validity and invariance of the measure were established, structural equation modeling was used to assess construct validity of the school climate measure by testing its relationship with physical activity, as well as the equivalence of that relationship between the 2 groups of girls (sixth and eighth graders), using standard procedures.²⁹

METHOD

Participants

Participants (N=605) were adolescent girls in the sixth (n=309) and eighth (n=296) grades recruited from communities in or near the 6 TAAG field centers: Baltimore, MD; Columbia, SC; New Orleans, LA; Minneapolis, MN; San Diego, CA; and Tucson, AZ. Nearly 80% of the sixth- (n=250) and eighth- (n=226) grade girls completed the measures both the initial time and again 2 weeks later, permitting an examination of the stability of the measurement instruments. The sixth-grade girls had a mean age of 11.5 (SD=0.6) years with racial/ethnic proportions of 45.6% white, 19.7% black, 14.2% Hispanic, 3.2% Asian/Pacific Islander, 1.9% American Indian, and 3.9% other; 11.3% of the sixth-grade girls did not report race/ethnicity. The eighth-grade girls had a mean age of 13.5 (SD=0.6) years with racial/ethnic proportions of 51.0% white, 17.6% black, 13.9% Hispanic, 3.0% Asian/Pacific Islander, 1.0% American Indian, and 3.0% other; 10.5% of the eighth-grade girls did not report race/ethnicity. There was not a statistically significant difference between sixth-and eighth-grade girls in the distribution of race/ethnicity, χ² (5, N=539) = 2.5, P=0.78. All participating schools were public schools, with a mean total enrollment of 872 (SD=448) students. The proportion of students receiving free or reduced-price lunch varied across schools, from a low of 31.8% to a high of 89.7% (mean = 43.5%, SD=12.8%).

Measures

School climate for girls’ PA

Fourteen candidate items were developed with a focus on school social systems and culture as they relate to girls’ PA. These items were pretested with convenience samples of approximately 25 sixth- and eighth-grade girls in Baltimore, MD and Columbia, SC. Based on feedback from the pretesters, modifications were made to the wording and formatting of the items, and the revised 14 items were included in the test-retest administration described above. Example items were “In my school, PE teachers act like they think it is more important for boys to be physically active than girls,” “In my school, the PE teachers like girls even if they aren’t good athletes,” and “In my school, boys make rude comments around girls who are being physically active.” Items were rated on a 5-point scale ranging from (1) disagree at lot to (5) agree a lot. Items were reverse coded prior to analysis so higher scores represented more favorable perceptions of school climate.

Physical activity

A modified version of the Physical Activity Questionnaire for Older Children (PAQ-C) reported elsewhere^30,31 was used. This measure was chosen because it has been validated for use with children of ages similar to the present sample. Also, its length and format minimized participant burden, which was a concern in this pilot study because all questions had to be answered by the students within a single class period. The original instrument used 9 questions to assess the level of participation in physical activity, which was defined as “sports, games, or dance that make you breathe hard, make your legs feel tired, or make you sweat.” Each item is scored on a 5-point scale, and the sum of the item scores is used as the indicator of physical activity. Internal consistency coefficients (Cronbach alpha) ranged from 0.79 to 0.89, and the test-retest stability coefficients across 2 weeks were 0.75 for boys and 0.82 for girls.³⁰ We used 5 original items that specifically assess activity in physical education classes, during the lunch period, right after school, in the evenings, and on the weekend. We removed an item pertaining to recess that was not relevant to our sample^30,31 and 3 other items that lacked specificity and were too time-consuming: an activity checklist, an item about general physical activity each day, and an item about overall physical activity. The 5-item measure of PA conformed to a single-factor model, and this model demonstrated factorial invariance between groups and across time in our previous analyses.

Procedure

The study procedure was approved by the university institutional review board at each of the 6 study centers. Written informed consent was provided by a parent or guardian, and each participant provided written assent at the time of survey administration. Testing was conducted in the spring of 2002, on 2 occasions separated by 2 weeks. Survey administration settings varied slightly across sites: some were in classrooms and others in cafeterias or other group settings. Survey data collectors participated in a centralized training to ensure that standardized procedures, scripts, and protocols were employed when collecting responses to the scales.

Data Analysis

Confirmatory factor analysis and structural equation modeling were used to test the factorial validity, factorial invariance, and construct validity of the measure of school climate. The analyses were performed using full-information maximum likelihood (FIML) estimation in AMOS 4.0 (SmallWaters Corp., Chicago, IL).³² FIML was selected because there were missing responses to items on the school climate questionnaire, ranging from 2.1 to 3.9% of missing responses per item. FIML is an optimal method for the treatment of missing data in SEM³² that has yielded accurate fit indices and parameter estimates with simulated missing data.^33,34

Model fit was assessed using multiple indices. The χ² statistic assesses absolute fit of the model to the data, and a nonsignificant χ² supports the perfect fit of the model.^35–37 Two recognized limitations of the χ² statistic are sensitivity to sample size and the assumption of the correct model.^35–37 Therefore, no restrictive model with positive degrees of freedom is able to fit real data, and such models often will be rejected by a formal significance test with a sufficiently large sample.^38,39 Accordingly, other fit indices must be used for judging model fit. The root mean square error of approximation (RMSEA) represents closeness of fit, and values approximating 0.06 and zero demonstrate close and exact fit of the model.^38,40 The 90% confidence interval (CI) around the RMSEA point estimate also should contain 0.06 or zero to indicate the possibility of close or exact fit. The comparative fit index (CFI) and non-normed fit index (NNFI) test the proportionate improvement in fit by comparing the target model with the independence model.^41,42 Minimally acceptable fit was based on CFI and NNFI values of 0.90;^41,42 values approximating 0.95 indicate good fit.⁴⁰ The parameter estimates, standard errors, z-statistics, and squared multiple correlations (SMCs) were inspected for sign and/or magnitude. Parameters with nonsignificant z-statistics and/or a sign opposite of expected direction have no substantively meaningful interpretation.³⁶ Large standard errors provide an indication that the parameter estimate is not reliable.⁴³

The test of factorial validity was performed with standard confirmatory factor analytic procedures. The procedures involved testing the fit of a measurement model for the items on the school climate scale. Standard procedures were used to establish the parameters in the factor loading, factor variance-covariance, and item uniqueness matrices for the measurement model.

The tests of factorial invariance (across grades and over the 2-week test-retest period) were performed with confirmatory factor analysis. The tests involved comparing models that imposed successive restrictions on model parameters for the equality of the overall structure, factor loadings, item uniquenesses, and factor variances and covariances.³⁵ The comparison of nested models was based on chi-square difference tests and changes in the values of the RMSEA, CFI, and NNFI. The criterion of −0.01 for a change in the CFI (CFI_{constrained model} − CFI_{unconstrained model}), for example, has been reported to be robust for testing the multigroup invariance⁴⁴ and has been employed in analyses of longitudinal factorial invariance.²⁹

Construct validity of the school climate measure was evaluated using structural equation modeling. We adopted a structural equation modeling approach because it provides a test of the relationship between school climate and physical activity that is not biased by measurement error. Moreover, we could test the direction and magnitude of the association between school climate and physical activity for equivalence between sixth- and eighth-grade students with the structural equation modeling approach. The measure of effect produced by the structural equation models reported here, standardized gamma (γ), represents the association of an exogenous variable (school climate) with an endogenous variable (self-reported physical activity) and is interpreted as the amount of change in the endogenous variable associated with a 1-unit change in the exogenous variable.

RESULTS

Factorial Validity

Initial tests with the sample of sixth-grade students indicated that the fit of a single-factor model to the 14 school climate items was not acceptable (χ² = 583.64, df=77, P<0.001, RMSEA=0.15 [90% CI=0.14–0.16], CFI=0.50, NNFI=0.41). Four items were identified as potentially problematic due to interpretation difficulties and possible redundancy with other items. Those items were removed, and the fit of the single-factor model remained unacceptable among the sixth graders (χ² = 204.66, df=35, P<0.001, RMSEA=0.13 [90% CI=0.11 – 0.14], CFI=0.66, NNFI=0.57). Items with nonsignificant factor loadings were then removed sequentially until 5 items remained that all had significant factor loadings, but the 5 items were still not represented by a single factor among the sixth graders (χ² = 70.18, df=5, P<0.001, RMSEA=0.21 [90% CI = 0.17–0.25], CFI=0.84, NNFI=0.67). Based on item content, 2 conceptually distinct, but correlated factors were identified: 1 pertaining to teachers (2 items) and 1 pertaining to boys (3 items). This 2-factor, correlated model provided an excellent fit in the sixth-grade girls (χ² = 2.04, df=4, P=0.73, RMSEA = 0.00 [90% CI = 0.00 – 0.06], CFI = 1.00, NNFI=1.01) and a good fit in the eighth-grade girls (χ² = 14.28, df=4, P=0.006, RMSEA=0.09 [90% CI=0.04 – 0.15], CFI = 0.97, NNFI = 0.93). The interfactor correlations were 0.57 and 0.33 in the samples of sixth- and eighth-grade girls, respectively. Table 1 presents the items retained in the final 2-factor solution, and the standardized factor loadings for both samples.

Table 1.

Items and Factor Loadings of the School Climate Measure in a Sample of Sixth and Eighth Graders^a

	Factor Loading for Sixth-grade Students	Factor Loading for Eighth-grade Students
Factor 1: Teachers
In my school, PE teachers act like they think it is more important for boys to be physically active than girls	.825	.809
In my school, most teachers think it is more important for boys to be physically active than girls.	.876	.964
Factor 2: Boys
In my school, boys make rude comments around girls who are being physically active.	.491	.545
In my school, being physically active around boys makes me uncomfortable.	.591	.489
In my school, boys stare too much at girls who are being physically active.	.795	.702

^aNote. Response Options: 1=Disagree a lot, 2=Disagree a little, 3=Neither agree nor disagree, 4=Agree a little, 5=Agree a lot. Items were reverse coded for analysis so higher scores reflected more favorable perceived school climate.

Factorial Invariance

The 2-factor, correlated model demonstrated strong invariance of the factor structure, factor loadings, and item uniquenesses, and acceptable invariance of the factor variances and covariance, between the sixth- and eighth-grade samples. The model demonstrated strong invariance of the factor structure, factor loadings, and factor variances and covariances, but not item uniqueness across 2 weeks within the combined sample of sixth- and eighth-grade girls based on the longitudinal invariance analysis. The stability coefficients for the factors pertaining to teachers and boys across 2 weeks were 0.59 and 0.56, respectively.

Construct Validity

Sixth-grade girls

The model specifying direct associations of the 2 correlated factors pertaining to teachers and boys with physical activity is displayed in Figure 1 and provided a perfect fit in the sample of sixth-grade girls (χ² = 38.81, df=32, P=0.19, RMSEA [90% CI]=0.03 [0.00–0.05], CFI=0.97, NNFI=0.96). There was a statistically significant direct association of the teacher factor with physical activity (γ₁₁ = 0.17), but not a significant association of the boy factor (γ_I2 = −0.06).

Figure 1.

Structural Equation Model Fit in Sixth- and Eighth-grade Girls

Note. Abbreviations: SC=school climate, PA=physical activity, D=disturbance term

Eighth-grade girls

The model specifying direct associations of the two correlated factors pertaining to teachers and boys with physical activity is displayed in Figure 1 and provided a good fit in the sample of eighth-grade girls (χ² = 59.02, df=32, P=0.003, RMSEA [90% CI]=0.05 [0.03–0.08], CFI=0.95, NNFI=0.94). There was a statistically significant direct association of the teacher factor with physical activity (γ₁₁ = 0.24), but not a significant association of the boy factor (γ₁₂ = −0.15).

Multigroup invariance analysis

We tested a final model that constrained the direct associations of the 2 correlated factors pertaining to teachers and boys on physical activity to be equal between the sixth- and eighth-grade girls. This enabled a statistical test of possible differences in the magnitude of those paths between age groups. The first model did not constrain the 2 paths to be equal between groups, and provided a good fit (χ² = 97.83, df=64, P=0.004, RMSEA [90% CI] = 0.03 [0.02–0.04], CFI=0.97, NNFI=0.96). The second model constrained the 2 paths to be equal between groups, and this model provided a good fit (χ² = 98.12, df=66, P=0.006, RMSEA [90% CI] = 0.03 [0.02–0.04], CFI=0.97, NNFI=0.96). There was not a statistically significant difference in fit between the 2 nested models (χ²_diff = 0.29, df=2, P=0.87). Therefore, the paths were similar in direction and magnitude between groups of sixth- and eighth-grade girls.

DISCUSSION

Our data support the hypothesis that perception of the school climate for girls’ physical activity is a construct that is measurable, factorially invariant, and related to physical activity among adolescent girls. The best fitting model consisted of 5 items that were represented by 2 correlated factors. The 2 correlated factors described girls’ perceptions of teachers and boys, respectively, in relation to girls’ being physically active. The school climate measure demonstrated factorial invariance across the 2 grade levels (sixth and eighth) and across a 2-week time window, allowing for valid comparisons of scores from the measure between grades and across time. The teacher factor, but not the boy factor, had a significant direct association with self-reported physical activity in both sixth- and eighth-grade girls. The direct association of the teacher factor with self-reported physical activity was of similar direction and magnitude across both grades.

The work reported here provides preliminary support for continuing to explore the role of school climate in girls’ PA behaviors. The TAAG main trial, currently in progress, will provide a stronger test of the school climate measure in several areas: (1) variability in school climate is expected to increase with 36 schools in the main trial compared with the 6 pilot schools represented in the present study; (2) precision will be increased with a substantially larger sample size than used in the current study; and (3) assessment of validity of the climate variable will be improved as physical activity will be measured objectively with accelerometers rather than with the PAQ-C used in the present study. Additionally, 3-day PA recalls using a modified version of the 3-day physical activity recall (3DPAR) will provide details concerning when, where, and with whom girls participated in PA.⁴⁵ Nevertheless, this preliminary test of a brief measure of a novel construct, school climate for girls’ PA, provides support for further investigation. Our hope in reporting these results is to encourage the testing of this construct in other settings as a step in further learning how school climate and girls’ PA are related. Interest in the school environment for PA, eating, and other health-related behaviors is increasing,^11,13,21 and measures such as this one could help provide insights for intervention, such as possible needs for marketing girls’ PA not only to girls themselves, but also to important secondary audiences such as teachers and boys, whose attitudes and behaviors may be influential.

Like many other school climate measures, ^14,46,47 the measure reported here assesses individuals’ perceptions rather than higher-level institutional characteristics. It would be of interest to conduct a parallel investigation of school climate for girls’ PA by exploring observable characteristics of the school environment, such as bulletin boards and displays, school publications and website, or interactions among teachers, boys, and girls. Such factors could contribute to the perceptions of school climate reported here, and might reveal promising strategies for school-level interventions to promote PA among adolescent girls.