Internal Consistency, Concurrent Validity, and Discriminant Validity of a Measure of Public Support for Policies for Active Living in Transportation (PAL-T) in a Population-based Sample of Adults

Abstract

Active living is a broad conceptualization of physical activity that incorporates domains of exercise; recreational, household, and occupational activities; and active transportation. Policy makers develop and implement a variety of transportation policies that can influence choices about how to travel from one location to another. In making such decisions, policy makers act in part in response to public opinion or support for proposed policies. Measures of the public’s support for policies aimed at promoting active transportation can inform researchers and policy makers. This study examined the internal consistency, and concurrent and discriminant validity of a newly developed measure of the public’s support for policies for active living in transportation (PAL-T). A series of 17 items representing potential policies for promoting active transportation was generated. Two samples of participants (n = 2,001 and n = 2,502) from Montreal, Canada, were recruited via random digit dialling. Analyses were conducted on the combined data set (n = 4,503). Participants were aged 18 through 94 years (58% female). The concurrent and discriminant validity of the PAL-T was assessed by examining relationships with physical activity and smoking. To explore the usability of the PAL-T, predicted scale scores were compared to the summed values of responses. Results showed that the internal consistency of the PAL-T was 0.70. Multilevel regression demonstrated no relationship between the PAL-T and smoking status (p > 0.05) but significant relationships with utilitarian walking (p < 0.05) and cycling (p < 0.01) for at least 30 minutes on 5 days/week. The PAL-T has acceptable internal consistency and good concurrent and discriminant validity. Measuring public opinion can inform policy makers and support advocacy efforts aimed at making built environments more suitable for active transportation while allowing researchers to examine the antecedents and consequences of public support for policies.

Introduction

Active living is a broad conceptualization of physical activity that incorporates domains of exercise; recreational, household, and occupational activities; and active transportation.¹ Examples of active living related to transportation include walking to local stores or services or cycling to work. Public policies have the potential to promote or impede active living by facilitating the creation of favorable built environments or by discouraging behaviors unfavorable to active living.² Examples of such policy interventions in transportation include, but are not limited to, modifying land-use regulations, implementing taxation on road use, and subsidizing public transportation.³ Research suggests that the implementation of policies supportive of active transportation have the potential to increase levels of physical activity in the general population.^4–9

Developing and implementing policies that are supportive of active transportation presents several challenges. Built environments are constructed and transformed by the joint actions of professionals from many fields, including urban planners and engineers, as well as social pressures exercised by the public on decision-makers and elected officials who enact policies related to the built environment. Although a growing body of research addresses the actions of urban planners and engineers in relation to active transportation, there are more limited data on public opinion related to potential changes to the built environment that would favor active transportation.¹⁰

In particular, recommendations for environmental and policy interventions suggest that it is important to “understand the agencies, organizations, and groups in diverse sectors that influence policies and environments relevant to physical activity.”^2(p396) Although not explicitly stated, the “public” should be considered a group that can influence policies. Political science researchers suggest that, “public opinion influences policy most of the time, often strongly.”^11(p29) The influence of public opinion on policy is likely strongest when policies are perceived as relevant by the public.^11,12 There is a lack of research examining public opinion in the context of environmental and policy interventions aimed at active transportation.¹³ Understanding public opinion as it relates to transportation is important because it can help to inform decision makers regarding citizen preferences about the implementation of specific policies. These citizen preferences can be used to inform decision makers. Similarly, understanding public opinion represents an object of theorization and interest among political scientists and public health researchers alike.

In spite of recommendations for policy interventions and agreement that public opinion influences policy, few measures have been developed to examine public opinion related to policies aimed at promoting active transportation.¹³ As a result, little is known about citizen preferences and agreement or disagreement with specific active living friendly transportation policies. The purposes of this study were to develop a measure of support for policies for active living in transportation (PAL-T) and to examine its internal consistency, concurrent, and discriminant validity.

Methods

Participant and Procedures

Two cross-sectional, population-based samples of adults were recruited via random digit dialing (n = 2,001 and n = 2,502) to participate in a telephone survey in the spring and the fall of 2009. The sampling design involved recruitment from a random sample of adults throughout the Island of Montreal, Canada (n = 2,001 participants in the spring and n = 2,502 in the fall), and oversampling of neighborhoods where a public bicycle share program had been implemented (n = 525 [26.2%] participants of n = 2001 in the spring and n = 646 [25.8%] of n = 2,502 in the fall; see also Fuller et al.).¹⁴ Oversampling was conducted in order to achieve sufficient statistical power for analyses in areas where a public bicycle share program had been implemented, on the basis that cycling is a relatively rare phenomenon in the area of study. In Canada in 2006, 0.6% of the population reported cycling as their primary mode of transportation. Response rates for the surveys were 32.6% for the spring and 34.6% for the fall samples. The two samples were combined to achieve a final sample of 4503 participants. The analysis reported here consisted of 4,117 respondents (91.4% of 4503) with complete sets of observations.

Ethical approval was obtained from the ethics committee of the Centre de Recherche du Centre Hospitalier de l’Université de Montréal (CRCHUM). Participants were recruited to participate in a 20-minute telephone survey conducted by a recognized polling firm with interviewers trained by the research team. Verbal informed consent was obtained prior to participation. Participant recruitment including up to five callbacks to improve response rate took 4–5 weeks for each survey. Inclusion criteria for survey participants were age at least 18 years and residing on the Island of Montreal, Canada. Exclusion criteria were as follows: not having a residential phone number and inability to respond to a telephone survey in either French or English.

Measures

The initial item pool was generated by authors DF and LG through brainstorming of potential local policy changes and recommendations that could be either favorable to (e.g., implementing traffic calming measures) or impede (e.g., highway construction) active transportation. To ensure that content was relevant to active transportation, items pertaining to walking, cycling, public, and automobile transportation policies were targeted. Through this brainstorming process 16 items representing active transportation friendly and non-friendly policies were generated. To establish face validity, author PM independently reviewed the 16 items and added an item (item 10). Following initial item development by DF, LG, and PM, the authors independently reviewed the 17 items for content and wording. Finally, DF and LG met to clarify item wording.

Policies for active living in transportation

In response to each of the 17 items pertaining to policies aimed at promoting active transportation, participants were asked to rate the extent to which they would agree with implementation of that specific policy in their residential neighborhood (see Table 2 for a description of each item). Responses were obtained for a 4-point scale: (1) completely agree, (2) somewhat agree, (3) somewhat disagree, and (4) completely disagree. A 4-point scale with no neutral choice was chosen to force participants to side one way or the other.

Table 2.

Observed proportions of persons residing on the Island of Montreal, Canada, in 2009 and falling into favorability response categories for each of 17 items pertaining to policies aimed at promoting active transportation, and corresponding coefficients from ordinal multilevel model

Item	Observed % response				Estimated coefficient (SE)+
	Completely agree	Partially agree	Partially disagree	Completely disagree
Item 8: Extending the subway to the east and west ends of the Island (n = 4,057)	85.9	9.2	1.2	2.1	2.86* (0.06)
Item 13: Stiffer fines for motorists caught speeding (n = 4,053)	71.0	13.7	5.2	8.5	1.78* (0.05)
Item 15: Stiffer fines for cyclists not obeying the traffic code (n = 4,054)	69.6	18.5	4.4	6.0	1.78* (0.05)
Item 16: Allowing more commercial establishments to settle around subway stations (n = 3,759)	53.5	27.9	4.5	5.4	1.35* (0.05)
Item 12: Reducing automobile speed limits from 50 to 30 km/h within a 3 km radius around schools (n = 4,054)	60.9	16.8	7.3	13.6	1.25* (0.04)
Item 11: Having more reserved lanes for cyclists (n = 4,003)	56.8	23.3	7.0	10.1	1.23* (0.04)
Item 14: Stiffer fines for pedestrians caught jay walking (n = 4,035)	54.4	23.0	8.1	12.5	1.07* (0.04)
Item 1: Allowing motor vehicle or bicycle to turn right when the traffic light is red (n = 3,947)	32.4	17.6	8.3	37.6	Reference Item
Item 2: Allowing the construction of a major highway within 2 km of your home (n = 3,844)	16.1	13.5	9.8	54.0	0.97* (0.04)
Item 7: Installing tramways on main boulevards on the Island of Montreal (n = 3,915)	50.8	21.0	5.6	17.7	0.91* (0.04)
Item 5: Closing down a commercial street to motor vehicles (n = 3,920)	35.0	24.1	11.5	24.7	0.26* (0.04)
Item 17: Reducing speed limits in the city from 50 to 40 km/h throughout the Island of Montreal (n = 3,989)	35.3	21.6	11.4	28.5	0.15* (0.04)
Item 3: Increasing the number of highways and the volume of traffic for motor vehicles coming in and out of the city (n = 3,859)	24.2	21.1	14.2	34.1	0.12§ (0.04)
Item 4: Implementing traffic calming measures such as making streets more narrow or blocking off street sections (n = 3,858)	27.5	22.8	12.6	30.8	−0.10§ (0.04)
Item 6: Installing toll booths to enter the Island of Montreal (n = 3,917)	29.7	20.3	7.4	37.7	−0.19* (0.04)
Item 9: Increase taxes on gasoline consumption to subsidize public transit (n = 4,019)	25.6	18.7	10.2	43.1	−0.45* (0.04)
Item 10: Increasing the cost of parking to finance (subsidize) public transit (n = 4,009)	21.9	18.5	11.5	45.4	−0.61* (0.04)

SE standard error

^§p < 0.05

*p < 0.001

Walking and cycling

Frequency of walking and cycling for different purposes were measured using a modified version of the 7-day recall long form of the International Physical Activity Questionnaire (IPAQ).¹⁵ The IPAQ is recommended for use in population based studies and has good test–retest reliability and convergent validity with accelerometers.¹⁵ The modified version has also been shown to have good reliability and validity.^6,16,17 In the modified version, participants reported how many days they walked or cycled for at least 10 minutes at a time in the previous 7 days.^6,7,16 Participants who reported at least one episode of walking or cycling for at least 10 minutes per episode were asked to estimate the average number of minutes they walked per day in the previous week. This is the standard procedure using the IPAQ. However, in addition, those reporting either walking or cycling were also asked if any of walking/cycling was done specifically to maintain health or fitness (i.e., leisure walking). Those reporting leisure walking or cycling were asked to estimate the number of days and the average amount of time per day. Utilitarian walking and cycling were estimated by subtracting the total walking or cycling minutes/week from the leisure walking or cycling minutes/week.⁶

Utilitarian walking and cycling were dichotomized as either meeting the American College of Sports Medicine physical activity guidelines of at least 30 minutes of activity on 5 days/week or not meeting the guideline.¹⁸

Smoking

A standard assessment procedure used for population surveillance in Quebec was used to assess smoking: Participants were asked to report if they were currently a smoker or if they smoked every day.

Analysis

The analysis strategy for the PAL-T measure was based on item-response theory.^19,20 Item-response theory suggests that the probability of agreement with an item is a mathematical function of the characteristics of the person (e.g., age, knowledge) and characteristics of the items (e.g., popular or controversial). Relevant to support for policies aimed at active transportation, persons highly in agreement with active transportation friendly policies should express support for both popular and controversial policies, whereas persons less favorably inclined should be in agreement only with items that are popular. Analyses allow for establishment of internal consistency across items and an item agreement map with the latter allowing for an estimate of the extent to which the whole universe of content has been included (i.e., does the item pool include both popular and controversial items). In the current case, a polytomous item response theory model (Samejima graded response model, also known as the cumulative logits model) was applied.²¹

Applying a multilevel modeling strategy, responses (level 1, n = 69,989) to the 17 PAL-T items were nested within the 4,117 respondents (level 2). Multilevel modeling accommodates missing data at level 1, in the present case responses to selected items. The dependent variable in all analyses was the ordinal response (i.e., completely agree, somewhat agree, somewhat disagree, and completely disagree) with complete agreement being the reference category. The multilevel analysis for the PAL-T included three steps. First, the variance was partitioned into between item and between respondent components. Second, the latent construct of PAL-T was examined by creating 16 dummy variables differentiating each item versus an arbitrarily selected reference item and then simultaneously entering them in the model, as level 1 predictors.

The coefficients estimating the relationship between complete agreement and dummy variables representing the other 16 items allowed for computation of an item agreement map (i.e., a difficulty map in the nomenclature of item-response theory) whereas the internal consistency estimate provided an indicator of reliability.²² Observed proportions of persons falling into each agreement response category were also computed.

As a third step, the concurrent and discriminant validities of the overall PAL-T were examined by including selected variables as univariate predictors of the intercept in the multilevel models. These three respective models tested associations between the PAL-T and reporting utilitarian walking 5 days/week for at least 30 minutes reporting cycling 5 days/week for at least 30 minutes and being a current smoker. The first two associations were designed to establish concurrent validity since support for selected policies should be most strongly associated with personal behavioral choices. The third association was used to estimate discriminant validity since there were no reasons to think that smoking status ought to be linked to public opinion despite evidence that smoking and physical activity are negatively correlated.¹⁸ Finally, a more practical method of using of the PAL-T was examined. Correlations were examined between empirical Bayes estimates of the PAL-T score from the multilevel model with the summed raw scores.²³

Results

Descriptive statistics pertaining to participant sociodemographic characteristics are shown in Table 1. Participants were, on average, 48.3 years old (SD = 17.1), and 58.9% were female. There was considerable variability with regard to the income, and level of education of the study participants.

Table 1.

Characteristics of 4,117 respondents on the Island of Montreal, Canada, in 2009

Characteristic	Percent (n)
Age (M = 48.25 years, SD = 17.06)
18–24 years	6.7 (n = 277)
25–34 years	18.6 (n = 766)
35–44 years	17.5 (n = 719)
45–54 years	20.4 (n = 840)
55–64 years	16.4 (n = 675)
65+ years	18.7 (n = 771)
Missing (n = 69)
Body mass index (M = 25.0 kg/m2)
Less than 18	1.7 (n = 68)
18–25	44.4 (n = 2,287)
25–30	30.6 (n = 1,260)
More than 30	12.1 (n = 500)
Missing (n = 2)
Education
Less than high school	28.0 (n = 1,151)
Trade school	7.3 (n = 302)
College degree	12.0 (n = 493)
University degree	51.9 (n = 2,136)
Missing (n = 35)
Income
Less than $20,000	12.8 (n = 526)
$20,000–50,000	30.0 (n = 1,236)
$50,000–100,000	24.1 (n = 994)
More than $100,000	11.5 (n = 475)
Missing (n = 866)
Sex
Male	41.1 (n = 1,691)
Female	58.9 (n = 2,426)
Missing (n = 0)
Smoker
Yes	21.0 (n = 826)
No	79.0 (n = 3,255)
Missing (n = 0)
Utilitarian walking (5 days for 30 minutes)
Yes	14.8 (n = 611)
No	83.5 (n = 3,438)
Missing (n = 70)
Utilitarian cycling (5 days for 30 minutes)
Yes	1.2 (n = 50)
No	98.8 (n = 4,067)
Missing (n = 0)

Between-Participant Variability and Internal Consistency

The first step of the multilevel analysis showed significant between-person variability in the likelihood of complete agreement with the reference item (=11,230, p < 0.001). Plausible value ranges extended from 22% (log-odds of −1.26) complete agreement with the reference item to 76% (log-odds of 1.18). The significant variance component indicates substantial between-person differences in agreement towards PAL-T. The internal consistency index (conceptually but not mathematically analogous to a Cronbach’s α), computed after inclusion of the 16 dummy variables to control for inter-item variability representing scale items, was 0.70.

Distribution of Responses to Items and Item Agreement Map

Table 2 shows the proportion of observed responses for each PAL-T item. Items are ordered as a function of ascending agreement. The item agreement map (Figure 1) was established using coefficients from the multilevel analysis. The item agreement map indicates variation across items with some having greater overall agreement and others, less so. The degree of support increases linearly. There were few gaps in the difficulty of items. Item hierarchy showed the policies that the population was most in agreement with included “extending the subway to the east and west ends of the Island” and “stiffer fines for motorists caught speeding.” Policies least associated with agreement were “installing toll booths to enter the Island of Montreal” and “implementing traffic calming measures such as making streets more narrow or blocking off street sections.” The policy that the population was least in agreement with was “increasing the cost of parking to subsidize public transit.”

FIGURE 1.

Item difficulty map for 17 items representing favorability towards active living friendly policies among 4,117 adults living on the island of Montreal in 2009. (Values are coefficients derived from ordinal multilevel models predicting complete agreement with proposed policy).

Concurrent and Discriminant Validity

There were no associations between the PAL-T and smoking status (odds ratio [OR] = 1.04; 95% confidence interval [CI] = 0.97–1.11) but statistically significant relationships were apparent in regard to likelihood of utilitarian walking for 30 minutes on at least 5 days/week (OR = 1.15; 95% CI = 1.07–1.24) and utilitarian cycling for 30 minutes on at least 5 days/week (OR = 1.83; 95% CI = 1.43–2.83) in the previous 7 days.

Practicality of PAL-T

The correlation between PAL-T estimates computed from the Samejima graded response multilevel model and summed item responses was large and statistically significant correlation (r = 0.76, p < 0.001). The summed value of responses was normally distributed (M = 34.82, SD = 7.68) making it appropriate for use as a continuous variable.

Discussion

The purpose of this study was to outline the development and utility, in terms of internal consistency, concurrent and discriminant validity, of a measure of agreement with PAL-T. A list of 17 items describing policies aimed at promoting active transportation that could be implemented in residential neighborhoods was identified. Internal consistency of the measure was acceptable (0.70). The item agreement map showed that the PAL-T captures policies ranging from popular to unpopular. Agreement with items was not clustered and the between-item variation increased linearly (i.e., the items were well distributed on the item agreement map). Examination of the concurrent validity indicated positive correlations with utilitarian walking and cycling and of discriminant validity showed no association with smoking status.

This study provides support for the utility of a new measure to capture public opinion related to policies specific to promoting active transportation, suitable for implementation by policy makers and public health officials. The PAL-T helps to fill a gap in active living research in assisting researchers, policy makers, and urban planners in understanding public opinion related to potential changes in the built environment. Past research has typically examined public perceptions related to general urban planning concepts, for example such as mixed land use, rather than specific interventions.^13,24 The significant between-person variability in the PAL-T measure suggests variability in public agreement with policies for active living in transportation. The overall proportion of complete agreement across items ranged from 16.1% to 85.9%, whereas the overall proportion of complete disagreement ranged from 2.1% to 54.0%.

An item agreement table (Figure 1) illustrates variation in agreement between specific policy items in the PAL-T. For policy makers, such a display of data can assist decision making. For example, specific policies such as extending the subway, stiffer fines for motorists and cyclist, and reduction of vehicle speed limits around schools all reflect a high proportion of agreement, and thus, are likely to be supported by the public. Where the population is not in agreement with items including implementing traffic calming measures and installing toll booths, greater promotion and public explanation of the value of these interventions may be appropriate.²⁵

The PAL-T is also a useful measure for researchers. The internal consistency, concurrent and discriminant validity of the PAL-T suggest that it is a potentially useful measure for understanding the determinants of utilitarian walking and cycling. Future research could examine potential clustering of PAL-T in different neighborhoods and use the PAL-T as a compositional variable.^26,27 This would advance research by providing a more complete examination of a comprehensive approach to population health promotion through the targeting of policy agendas related to active living and active transportation research.^28–30 In order to capture the breadth of active living, future research could develop measures of agreement to policies related to other domains. For example, it is possible to develop a PAL-W (Policies for Active Living in Workplaces) to measure workplace policies for active living.

Strengths and Limitations

Strengths of the present study include comprehensive assessment of internal consistency, concurrent and discriminant validity and the use of population-based samples. To correctly capture public opinion, population based sampling is essential.¹³

Despite these strengths, three limitations apply to the findings. The PAL-T was developed in Montreal, Canada. As a consequence, some items may not be applicable to urban areas in other industrialized nations. To ensure that the PAL-T is amenable to use in other cities, replication studies examining internal consistency, concurrent and discriminant validity should be conducted. Although the PAL-T was designed to measure population-level support for policies aimed at active transportation, the 35.8% response rates and survey sampling from landline telephones results in a sample that may not be representative of the target population. Finally, physical activity behaviors were measured by self-report and might overestimate activity levels, compared to objective measurement such as accelerometry.

Conclusions

The study responds to research recommendations for measures of agreement to policies for active living in transportation and potential groups that influence those policies. The PAL-T is a measure of public opinion in relation to potential changes in built environment resulting from public policies. It has good concurrent and discriminant validity and acceptable internal consistency. The PAL-T is also a measure that can be used to guide decision makers without the need for advance knowledge in measurement and statistics. Measuring public opinion via policies for active living in transportation can assist decision making for policy makers.