Development and Validation of a Smoking Media Literacy Scale for Adolescents

Abstract

Objectives

To develop a smoking media literacy (SML) scale by using empiric survey data from a large sample of high school students and to assess reliability and criterion validity of the scale.

Design

On the basis of an established theoretical framework, 120 potential items were generated, and items were eliminated or altered on the basis of input from experts and students. Cross-sectional responses to scale items, demographics, smoking-related variables, and multiple covariates were obtained to refine the scale and determine its reliability and validity.

Setting

One large Pittsburgh, Pa, high school.

Participants

A total of 1211 high school students aged 14 to 18 years.

Main Outcome Measures

Current smoking, susceptibility to smoking, attitudes toward smoking, and smoking norms.

Results

Factor analysis demonstrated a strong 1-factor scale with 18 items (α = 0.87). After controlling for all covariate data, SML had a statistically significant and independent association with current smoking (P = .01), susceptibility (P<.001), and attitudes (P<.001), but not norms (P = .42). Controlling for all covariates, an increase of 1 point on the 10-point SML scale was associated with a 22% decrease in the odds of being a smoker and a 31% decrease in the odds of being susceptible to smoking.

Conclusions

Smoking media literacy can be measured with excellent reliability and concurrent criterion validity. Given the independent association between SML and smoking, media literacy may be a promising tool for future tobacco control interventions.

Cigarette smoking is the leading cause of preventable death and disease in the United States,¹ and about 90% of those who die from smoking began during adolescence. ² However, traditional school-based smoking prevention programs have not been successful in affecting clinically relevant smoking behaviors.^3–6

Youth aged 8 to 18 years are exposed to 8 hours 33 minutes of mass media content daily,⁷ including a substantial number of positive portrayals of cigarette smoking. ^8–10 Research has demonstrated a strong association between exposure to certain mass media messages and smoking in adolescents. For instance, more than half of adolescent smoking initiation has been linked to watching smoking in movies. ^11–13 Exposure to mass media messages such as promotions and advertisements also significantly increases the risk of smoking initiation by adolescents.^2,14–17

Media literacy therefore represents a promising framework for development of innovative school-based tobacco control programs.¹⁸ Acknowledging the effects of mass media on attitudes and behavior, media literacy teaches youth to understand, analyze, and evaluate advertising and other mass media messages, enabling them to actively process media messages rather than passively remain message targets.^19,20 Media literacy has been shown to be both feasible and teachable,^20,21 making it attractive as an intervention.

Recognizing its potential, the American Academy of Pediatrics,¹⁸ the Centers for Disease Control and Prevention,²² and the US Office of National Drug Control Policy²³ have called for the use of media literacy to reduce harmful health behaviors such as smoking. One consistent limitation of smoking media literacy (SML) evaluations, however, is the lack of a reliable, validated scale measuring this construct in youth. Development of such a psychometrically appropriate scale is necessary to evaluate the effectiveness of media literacy interventions designed to reduce adolescent smoking.

To validate a measure of SML, it is necessary to apply an appropriate theory of health behavior. The theory of reasoned action has been used to accurately predict adolescent smoking.^24–27 According to this theory, an individual’s behavior is determined by his or her intention to perform the behavior, which is in turn predicted by his or her attitude toward the behavior and perception of norms regarding it.²⁴ Media literacy theoretically affects both attitudes and norms involving smoking (Figure 1).

Figure 1.

The theory of reasoned action, applied to mass media and smoking. Media literacy interventions, which examine the manipulative construction of tobacco industry messages and the salient omissions of those messages, would be expected to decrease positive attitudes toward smoking. Media literacy would also be expected to correct misperceptions regarding the normality of smoking by (1) exposing the industry techniques used to make behaviors seem more normative and (2) emphasizing the difference between “television reality” and “real reality.”

The aims of this study were (1) to develop a scale measuring SML in adolescents, using empiric survey data from a large sample of high school students, and (2) to assess internal consistency (ie, reliability) and concurrent criterion validity of the SML scale.

METHODS

Our study was conducted in 3 distinct phases, in which we (1) rigorously developed a pool of potential SML scale items, (2) collected empiric cross-sectional data from adolescents, and (3) refined the scale and assessed its reliability and validity.

PHASE 1: ITEM DEVELOPMENT

Two major accepted theoretical models exist describing media literacy^28,29; although the models overlap substantially, there are differences in emphasis. A British model emphasizes understanding (1) the purposes of media producers and characteristics of target audiences, (2) the multiple complex production techniques used to convey meaning, and (3) the ability to distinguish media representations from reality.²⁸ A US model emphasizes that (1) media messages are carefully constructed with the use of their own complex language, (2) different individuals interpret messages differently, (3) messages contain inherent values and perspectives, and (4) media messages are usually created for profit and/or power.²⁹ To maximize content validity of the scale, we combined the 2 models into a comprehensive framework integrating core concepts from each model (Table 1). We then developed 120 Likert-type scale items (strongly disagree, disagree, agree, strongly agree), with 15 items representing each of the 8 core concepts. We developed items related to both persuasive media (such as promotions and advertisements) and narrative media (such as episodes of smoking in films and on television) because of the important role each genre plays in media literacy.³⁰ We also included both general and smoking-specific items.

Table 1.

Media Literacy Framework^*

Media Literacy Domain	Related Media Literacy Core Concepts
Authors and audiences (AA)	AA1: authors create media messages for profit and/or influence
	AA2: authors target specific audiences
Messages and meanings (MM)	MM1: messages contain values and specific points of view
	MM2: different people interpret messages differently
	MM3: messages affect attitudes and behaviors
	MM4: multiple production techniques are used
Representation and reality (RR)	RR1: messages filter reality
	RR2: messages omit information

^*Core concepts from 2 major theories of media literacy²⁸,²⁹ were integrated into this model.

We distributed this pool of items for review to a convenience sample of 8 leading national experts in media literacy, tobacco control, and public health. We also held 2 hour-long focus groups with 9th- to 11th-grade adolescents. One was held at a primarily white high school in a middle-income neighborhood (8 students) and the second at a predominantly African American high school in a low-income neighborhood (11 students). Items were eliminated or altered on the basis of consensus of both experts and students, resulting in a 51-item pool, with several items representing each of the 8 core concepts of media literacy.

PHASE 2: DATA COLLECTION IN A DEVELOPMENT SAMPLE

We administered this refined item pool to a sample of all students aged 14 to 18 years at a large Pittsburgh, Pa, public high school (enrollment, 1690). In addition to media literacy items, we asked students to provide demographic information, smoking-related data, and covariate information. Demographic information included age, sex, parental education (as a surrogate for socioeconomic status), race, and ethnicity. Smoking-related data obtained measured 4 smoking-related variables defined by the theory of reasoned action: (1) current smoking, defined as having smoked in the past 30 days; (2) intention to smoke, assessed with Pierce and coworkers’ reliable and valid susceptibility scale³¹; (3) attitudes toward smoking, assessed with 18 items based on Buller and coworkers’ reliable and valid scale³²; and (4) smoking subjective norms, assessed with 3 items based on the Fishbein-Ajzen-Hansen questionnaire.³³ A high score on this norms scale (stronger “antismoking norms”) indicates that the individual feels that those close to him or her do not approve of smoking. Covariates obtained included media use habits, parent smoking, friend smoking, sibling smoking, stress, depression, self-report of grades, knowledge of the effects and addictiveness of tobacco, demanding parenting, responsive parenting, sensation seeking, and rebellious behavior. Covariates were measured with previously validated and/or commonly used scales.^11,31,32

The vast majority (1402) of the 1525 students eligible to complete the survey participated (Figure 2). Before data analysis, we defined specific criteria to detect and eliminate questionnaires with poor data quality. If 3 or more responses were deemed impossible or extremely improbable (such as claims to have smoked an average of 120 cigarettes per day), that respondent’s data were eliminated from the analysis. By this process, data from 44 students (3%) were eliminated. In addition, students were asked in a final survey item to appraise their honesty with the survey. Those who admitted having been dishonest (147 students) were eliminated from the analysis, resulting in a final sample size of 1211 (86% of returned surveys). The unpaired, 2-tailed t test, an exact linear rank test for comparing 2 ordered multinomial proportions (using a Wilcoxon statistic), and a χ² test confirmed that those eliminated from the analysis were no different in terms of age, race, or reported parental education, respectively. Those eliminated were, however, more likely to be male (71% vs 48%; P<.001).

Figure 2.

Study population and response rate.

Approval for the project was granted by both the superintendent of the school district and the institutional review board of the University of Pittsburgh. Both bodies agreed to a waiver of parental informed consent, since students did not place their names or any other unique personal identifiers on the questionnaires. All students completed the questionnaire during social studies classes, and those who completed the questionnaire were given a packet of trail mix.

PHASE 3: SCALE REFINEMENT AND DETERMINATION OF RELIABILITY AND VALIDITY

We performed iterative principal components analysis (PCA) using varimax rotation with the 51 media literacy items to determine the underlying factor structure produced by these items.³⁴ The first iteration of PCA showed 1 strong factor with an eigenvalue of 8.2 explaining 53% of the variance, a much weaker but possible second factor, and a scree plot indicating a likely 1-factor solution. We conducted a second PCA on the 23 items that maintained a loading of at least 0.45 on 1 of the first 2 factors. This PCA resulted in a conclusive 1-factor solution with an eigenvalue of 6.0 explaining 87% of the variance (the second factor’s eigenvalue was 0.9). Eighteen of the 23 items were related to this primary factor with a loading of greater than 0.45, and all were retained in the scale (Table 2). We used a cutoff factor loading of 0.45 to ensure that the scale did not contain too many or too few items and to ensure that the selected items were highly correlated with the underlying construct of media literacy. Qualitative analysis of the final 18 items confirmed that the scale did seem to measure SML and not other competing constructs. For instance, the items appropriately represented each of the theoretical domains of the framework (Table 2). In addition, retained items related to both narrative (items 7, 10, 12, and 14) and persuasive media, and some items were general (items 7, 10, 11, 12, and 18) while others were smoking-specific. Finally, some items (items 3 and 5) had strong cynical sentiment, whereas other items (items 7, 8, 9, and 12) were neutral in tone. This is consistent with the theoretical construct of media literacy: cynicism and anti-industry attitudes can result from critical appraisal of media messages, but it is only one aspect of media literacy. The final 18-item scale has excellent internal consistency, with a Cronbach α = 0.87. For ease of intuitive interpretation and potential future application, the resulting 54-point SML scale was converted to a 10-point scale by dividing the raw 54-point score by 5.4.

Table 2.

Scale Item Core Concepts, Factor Loadings, and Correlations With Smoking Outcomes^*

Item No.	Related Core Concept†	Factor Loading	Pearson r Between Item and Current Smoking	Pearson r Between Item and Susceptibility to Smoking
1	AA1	0.47	−0.10	−0.10
2	AA1	0.58	−0.01	−0.14
3	AA1	0.62	−0.10	−0.19
4	AA2	0.54	−0.05	−0.10
5	MM1	0.54	−0.09	−0.20
6	MM1	0.58	−0.12	−0.13
7	MM2	0.51	−0.03	−0.11
8	MM2	0.55	−0.02	−0.14
9	MM2	0.52	−0.06	−0.15
10	MM3	0.54	−0.12	−0.13
11	MM3	0.60	−0.12	−0.15
12	MM4	0.48	−0.08	−0.20
13	MM4	0.48	−0.15	−0.15
14	RR1	0.47	−0.12	−0.12
15	RR1	0.62	−0.17	−0.17
16	RR1	0.55	−0.19	−0.24
17	RR2	0.56	−0.17	−0.22
18	RR2	0.66	−0.20	−0.22

^*All items were tested on a 4-point Likert scale: strongly disagree, disagree, agree, or strongly agree. Wording of actual items is available from the authors. For boldface entries, P<.05 by the Dunn-Sidak method of correction.

^†See Table 1 for explanation.

We then assessed concurrent criterion validity by examining associations between students’ SML scale values and smoking-related variables and covariates in the theory of reasoned action. We first computed pairwise Pearson correlation coefficients between SML and all smoking measures and covariates. We also computed Pearson correlation coefficients between the major clinically significant smoking measures (current smoking and susceptibility) and each of the individual SML items. Significance was defined with a P<.05 but was corrected by the Dunn-Sidak method to reduce the likelihood of false-positive significant results.³⁵ When variables were dichotomous, 2-group t tests of the SML scores were used instead. Regression techniques were then used to determine whether associations between SML and each of the 4 theory of reasoned action variables remained after controlling for major factors known to predict smoking: sex, age, socioeconomic status, media use habits, parent smoking, friend smoking, sibling smoking, stress, depression, self-report of grades, knowledge of the effects and addictiveness of tobacco, demanding parenting, responsive parenting, sensation seeking, and rebellious behavior. Multiple linear regression was used for continuous outcome variables and logistic regression was used for binary outcome variables. Race and Hispanic ethnicity were not included in the regression model because of the extremely low proportions of nonwhite and Hispanic participants.

RESULTS

The sample had a nearly equal sex distribution, and mean age was 15.9 years (Table 3). The sample was predominantly white, with few African Americans and Latinos. Of the sample, 19% reported smoking in the past 30 days and 50% were classified as susceptible to smoking.

Table 3.

Baseline Respondent Characteristics

Characteristic	Finding
Age, mean (SD), y	15.9 (1.2)
Sex, No. (%) M	572 (47.7)
Race, No. (%)
White	1092 (91.7)
Black	49 (4.1)
Other	50 (4.2)
Hispanic ethnicity, No. (%)	11 (0.9)
Parental education, No. (%)*
≥1 Parent did not graduate from high school	58 (5.2)
Both parents graduated from high school, neither from college	365 (32.9)
1 Parent graduated from college, 1 from high school	320 (28.9)
Both parents graduated from college	365 (32.9)
Current smoker, No. (%)†	216 (19.0)
Susceptible to smoking, No. (%)‡	575 (50.0)
Prosmoking attitude score, mean (SD) (range, 18–72)	36.2 (9.1)
Antismoking norms score, mean (SD) (range, 3–12)	8.7 (1.9)
Household member smokes, No. (%)	468 (38.9)
Close friend smokes, No. (%)	628 (56.9)
Sibling smokes, No. (%)	267 (22.7)
Smoking media literacy score, mean (SD) (range, 0–10)	6.9 (1.3)

^*Parental education was used as a surrogate for socioeconomic status.

^†Current smoking was defined as smoking at least once in the past 30 days.

^‡Susceptibility to smoking was defined by at least 1 positive response on the reliable and valid 3-item scale of Pierce et al.³¹

The SML score was significantly lower in current smokers (t = 6.60, P<.001) and in those susceptible to smoking (t = 9.60, P<.001). Pairwise Pearson correlation coefficients showed SML to be highly negatively associated with prosmoking attitudes (r = −0.49, P<.001) and positively associated with antismoking norms (r = 0.22, P<.001).

Of the covariates, SML was positively associated with socioeconomic status (r = 0.13, P = .003), responsive parenting (r = 0.18, P<.001), demanding parenting (r = 0.22, P<.001), and self-report of grades (r = 0.27, P<.001). It was negatively associated with rebelliousness (r = −0.26, P<.001) and sensation seeking (r = −0.12, P = .01). The level of SML was also lower in those with siblings (t = 3.62, P<.001), parents (t = 3.61, P<.001), and friends (t = 7.26, P<.001) who smoke. It was not significantly correlated with age, sex, race, Hispanic ethnicity, knowledge of harm and addictiveness of tobacco, self-esteem, depression, or stress.

Table 2 shows the Pearson correlations between each individual item and (1) current smoking and (2) susceptibility to smoking. Each of the items tended to be associated with a lower likelihood of current smoking and smoking susceptibility, indicated by the negative correlation coefficients. In addition, 9 of the 18 coefficients had statistically significant negative associations with current smoking, and 16 of the 18 items had statistically significant negative associations with being susceptible to smoking.

Multivariate regression analyses (Table 4) showed that, after controlling for all covariates, the SML score was independently associated with current smoking (P = .01), susceptibility to smoking (P<.001), and antismoking attitudes (P<.001). After controlling for all of these cofactors, however, the SML score was not independently associated with smoking norms (P = .42). Logistic regression was used for binary outcomes (smoking and susceptibility), so we report odds ratios with 95% confidence intervals. Multiple linear regression was used for the continuous outcomes (attitude and norms), so for these outcomes we report the regression coefficients, standard errors, and P values. These P values help determine whether the regression coefficients are significantly greater than zero, indicating that the variable being tested is an independent predictor of the outcome.

Table 4.

Multivariate Associations Between SML and Smoking^*

	Odds Ratio (95% CI)		Positive Attitude Toward Smoking		Antismoking Norms
	Current Smoking†	Susceptibility to Smoking‡	Coefficient (SE)	P Value	Coefficient (SE)	P Value
SML (1 point on 10-point scale)	0.78 (0.65–0.95)	0.69 (0.59–0.80)	−2.26 (0.20)	<.001	0.04 (0.05)	.42
Age (1 yr)	1.32 (1.06–1.64)	0.92 (0.79–1.08)	−0.04 (0.22)	.84	−0.13 (0.05)	.02
Sex (M vs F)	1.21 (0.74–1.98)	0.99 (0.69–1.43)	−0.62 (0.51)	.23	−0.17 (0.13)	.18
SES (1 point on 5-point scale based on parental education)	1.12 (0.86–1.46)	1.13 (0.93–1.37)	0.05 (0.27)	.86	0.00 (0.07)	.97
Smoking knowledge (1 point on 5-point scale)	0.76 (0.60–0.96)	0.99 (0.84–1.18)	−0.60 (0.24)	.01	0.20 (0.06)	.001
Electronic media use (1 h/d)	1.02 (0.98–1.07)	0.98 (0.95–1.02)	−0.05 (0.05)	.33	0.01 (0.01)	.31
Responsive parenting (1 point on 7-point scale)	0.96 (0.77–1.21)	0.99 (0.82–1.20)	−0.26 (0.26)	.32	0.15 (0.06)	.02
Authoritative parenting (1 point on 7-point scale)	1.04 (0.84–1.29)	0.93 (0.78–1.10)	−0.64 (0.24)	.01	0.25 (0.06)	<.001
Sensation seeking (1 point on 7-point scale)	1.27 (1.01–1.59)	1.38 (1.17–1.63)	0.71 (0.22)	.001	−0.02 (0.05)	.67
Rebelliousness (1 point on 7-point scale)	1.63 (1.28–2.07)	1.47 (1.21–1.78)	1.04 (0.27)	<.001	−0.28 (0.06)	<.001
Depression (1 point on 7-point scale based on PRIME-MD)	1.21 (0.97–1.50)	1.10 (0.93–1.29)	2.28 (0.22)	<.001	−0.03 (0.06)	.54
Self-esteem (1 point on 7-point scale)	0.97 (0.78–1.20)	0.81 (0.69–0.95)	−0.04 (0.22)	.87	−0.02 (0.05)	.77
Stress (1 point on 4-point scale)	1.18 (0.86–1.63)	1.07 (0.85–1.34)	−0.12 (0.32)	.71	−0.05 (0.08)	.49
Grades (1 point on self-reported 4-point scale)	0.59 (0.39–0.88)	0.90 (0.65–1.23)	−0.26 (0.46)	.56	0.14 (0.11)	.21
Sibling smoking (yes vs no)	1.73 (1.03–2.89)	1.20 (0.75–1.91)	−0.79 (0.66)	.23	−0.28 (0.16)	.07
Parent smoking (yes vs no)	2.01 (1.24–3.26)	1.09 (0.75–1.60)	1.10 (0.54)	.04	−0.52 (0.13)	<.001
Friend smoking (yes vs no)	15.18 (5.84–39.41)	4.02 (2.75–5.88)	3.26 (0.57)	<.001	−0.84 (0.14)	<.001

Abbreviations: CI, confidence interval; PRIME-MD, Primary Care Evaluation of Mental Disorders questionnaire; SES, socioeconomic status; SML, smoking media literacy.

^*Boldface values are statistically significant at P<.05.

^†Defined as smoking at least once in the past 30 days.

^‡Defined by at least 1 positive response on the reliable and valid 3-item scale of Pierce et al.³¹

COMMENT

This study shows that SML is a construct that can be adequately measured with a Likert-type scale with promising reliability and validity. Internal consistency of the developed scale is excellent (Cronbach α = 0.87). Content validity appears strong because scale items were based on a carefully developed framework integrating the most accepted models of media literacy and because the resultant scale contains items representing each of the framework’s core concepts. In addition, the associations noted in this study seem to support the scale’s concurrent criterion validity. As would be expected, those with higher media literacy were less likely to smoke, less susceptible to smoking, less likely to have positive attitudes toward smoking, and more likely to have antismoking norms. Nearly all of the individual SML items were significantly associated with reduced susceptibility to smoking, and 9 of 18 were significantly associated with reduced current smoking, even attitudinally neutral statements such as “People are influenced by advertising.”

After controlling for major known covariates of smoking, SML retained its significant relationship with smoking, intention to smoke, and attitudes, but not smoking norms. There are several potential reasons for this finding. First, media literacy may indeed have less of an association with smoking-related norms than has been hypothesized. Second, this may signify a weakness in the SML scale’s ability to fully capture the intended construct of media literacy. Third, the mean for the antismoking norms was relatively high, so an association might be lacking because the distribution was limited or skewed. Finally, it is possible that our measurement of smoking norms was not ideal. Although we selected relevant items from a scale shown to have acceptable reliability and validity,³³ we did not include all items from the original scale. It is also possible that even the full Fishbein-Ajzen-Hansen questionnaire does not completely assess the construct of “smoking norms” because it includes items assessing the subject’s sense of approval of smoking solely by parents (father and mother) and peers (best friend, friends, romantic partner). Because youth behavior is likely to be influenced not only by parents and peers but also by coaches, mentors, teachers, actors, sports stars, and other important public figures, it may be necessary to develop amore comprehensive smoking norms scale based on the model of the successful Fishbein-Ajzen-Hansen questionnaire.

These findings suggest that media literacy may be a useful intervention with regard to tobacco control. In this adolescent population, an increase of 1 point on the 10-point SML scale was associated with a 22% decrease in the odds of being a smoker and a 31% decrease in the odds of being susceptible to smoking, even after controlling for multiple known smoking covariates. Indeed, as shown by logistic regression, the association of smoking with media literacy was stronger than the association of smoking with many other factors thought to be important predictors of smoking, such as knowledge of the harm and addictiveness of smoking, depression, self-esteem, socioeconomic status, responsive parenting, demanding parenting, and stress level. This would imply that SML may be an important part of comprehensive tobacco control interventions, especially since it is feasible and teachable.

This research had several limitations. First, the study population was drawn from a single large high school and was largely homogeneous in terms of its racial and ethnic background. This scale should therefore be tested and these results confirmed in more diverse populations. Baseline values for smoking and susceptibility approximate previously reported values, however.^31,32 For instance, the Monitoring the Future study³⁶ recently reported that 16% of 10th-grade students and 25% of 12th-grade students reported current (30-day) smoking. These values are similar to the overall 19% rate we found among all 9th through 12th graders. Also, our study addressed only content (face) and concurrent criterion validity, so it will be particularly important to assess construct validity of the scale in the future. Although there is currently no gold standard for measuring an individual’s media literacy, there are accepted scales used in educational settings that could be adapted for smoking-specific media.³⁷ Students’ scores on these measures could be compared with SML scores to support or weaken this scale’s construct validity. It will also be important to confirm these findings in a longitudinal setting. Although a cross-sectional study can show concurrent associations between SML and smoking, the more clinically relevant question remains to be answered: whether individuals with different levels of media literacy take up smoking at different rates. This question could ideally be answered with a prospective cohort study.

Given the substantial exposure of adolescents to mass media messages, many of which have been shown to successfully promote smoking, it is not surprising that organizations such as the American Academy of Pediatrics and the Centers for Disease Control and Prevention recommend media literacy—the systematic assessment and evaluation of mass media messages—to buffer the impact of mass media messages on adolescent smoking.^18,22 To evaluate such programs with appropriate rigor, however, it was necessary to develop a scale measuring the construct of SML in adolescents. This scale seems to have strong psychometric properties, and its association with theoretically derived markers of smoking suggests the potential utility of SML as an intervention in this population.