Media Literacy Interventions: A Meta-Analytic Review

Abstract

Although numerous media literacy interventions have been developed and delivered over the past 3 decades, a comprehensive meta-analytic assessment of their effects has not been available. This study investigates the average effect size and moderators of 51 media literacy interventions. Media literacy interventions had positive effects (d=.37) on outcomes including media knowledge, criticism, perceived realism, influence, behavioral beliefs, attitudes, self-efficacy, and behavior. Moderator analyses indicated that interventions with more sessions were more effective, but those with more components were less effective. Intervention effects did not vary by the agent, target age, the setting, audience involvement, the topic, the country, or publication status.

Harmful effects of the media have been documented in a range of domains, including violence (e.g., Paik & Comstock, 1994), sexual behavior (e.g., Allen, D’Alessio, & Brezgel, 1995; Hestroni, 2007), and body image (e.g., Holmstrom, 2004). Media literacy interventions refer to education programs designed to reduce harmful effects of the media by informing the audience about one or more aspects of the media, thereby influencing media-related beliefs and attitudes, and ultimately preventing risky behaviors.

Several studies have reviewed the effects of media literacy interventions on topics such as violence (Cantor & Wilson, 2003), sexual behavior (Allen, D’Alessio, Emmers, & Gebhardt, 1996), and advertising (Livingstone & Helsper, 2006). A review by Bergsma and Carney (2008) examined the contexts and process of effective media literacy interventions focusing on health promotion. However, a comprehensive meta-analysis of media literacy interventions is not yet available. The present study assesses the average effect size of media literacy interventions and the conditions under which such interventions are more effective. We begin with a brief overview of the conceptual basis for media literacy interventions, followed by a review of outcomes and moderators of media literacy intervention effects.

Media Literacy Interventions

Conceptual Basis

Scholars have advanced divergent conceptualizations of media literacy and have failed to reach a broad consensus on the definition of media literacy.¹ However, they generally agree that media literacy centers on specific knowledge and skills that can help critical understanding and usage of the media (e.g., Hobbs, 1998; McCannon, 2009; Martens, 2010). For example, Aufderheide (1993) has defined media literacy as “the ability to access, analyze, evaluate, and communicate messages in a variety of forms,” and has suggested that the fundamental objective of media literacy is to help audiences maintain “critical autonomy in relationship to all media.” Silverblatt (2001) defined media literacy as a “critical thinking skill that allows audiences to develop independent judgments about media content.” Critical thinking is relevant not only to receiving messages but also to producing meaning (Kellner & Share, 2005) and “learning to create one’s own messages” (Hobbs, 1998).

Some scholars have further differentiated the types of media literacy. For example, Meyrowitz (1998) classified media literacy into content, grammar, and medium literacy. Content literacy concerns knowledge of ideas and values represented in media messages. The knowledge that media content represents embedded values and points of view is central to the concept of media literacy. Grammar literacy focuses on knowledge of the techniques used in textual and visual messages, such as angles, cuts, zooms, and juxtaposition. Medium literacy concerns knowledge of different characteristics of the media. For example, some media are unisensory whereas others are multisensory. In addition, structure literacy concerns knowledge of the context of media production and consumption, such as the role of media institutions in the production of media messages (Lewis & Jhally, 1998). Knowledge of how commercial media differ from public media can be a form of structure literacy.²

We view that media literacy intervention aims to enhance criticism by increasing knowledge of the media, awareness of the influence of the media, and the ability to assess the realism of the media representation of reality. By doing so, media literacy interventions are expected to reduce the impact of the media on audiences’ beliefs, attitudes, norms, and behaviors. This meta-analysis examines the extent to which extant media literacy interventions have been able to influence these outcomes.

Theoretical Perspectives

Prior media literacy studies have examined various outcomes, some of which are relevant to understanding of the media and others are relevant to behaviors. A broad range of outcomes were examined in the studies because they were based on various theoretical frameworks. Inoculation theory (McGuire, 1964) has provided a rationale for media literacy education.³ The theory consists of threat and refutational preemption components that can protect the audience against subsequent attacks. Thus, by providing knowledge and skills to refute media messages, literacy interventions may help audiences to resist the influence of harmful media content.

Although inoculation theory was not developed specifically for media literacy education, some models were. For example, Potter’s (2004) cognitive processing model of media literacy suggests that knowledge structures and skills are fundamental to the acquisition of media literacy. Knowledge structures are sets of organized information in an individual and provide the contexts with which individuals make sense of media messages. Skills include those relevant to analysis, evaluation, grouping, induction, deduction, synthesis, and abstraction. The message interpretation process (MIP; Austin et al., 2002) model is another theoretical framework specific to media literacy. The MIP model suggests that messages guide decision making through logical and affective processes such as skepticism, realism, norms, and outcome expectancies. The MIP model includes affective aspects of media-based decisions, which distinguishes it from cognitively-focused frameworks such as inoculation theory and Potter’s model. In addition, Messaris’s theory of visual persuasion (1997) is relevant to media literacy. According to Messaris, visual persuasion may be more effective than verbal persuasion because of the indexical, iconic, and syntactic characteristics of visual signs. Thus, Messaris (1997) argued that learning the grammar of visual media may be useful for resisting the influence of media messages.

Outcomes

The above conceptualizations and theories suggest a range of outcomes for media literacy interventions. Such outcomes may be classified into two broad categories: media-relevant outcomes and behavior-relevant outcomes. Austin et al.’s (2002), Messaris’s (1997), and Potter’s (2004) models suggest that interventions can influence media-relevant outcomes. One type of media-relevant outcome is knowledge, including knowledge of specific construction techniques used to persuade audiences (e.g., Harts, 1997; Hobbs & Frost, 2003) and knowledge about advertising (e.g., Buijzen, 2007). Another type is criticism, including concepts such as understanding of persuasive intent (Austin & Johnson, 1997a) and skepticism (Austin et al., 2005). The third type is influence, which refers to one’s awareness of the influence of the media on audiences. Finally, realism refers to the extent to which one believes that the portrayal of persons or events in the media corresponds with those in the real world. Previous studies have found that interventions can increase the audience’s media knowledge (Austin et al., 2005), media criticism (e.g., Austin et al., 2005), and awareness of media influence (e.g., Duran et al., 2008) while reducing the perceived realism of media messages (e.g., Austin et al., 2007).

The MIP model further suggests that interventions can influence behavior-relevant outcomes. On the basis of the theory of reasoned action (TRA; Fishbein & Ajzen, 1975) and the integrative model of behavior change (IM; Fishbein & Yzer, 2003), behavioral outcomes can be classified into behavioral beliefs, attitudes, norms, self-efficacy, and behaviors. Behavioral beliefs concern one’s perception about the outcomes of performing a behavior; attitudes refer to one’s overall evaluation of performing a behavior; norms refer to one’s perception about the behaviors and attitudes of a social reference group (e.g., friends or family); and self-efficacy refers to one’s perceived ability to perform a behavior. Previous studies have found that interventions can reduce the frequency of risky or antisocial behaviors (Austin et al., 2005) as well as behavioral intentions (Banerjee & Greene, 2007) by inducing negative attitudes (e.g., Banerjee & Greene, 2006) toward and negative behavioral beliefs (Austin et al., 2007; Gonzales et al., 2004) about such behaviors. In addition, media literacy interventions can reduce the likelihood of an individual engaging in the behaviors by reducing normative pressure (Austin et al., 2005) and increasing self-efficacy (Austin et al., 2005).

Based on the above discussion, we hypothesize that media literacy interventions will increase audiences’ knowledge, criticism, awareness of the influence of the media, while reducing their perceptions of realism. Further, we hypothesize that media literacy interventions will reduce risky or antisocial behaviors, increase negative behavioral beliefs about and negative attitudes toward such behaviors, reduce normative beliefs about such behaviors, and increase self-efficacy to avoid such behaviors.

Moderators

Moderating variables in media literacy studies generally fall into three classes: agent (or source), audience, and treatment (Potter and Byrne, 2007). The agent refers to the individual who delivers the intervention, and extant media literacy interventions have typically involved teachers, peer students, and researchers. An agent can be a teacher (e.g., Dysart, 2008; Wade et al., 2003) or peer students who are trained by the researcher (e.g., Austin et al., 2005, 2007). In many studies, the researcher served as an agent playing the role of a teacher or a parent (e.g., Banerjee & Greene, 2006; Huesmann et al., 1983). However, studies typically have not directly tested whether intervention effects vary by the type of agent.

Related to the role of agent noted above is the setting in which media literacy interventions are delivered. When the agent is the teacher, media literacy interventions are typically implemented in school settings. However, when the agent is the researcher, they are implemented in various settings such as schools (e.g., Dysart, 2008), communities (e.g., Comer et al., 2008), and labs (e.g., Divsalar, 2006). The setting in which an intervention is delivered (e.g., a more naturalistic or controlled setting) may play a role in determining its effects. In this regard, we examine whether the effects of media literacy interventions are a function of the setting.

Although children or adolescents have been the primary audience of media literacy interventions (Potter, 2004), some studies have included college students or adults (e.g., Irving & Berel, 2001; Ramasubramanian & Oliver, 2007). The age of the audience may be an important factor in the effects of media literacy interventions. For example, Piaget (1952) suggested that as children move from the concrete operational stage (ages 8–12) to the formal operational stage (age 13 and above), they start to think in abstract ways. Thus, they may develop the ability to better understand media literacy education (see also Livingstone & Helsper, 2006).

The extent to which an intervention involves its audience may determine its effectiveness. For example, Banerjee & Greene (2006) found that an intervention including the production of messages is more effective than that focusing only on the analysis of messages. This suggests that interventions with active audience involvement (e.g., discussion or production activities) may be more effective than those with passive audience involvement (e.g., lessons only). Banerjee & Green view that media literacy interventions with active audience involvement can be more effective because they elicit greater mental efforts and comprehension than interventions with passive audience involvement. Thus, media literacy programs with active involvement components may be more effective than those with passive components.

As discussed above, media literacy is a multidimensional construct. Consequently, intervention treatment includes multiple types of literacy, such as content, grammar, medium, and structure literacy (Meyrowitz, 1998; Lewis & Jhally, 1998). Some interventions focused on a single type of media literacy, such as content literacy (e.g., Abelman & Courright, 1983) whereas others addressed multiple types (e.g., Dysart, 2008). However, studies have not examined whether these factors work together to produce additive effects. Thus, the present meta-analysis tests whether combining these factors would produce stronger effects of media literacy education.

The frequency of media literacy intervention sessions may lead to differences in the effects. Some media literacy interventions employed a single session (e.g., Irving et al., 1998), whereas others involved multiple sessions (e.g., Hennessey, 2008). Providing an appropriate dose of intervention sessions is considered crucial in achieving the desired outcome of an intervention (e.g., Hornik, 2002). In the present meta-analysis, we examine whether the effects of media literacy interventions vary as a function of the frequency of intervention sessions.

Media literacy interventions have addressed an increasingly wide range of topics, including alcohol, tobacco, violence, body image, sex, commercialism, and social issues. Thus, it is important to examine whether the effects of media literacy interventions vary according to the topic. Although a number of countries have adopted media literacy interventions, some countries such as the U.K., Canada, and Australia have a longer history of media literacy education than others due to social, policy, and regulatory factors (Kubey, 2003). Thus, this meta-analysis examines whether the effectiveness of media literacy interventions varies by the country (i.e., where the intervention is implemented). Finally, we will examine whether publication status (e.g., published or unpublished) determines the effects of media literacy interventions to test a possible publication bias. Thus, we examine whether the effects of media literacy interventions are moderated by the agent, the setting, audience age, audience involvement, intervention treatment, session frequency, the topic, the country, and publication status.

Method

Literature Search

Studies were searched using various databases, including Communication Abstracts, PsychInfo, PubMed, Proquest Dissertations and Theses Fulltext, and Google Scholar. The key-words used were “media literacy,” “media literacy intervention,” “media literacy curriculum,” and “media literacy program.” We also used combinations of key-words, such as “intervention,” “advertising,” and “skepticism,” to identify additional studies.⁴ We included all studies that were published before the cut-off point of December, 2009.

Of the 127 relevant studies, we selected 51 articles by excluding those that did not meet the following criteria. First, studies must have used quantitative methods. Although qualitative studies can provide an in-depth understanding of what students learn from media literacy programs, a meta-analysis should aggregate quantitative data.⁵ Second, studies must have tested the impact of a structured media literacy intervention on audiences.⁶ A typical intervention includes a program in which participants (a) learn about media production and its effects on audiences in lecture, print or video format, (b) are involved in various activities (e.g., discussions and homework) relevant to the program, or (c) produce their own media messages based on what they have learned from the program. Third, studies must have included one of the following outcomes of media literacy: knowledge, criticism, influence, realism, behavioral beliefs, attitudes, norms, self-efficacy, or behaviors.⁷ Fourth, studies must have reported statistical information required for a meta-analysis.⁸ Finally, studies had to be written in English. Fifty-one independent studies met all of the above criteria and were included in this meta-analysis.⁹

Outcomes

After selecting the list of studies that meet the above criteria, we identified a set of outcomes that were consistently included in the studies. The nine outcomes were as follows: knowledge, criticism, influence, realism, beliefs, attitudes, norms, self-efficacy, and behaviors. For knowledge, studies have typically used the term to refer to knowledge of specific construction techniques used to persuade audiences (e.g., Hobbs & Frost, 2003) as well as general knowledge of advertising (e.g., Buijzen, 2007).¹⁰ For criticism, studies have used terms such as understanding of persuasive intent (Austin & Johnson, 1997a) as well as skepticism (Austin et al., 2005). Beliefs about media myths (Pinkleton et al., 2008) and attitudes toward media messages or advertising (Austin et al., 2007) were also classified as criticism because these reflect critical perceptions of media content. For influence, studies have typically examined the extent to which respondents believed that the media can influence audiences (e.g., Duran et al., 2008). For realism, studies have typically used terms such as realism (Austin et al., 2007; Huesmann et al., 1983). For beliefs, studies used terms such as expectancy (Austin et al., 2007) and health consequences of tobacco use (Gonzales et al., 2004). For attitudes, studies have typically used the term attitudes (e.g., Banerjee & Greene, 2006), and for norms, studies have typically used the term norms (Austin et al., 2005). Most of the studies included in this meta-analysis have considered descriptive norms.¹¹ For self-efficacy, studies have typically used the term efficacy (Austin et al., 2005). Finally, for behaviors, we grouped actual behaviors (Austin et al., 2005) as well as behavioral intentions (Banerjee & Greene, 2007) into a single category.

Moderators

We examined the following moderators: the agent, the setting, target age, audience involvement, intervention treatment, session frequency, the topic, publication status, and the country. Agents were coded as teacher, peer, researcher, or other. Related to the agent, settings were coded as school, community, both (school and community), or other (e.g., a lab). Target age was coded using the mean age of the respondents in a study. Audience involvement was classified into the following categories: (1) passive interventions involving only a lecture with print or audio-visual materials, (2) interactive interventions involving various activities such as in-class discussions, role playing, and computerized games, and (3) production interventions involving the actual production of messages. Intervention treatment was operationalized as the number of media literacy components. We examined each study to determine whether it included content, medium, grammar, and structure literacy components. Then, we counted the total number of components included in a study. For example, if a study included content and grammar literacy, then we coded that there were two treatment components in the study. The number of components included in each intervention could range from one to four. All studies included at least one literacy component and some included all four. For session frequency, the total number of sessions in the intervention was coded, and the number of sessions ranged from 1 to 40 sessions (mean = 3.62, median = 2). Topics were coded as alcohol, tobacco, drugs, body image and eating, violence, sex, advertising and marketing, social issues (e.g., racism, gender role), and general. Countries included the U.S., Australia, Canada, the Netherlands, and Tanzania. Because there were only eight studies conducted outside the U.S., all non-U.S. studies were merged into a single category. Publication status was coded as either published or unpublished.

Effect Size Calculation

To compute the meta-analytic effect size, the results of all studies were converted into a common effect-size metric (d) by using the following equation:

$$d=\frac{{\mathrm{X̄}}_1-{\mathrm{X̄}}_2}{S_{\mathit{\text{pooled}}}}(S_{\mathit{\text{pooled}}}=\sqrt{\frac{s_1^2(n_1-1)+s_2^2(n_2-1)}{n_1+n_2-2}})$$

In other words, the effect size represents the difference between the treatment group (i.e., the media literacy intervention group) and the control group. When there was more than one treatment group in a study, the overall mean across treatment groups was compared with that of the control group. In addition, we included the statistics (e.g., means and standard deviations) from immediate posttests but not the statistics from the delayed posttests. This is because the time lag for the immediate posttest was comparable across studies included in this meta-analysis, whereas it was not for the delayed posttest. That is, immediate posttests were typically administered on the day of the intervention, whereas delayed posttests were conducted days to a year after the intervention. A statistical software package Comprehensive Meta-Analysis Version 2 was used for calculating effect sizes and for conducting meta-analyses.

Analysis Methods

First, the average effect size was computed across all outcome measures, including knowledge, criticism, realism, influence, beliefs, attitudes, norms, self-efficacy, and behaviors. Then the average effect size was computed for each outcome. Of the 51 independent studies, the numbers of studies reporting the effects of media literacy interventions on each outcome were as follows: 10 for knowledge, 24 for criticism, 14 for realism, 3 for influence, 7 for beliefs, 22 for attitudes, 7 for norms, 3 for self-efficacy, and 25 for behavior. The analyses were conducted using random-effects analyses based on Hedges & Olkin’s (1985) procedure.¹² Finally, analyses were conducted to test the effects of potential moderators. The moderator analyses were based on the average effect size of each study, combining all outcomes. The categorical moderators (i.e., the agent, the setting, audience involvement, the topic, publication status, and the country) were analyzed using mixed-effects categorical analyses, whereas the continuous moderators (i.e., target age, intervention treatment, and session frequency) were analyzed using meta-regression.

Results

The Average Effect Size of Media Literacy Interventions

The mean effect size of media literacy interventions, weighted by sample size, was .37 (p < .001), with a 95% confidence interval ranging from .27 to .47. This suggests that, overall, the effects of media literacy interventions were significant in the positive direction (see Table 1).

Table 1.

Summary of Moderator Analyses

Variable	K	d	95% CI	QB	p
Total set	51	0.37***	0.27, 0.47
Agent				7.08	0.07
Teacher	11	0.51**	0.22, 0.81
Peer	4	0.31*	0.05, 0.56
Researcher	24	0.42***	0.30, 0.54
Other or not specified	12	0.17*	0.004, 0.34
Setting				2.62	0.45
School	37	0.36***	0.24, 0.48
Community	3	0.54***	0.28, 0.79
Both	2	0.21	−0.11, 0.54
Other or not specified	9	0.41**	0.17, 0.65
Audience Involvement				1.79	0.62
Passive	15	0.50***	0.24, 0.75
Interactive	22	0.33***	0.24, 0.41
Production	11	0.33***	0.15, 0.52
Other or not specified	3	0.21	−0.29, 0.71
Topic				7.67	0.47
Advertising	8	0.36***	0.18, 0.55
Alcohol	3	0.42***	0.28, 0.56
Body image & Eating	16	0.28***	0.17, 0.39
Drug	1	0.27**	0.08, 0.47
General literacy	4	0.56+	−0.03, 1.14
Sex	1	0.38***	0.21, 0.55
Social	3	0.24	−0.15, 0.63
Tobacco	6	0.20*	0.04, 0.36
Violence	9	0.50**	0.16, 0.84
Country				0.80	0.37
U.S.	43	0.34***	0.24, 0.43
Non U.S.	8	0.52*	0.13, 0.92
Publication status				0.76	0.38
Published	42	0.39***	0.27, 0.50
Unpublished	9	0.29**	0.11, 0.47

Note.

⁺p < .10.

^*p < .05.

^**p < .01.

^***p < .001.

K refers to the number of independent effect sizes considered for each group.

The Average Effect Size by Outcomes

Media literacy interventions had positive effects on almost all outcome measures: knowledge (d=1.12, p < .001, 95% CI: .77 to 1.47), criticism (d=.29, p < .001, 95% CI: .20 to .38), realism (d=.54, p < .001, 95% CI: .24 to .84), influence (d=.60, p < .05, 95% CI: .09 to 1.11), beliefs (d=.23, p < .001, 95% CI: .12 to .35), attitudes (d=.28, p < .001, 95% CI: .17 to .39), self-efficacy (d=.34, p < .001, 95% CI: .18 to .50), and behaviors (d=.23, p < .001, 95% CI: .15 to .31). Thus, our hypothesis that media literacy interventions increase audiences’ knowledge of the media, criticism of the media, awareness of the influence of the media, while reducing media realism was supported. In addition, the hypothesis that media literacy interventions reduce risky or antisocial behaviors, increase negative behavioral beliefs about and negative attitudes toward such behaviors, and increase self-efficacy to avoid such behaviors was supported. However, the effect on norms (d=.18, p = .08, 95% CI: −.02 to .39) was not supported.

Moderators

For the categorical moderators, Table 1 provides a list of effect sizes for each categorical group. The effects of media literacy interventions did not vary according to the agent, the setting, audience involvement, the topic, the country, or publication status.

For the continuous moderators, the meta-regression results indicate that session frequency (b = 0.008, SE b = 0.002, p < .001) and intervention treatment (b = −0.05, SE b = 0.02, p < .05) were significant moderators. For session frequency, studies with more intervention sessions reported larger effect sizes. For intervention treatment, however, the direction was somewhat unexpected. Studies with more intervention treatment components reported smaller effects. Target age (b = 0.001, SE b = 0.01, p = .86) was not a significant predictor.

Discussion

This meta-analysis contributes to the media literacy literature by a) providing a summary of the general effects of media literacy interventions, b) examining the effects of media literacy interventions on different types of outcomes (i.e., media-relevant vs. behavior-relevant), and c) specifying the moderators that influence the effect size of media literacy interventions.

First, media literacy interventions were generally effective (d=.37). Media literacy interventions had positive effects on most of the outcomes considered in this meta-analysis, although the effect on norms was marginally significant. The results suggest that it is possible to employ media literacy interventions to address the harmful effects of mass media. The effects of the media on risky behaviors (e.g., violence, smoking, and underage drinking) have long been criticized. The results of this study suggest that media literacy interventions may be an effective approach for reducing potentially harmful effects of media messages. Intervention effects were found across divergent topics for diverse audiences, for a broad range of media-related (e.g., knowledge) and behavior-related (e.g., attitudes and behaviors) outcomes. The results that intervention effects did not vary according to target age, the setting, audience involvement, and the topic suggest that interventions can be equally effective across a spectrum of settings (e.g., school, community, or lab), age groups, levels of audience involvement, and topics (e.g., alcohol, violence, and sex). Media literacy interventions is particularly important with the development of social media because the quality of information circulated through social media (e.g., Twitter) is not guaranteed, and thus, audiences’ literacy has become more important than ever.

Second, the present study clarified a list of outcomes that can be influenced by media literacy interventions based on a review of a number of theoretical frameworks relevant to media literacy. Although this meta-analysis was not able to conduct a direct test of differences in the effect size by the type of outcome, the results indicate that media literacy interventions have differential effects on the two types of outcomes: media-relevant outcomes and behavior-relevant outcomes. That is, media literacy interventions may have greater effects on media-relevant outcomes (e.g., knowledge and realism) than on behavior-relevant outcomes (e.g., attitudes and behaviors). This may be because media literacy interventions focus more on media-related content than on behavior-related content. Because most media literacy interventions have focused on enhancing critical thinking rather than on inducing behavior change, media literacy interventions may have greater impact on media-related outcomes. Further, this may be explained by the idea that media-relevant outcomes are more proximal or immediate outcomes of media literacy interventions, whereas behavior-relevant outcomes are more distal outcomes. Proximal outcomes are those that are more immediately influenced by media literacy interventions, whereas distal outcomes are those that are more remotely influenced by interventions.¹³ Thus it is not surprising that media literacy interventions will have stronger effects on media-related outcomes than on behavior-relevant outcomes. Future media literacy interventions may benefit by actively combining and integrating media-related educational components and behavior-related educational components (see Primack, Fine, Yang, Wickett, & Zickmund, 2009).

Finally, the present meta-analysis identified the moderators that influence the effect size of media literacy interventions. The results of the moderator analyses indicated that the impact of media literacy interventions increased as a function of the number of sessions, suggesting that media literacy interventions are more likely to be successful when the program is reinforced through multiple sessions. This result is consistent with Hornik’s (2002) argument regarding campaign effects based on the dose-response relationship.

The findings of this study suggest areas of future research. In terms of intervention treatment, the results are somewhat unexpected. In general, interventions with fewer components were more effective than those with more. There can be at least two possible explanations for this finding. One explanation is that interventions with more components are likely to contain too much information, resulting in information overload and information loss. This can be explained by the limited capacity model of information processing (Lang, 2000). Another possible explanation is that interventions with more components are more likely to be confusing, particularly to young audiences who are less likely to be cognitively sophisticated. If so, a simple and more focused education program may enhance the effects of a media literacy intervention. Thus, future research should examine the effects of media literacy interventions by varying the number of treatment components and considering the audience’s age.

The agent was not a significant moderator. In other words, the effect size of the intervention did not vary by the agent. Existing evidence on the role of agents in intervention effects is not entirely consistent. Some studies have found that experts are more effective than nonexperts (Durantini, Albarracin, Mitchell, Earl, & Gillette, 2006), while others suggest that peers are more effective than nonpeers (Webel, Okonsky, Trompeta, & Holzemer, 2010). Expert-led interventions may be more effective because of their knowledge, experience, and authority, whereas nonexpert-led interventions may be more effective because of perceived similarity and identification. The effect of agents in media literacy interventions may be clarified when future interventions directly compare the effects delivered by experts and peers.

This meta-analysis has some limitations. First, some unpublished studies, such as conference papers, were not included in the analysis. However, the average effect size for published studies was not larger than that for unpublished ones, indicating no publication bias. In addition, Rosenthal’s fail-safe N for this study was 4,688, which indicates the number of missing studies required for making the results of this meta-analysis nonsignificant. This suggests that it is unlikely that our findings are due to publication bias.¹⁴ Second, some studies were excluded from this meta-analysis because of the lack of necessary statistical information.

Overall, this study suggests that media literacy interventions are effective. Positive effects of media literacy interventions were observed across diverse agents, target age groups, settings, topics, and countries. Future research could further examine the specific types of intervention treatments that can enhance media-relevant and behavior-relevant outcomes.

Figure 1.

A forest plot of effect sizes and 95% confidence intervals for each study