Mass media interventions for reducing mental health‐related stigma

Abstract

Background

Mental health‐related stigma is widespread and has major adverse effects on the lives of people with mental health problems. Its two major components are discrimination (being treated unfairly) and prejudice (stigmatising attitudes). Anti‐stigma initiatives often include mass media interventions, and such interventions can be expensive. It is important to know if mass media interventions are effective.

Objectives

To assess the effects of mass media interventions on reducing stigma (discrimination and prejudice) related to mental ill health compared to inactive controls, and to make comparisons of effectiveness based on the nature of the intervention (e.g. number of mass media components), the content of the intervention (e.g. type of primary message), and the type of media (e.g. print, internet).

Search methods

We searched eleven databases: the Cochrane Central Register of Controlled Trials (CENTRAL, The Cochrane Library, Issue 7, 2011); MEDLINE (OvidSP),1966 to 15 August 2011; EMBASE (OvidSP),1947 to 15 August 2011; PsycINFO (OvidSP), 1806 to 15 August 2011; CINAHL (EBSCOhost) 1981 to 16 August 2011; ERIC (CSA), 1966 to 16 August 2011; Social Science Citation Index (ISI), 1956 to 16 August 2011; OpenSIGLE (http://www.opengrey.eu/), 1980 to 18 August 2012; Worldcat Dissertations and Theses (OCLC), 1978 to 18 August 2011; metaRegister of Controlled Trials (http://www.controlled‐trials.com/mrct/mrct_about.asp), 1973 to 18 August 2011; and Ichushi (OCLC), 1903 to 11 November 2011. We checked references from articles and reviews, and citations from included studies. We also searched conference abstracts and websites, and contacted researchers.

Selection criteria

Randomised controlled trials (RCTs), cluster RCTs or interrupted time series studies of mass media interventions compared to inactive controls in members of the general public or any of its constituent groups (excluding studies in which all participants were people with mental health problems), with mental health as a subject of the intervention and discrimination or prejudice outcome measures.

Data collection and analysis

Two authors independently extracted data and assessed the risk of bias of included studies. We contacted study authors for missing information. Information about adverse effects was collected from study reports. Primary outcomes were discrimination and prejudice, and secondary outcomes were knowledge, cost, reach, recall, and awareness of interventions, duration/sustainability of media effects, audience reactions to media content, and unforeseen adverse effects. We calculated standardised mean differences and odds ratios. We conducted a primarily narrative synthesis due to the heterogeneity of included studies. Subgroup analyses were undertaken to examine the effects of the nature, content and type of mass media intervention.

Main results

We included 22 studies involving 4490 participants. All were randomised trials (3 were cluster RCTs), and 19 of the 22 studies had analysable outcome data. Seventeen of the studies had student populations. Most of the studies were at unclear or high risk of bias for all forms of bias except detection bias.

Findings from the five trials with discrimination outcomes (n = 1196) were mixed, with effects showing a reduction, increase or consistent with no evidence of effect. The median standardised mean difference (SMD) for the three trials (n = 394) with continuous outcomes was ‐0.25, with SMDs ranging from ‐0.85 (95% confidence interval (CI) ‐1.39 to ‐0.31) to ‐0.17 (95% CI ‐0.53 to 0.20). Odds ratios (OR) for the two studies (n = 802) with dichotomous discrimination outcomes showed no evidence of effect: results were 1.30 (95% CI 0.53 to 3.19) and 1.19 (95% CI 0.85 to 1.65).

The 19 trials (n = 3176) with prejudice outcomes had median SMDs favouring the intervention, at the three following time periods: ‐0.38 (immediate), ‐0.38 (1 week to 2 months) and ‐0.49 (6 to 9 months). SMDs for prejudice outcomes across all studies ranged from ‐2.94 (95% CI ‐3.52 to ‐2.37) to 2.40 (95% CI 0.62 to 4.18). The median SMDs indicate that mass media interventions may have a small to medium effect in decreasing prejudice, and are equivalent to reducing the level of prejudice from that associated with schizophrenia to that associated with major depression.

The studies were very heterogeneous, statistically, in their populations, interventions and outcomes, and only two meta‐analyses within two subgroups were warranted. Data on secondary outcomes were sparse. Cost data were provided on request for three studies (n = 416), were highly variable, and did not address cost‐effectiveness. Two studies (n = 455) contained statements about adverse effects and neither reported finding any.

Authors' conclusions

Mass media interventions may reduce prejudice, but there is insufficient evidence to determine their effects on discrimination. Very little is known about costs, adverse effects or other outcomes. Our review found few studies in middle‐ and low‐income countries, or with employers or health professionals as the target group, and none targeted at children or adolescents. The findings are limited by the quality of the evidence, which was low for the primary outcomes for discrimination and prejudice, low for adverse effects and very low for costs. More research is required to establish the effects of mass media interventions on discrimination, to better understand which types of mass media intervention work best, to provide evidence about cost‐effectiveness, and to fill evidence gaps about types of mass media not covered in this review. Such research should use robust methods, report data more consistently with reporting guidelines and be less reliant on student populations.

Plain language summary

Mass media interventions for reducing stigma towards people with mental health problems

People define stigma in various ways. In this review we focus on two key aspects of stigma: discrimination (treating people unfairly because of the group they belong to) and prejudice (negative attitudes and emotions towards certain groups). People with mental health problems often experience stigma. It can have awful effects on their lives. Mass media are media that are intended to communicate with large numbers of people without using face‐to‐face contact.  Examples include newspapers, billboards, pamphlets, DVDs, television, radio, cinema, and the Internet. Anti‐stigma campaigns often include mass media interventions, and can be expensive, so it is important to find out if the use of mass media interventions can reduce stigma.

We reviewed studies comparing people who saw or heard a mass media intervention about mental health problems with people who had not seen or heard any intervention, or who had seen an intervention which contained nothing about mental ill health or stigma. We aimed to find out what effects mass media interventions may have on reducing stigma towards people with mental health problems.

We found 22 studies involving 4490 people.  Five of these studies had data about discrimination and 19 had data about prejudice. We found that mass media interventions may reduce, increase, or have no effect on discrimination. We found that mass media interventions may reduce prejudice. The amount of the reduction can be considered as small to medium, and is similar to reducing the level of prejudice from that associated with schizophrenia to that associated with major depression. The quality of the evidence about discrimination and prejudice was low, so we cannot be very certain about these findings. Only three studies gave any information about financial costs and two about adverse affects, and there were limitations in how they assessed these, so we cannot draw conclusions about these aspects.

Summary of findings

for the main comparison.

Mass media compared with inactive control for reducing mental health‐related stigma
Patient or population: General public or any of its constituent groups (excluding groups comprising solely of people with mental ill health) Settings: Any Intervention: Mass media Comparison: Inactive control
Outcomes	Relative effect (95% CI)7	No of Participants (studies)	Quality of the evidence (GRADE)	Comments
Discrimination (i) not registering to attend focus group with people with schizophrenia (immediate)1 (ii) not visiting sheltered workshop for people with mental illness (9 months)2 (iii) distance placed between chairs when anticipating meeting with person with Tourette's syndrome3	ORs (802 participants, 2 studies) (i) 1.30 (0.53 to 3.19) (i) 1.19 (0.85 to 1.65) SMDs (394 participants, 3 studies) (iii) median SMD ‐0.25 Range of SMDs ‐0.85 (‐1.39 to ‐0.31) to ‐0.17 (‐0.53 to 0.20)	1196 (5)	⊕⊕⊝⊝ low	Quality of evidence downgraded for study limitations (risk of bias) and indirectness of measures (see Quality of the evidence for full details)
Prejudice Multiple scales4 Follow‐up (i) Immediate5 (ii) 1 week to 2 months6 (iii) 6 to 9 months7	Median SMD (i) ‐0.38 (ii) ‐0.38 (iii) ‐0.49 Range of SMDs ‐2.94 (‐3.52 to ‐2.37) to 2.40 (0.62 to 4.18) Median SMDs are equivalent to reductions of 0.22, 0.22 and 0.29 points on the 1 to 4 point SDS scale [7], which is equivalent to reducing the level of prejudice from that associated with symptoms of schizophrenia to the level associated with symptoms of major depression8	3176 (19)	⊕⊕⊝⊝ low	Quality of evidence downgraded for study limitations (risk of bias) and indirectness of populations (see Quality of the evidence for full details)
Cost (i) Audiovisual public service announcement (ii) a) CD‐ROM b) printed manual (iii) magazine article	Relative costs (i) 100 US dollars (equivalent to £64 GBP) vs. nil (ii) a) 35,000 Australian dollars (£22,404 GBP) for 250 e‐learning CDs @ 140 dollars per CD (ii) b) 7,140 Australian dollars(£4,570 GBP) for 238 manuals @ 30 dollars per manual vs. nil (iii) 'printing costs' vs 'printing costs'	416 (3)	⊕⊝⊝⊝ very low	Quality of evidence downgraded for inconsistency of results, imprecision, and 'other' (data only being available on request, lack of cost‐effectiveness data) (see Quality of the evidence for full details)
Unforeseen adverse effects	Statements in two studies: (i) ‘Given that this was an educational intervention with a non‐clinical sample, there was no formal enquiry about adverse events. Informally, no adverse events were reported'; (ii) ‘No adverse effects, such as an increase in stigma as a result of the intervention, were identified’	2 (455)	⊕⊕⊝⊝ low	Quality of evidence downgraded for study limitations (risk of bias) and imprecision (see Quality of the evidence for full details)
CI: Confidence interval; OR: Odds ratio; SMD: Standardised mean difference
GRADE Working Group grades of evidence High quality: Further research is very unlikely to change our confidence in the estimate of effect. Moderate quality: Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate. Low quality: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate. Very low quality: We are very uncertain about the estimate.

[1] Penn 2003

[2] Yoshida 2002

[3] Woods 2002; Woods 2003; Woods 2005

[4] 28 different measures were used (see Characteristics of included studies)

[5] Brown 2010; Bunn 2009; Coleman 2005; Corrigan (submitted); Finkelstein 2008; Iobst 2008; Matthews 2009; Morgan Owusu 2002; Penn 2003; Smith 2007; Woods 2002; Woods 2003; Woods 2005; Yoshida 2002

[6] Russell 1988; Demyan 2009; Brown 2010; Jorm 2010a

[7] Yoshida 2002; Finkelstein 2008; Jorm 2010a [OR:1 indicated decreased stigma, > 1 indicates increased stigma; SMD < 0 indicates decreased stigma, > 0 indicates increased stigma]

[8] Link 1999 This reference for the scale also reports a population standard deviation of 0.59 for social distance in relation to schizophrenia using the SDS from the observational study, General Social Survey 1996, USA.  It gives SDS scores for different mental health conditions which are used for further interpretation.

Background

Description of the condition

Stigma has been defined and conceptualised in a number of different ways. The conceptual framework used in this review is that stigma comprises ignorance (lack of knowledge), prejudice (stigmatising attitudes) and discrimination (being treated unfairly, a behaviour concept) (Thornicroft 2007). Our review focused on the latter two concepts: prejudice and discrimination, with knowledge as a secondary outcome. This is because what constitutes de‐stigmatising knowledge is a contested issue. For example, Slade has described the paradigmatic arguments about fundamental ways (clinical models, disability models, diversity models) to understand experiences labelled as mental illness, and highlights the lack of agreement over these (Slade 2009). There is also significant disagreement between different professional groups about what treatments are helpful (Lauber 2005). Our focus on discrimination and prejudice was also because these are central to most conceptualisations of stigma, e.g. (Link 2001a, Corrigan 2005), but these other models only contain one aspect of knowledge, namely stereotype awareness. A further pragmatic consideration was that because the review covered interventions which did not have a stated aim of reducing stigma, having a stigma‐related outcome was a key aspect in inclusion decisions. Whilst prejudice and discrimination are always stigma‐related, knowledge is a much broader concept and may or may not be related to stigma, and therefore would have been problematic as a primary outcome.

Discrimination and prejudice were relevant concepts for this review because they focus on stigmatisers (the targets of the mass media interventions reviewed here) rather than stigmatised people. Some commentators focus on aspects of prejudice, viewing stigma as a social process of 'othering', blaming and shaming (Deacon 2006), whereas others have argued for a purely discrimination‐based conceptual framework (Sayce 1998). Phelan and colleagues have investigated the possible similarity between the concepts of stigma and prejudice, and concluded that the two models have much in common, with most differences being a matter of focus and emphasis (Phelan 2008). Discrimination and prejudice are key elements in Rüsch's discussion of Link's (Link 2001a) conceptualisation of the stigma process as labelling, separation, stereotype awareness, stereotype endorsement, prejudice, and discrimination in a context in which social, economic, or political power is exercised (Rüsch 2005). Discrimination and prejudice are also core elements in Corrigan's framework (Corrigan 2005). In this review, in line with the Thornicroft 2007 model, we used the term 'discrimination' to refer to behavioural elements such as observed discriminatory behaviour and discrimination experiences reported by people with mental health problems, although we recognise that discrimination can also operate at the structural level, for example in discriminatory media reporting, policy and legislation (Corrigan 2004c). Following the same model, we used the term 'prejudice' to encompass concepts such as attitudes towards, stereotypes about, emotional reactions to, and desire for social distance from, people with mental ill health.

Mental health‐related stigma is widespread. A recent survey of public reactions to case descriptions of people with schizophrenia and major depression, involving nationally‐representative samples in 15 countries in Africa, Asia, Australasia, Europe, and in both North and South America, found significant levels of public stigma in all countries studied, although there was some variation between them (Pescosolido 2009). A US study using the same methodology found that in 2006, 62% of the public reported being unwilling to work closely with people with schizophrenia, and 52% were unwilling to socialise with them (Pescosolido 2010). The figures for depression were 47% and 30% respectively (Pescosolido 2010).  Furthermore, some studies have reported a worsening of certain attitudes in recent years (Angermeyer 2005; Mehta 2009). A 2009 study investigating the discrimination experiences of 739 people with schizophrenia in 27 countries found that negative discrimination was experienced by 47% in making or keeping friends, by 43% from family members, by 29% in finding a job, 29% in keeping a job, and by 27% in intimate or sexual relationships (Thornicroft 2009). Stigma can be compounded by other axes of difference. For example people with mental ill health who belong to other groups facing stigma and discrimination, such as those from black and ethnic minority groups, lesbian and gay individuals, and asylum seekers, may be particularly disadvantaged (e.g. Gary 2005). Furthermore, both mental ill health itself and mental health‐related discrimination and prejudice can make people more likely to become members of other groups subject to stigma, such as those experiencing homelessness, unemployment and poverty.

Stigma has major adverse effects on the lives of people with mental health problems (McDaid 2008). Public attitudes commonly include stereotypes of incompetence, beliefs about dangerousness, attributions of blame, expectations of poor prognosis, negative emotional responses, and a desire for social distance (Hinshaw 2000). Each of these can directly affect the well‐being and quality of life of people with mental ill health. People with mental health problems experience significant discrimination which spans all major domains of life (Thornicroft 2006; Thornicroft 2009) and includes exclusion from employment (Stuart 2006b) with consequent poverty, negative impacts on intimate relationships and parenting (Hinshaw 2005), reduced access to and engagement with mental health services (Corrigan 2004b), and poorer physical health care (Jones 2008). Discrimination and prejudice can also have significant negative effects on the way that people with mental ill health feel about themselves, such as inducing internalised stigma (Corrigan 2002b; Ritsher 2003). In addition, the anticipation of discrimination can lead people to use strategies of avoidance and concealment, which may further contribute to social exclusion and poor quality of life (Thornicroft 2009). Mental health‐related stigma also affects families and others close to the person with mental ill health, and these people can experience 'courtesy stigma' or 'stigma by association' (Corrigan 2004a). In addition stigma has damaging effects at the societal level, robbing the community of the contributions that people with mental ill health could make were it not for stigma, and helping to maintain fear about mental illness (Corrigan 2005). Negative media reporting ‐ a form of discrimination in itself ‐ also shapes attitudes and influences behaviour, thereby producing or reinforcing stigma (Wahl 1995).

Description of the intervention

Mass media has the potential to de‐stigmatise as well as to stigmatise (Philo 2010). This review focused on mass media interventions, rather than on other types of intervention, because such interventions are able to reach large numbers of people and so have the potential for achieving population‐level change. Large scale change may be difficult with other types of intervention.

Following Bala 2008 and Brinn 2010, we defined mass media as channels of communication intended to reach large numbers, which are not dependent on person‐to‐person contact. A mass media intervention is one that uses such channels. There are many different forms of mass media, for example: print (e.g. newspapers, magazines, billboards, pamphlets, flyers, coasters); recordings (e.g. audio cassettes, videos, CDs, DVDs); radio; television; cinema; mobile phones (e.g. mobile device applications); and the Internet (e.g. websites, blogs, podcasts, viral messaging, social networking sites) (Donovan 2003).

Not all mass media interventions that may reduce stigma have an explicit intention to do so.  Examples may include the positive portrayal of a person with a mental health problem on television without a planned intention, or media coverage of a celebrity’s diagnosis with a mental illness.  Some health promotion campaigns may also reduce stigma, even though this is not their primary purpose.

Interventions vary in the extent to which they target particular groups. Some are directed at the general population and some are targeted at specific groups, for example young people or employers. Mass media interventions may come from various sources, including governments, community groups and organisations. An intervention may focus on stigma in relation to mental ill health in general, a specific mental health condition, or all forms of disability including mental health disabilities. Interventions may be based, implicitly or explicitly, on diverse conceptualisations of stigma or mental health problems, and may use different theories to underpin the design of the interventions (see How the intervention might work). Interventions sometimes take place at a single time point, or may be short‐term or sustained over a long period. Furthermore they vary in intensity (e.g. extent and frequency of advertising) and reach (e.g. proportion of intended population who see the advertisements).

How the intervention might work

In many respects, mass media interventions to combat stigma work using the same mechanisms operating in advertising and marketing. When these techniques are applied to address social issues rather than to sell commercial products or to promote a particular organisation, this is referred to as social marketing (Donovan 2003). However, it is recognised that social and commercial marketing differ in significant ways, most markedly in that the attitudes and behaviours which social marketing seeks to change are often more complex and hence more challenging to change than commercial behaviour (Donovan 2003).

Social marketing draws on several models of communication and persuasion, and uses various behaviour change theories. A number of these derive from, or overlap with, those from the health psychology, social psychology, public health or health promotion fields. Some of the major theories include: the theory of reasoned action; the health belief model; the transtheoretical (stages of change) model; the theory of planned behaviour; social learning theory; the Rossiter‐Percy motivational model; the diffusion theory model; and the elaboration likelihood model (Donovan 2003; Noar 2006). Symbolic communication and modelling are also processes thought to be important in mass media interventions (Bandura 2001). The mass media operates by potentially influencing not only individuals but also communities and policy makers (Andreasen 2006).

It is not uncommon for mass media material to contain some form of personal narrative from people who have experienced mental health problems, such as celebrities, or members of the public sharing stories about themselves and their lives. These may reduce stigma because they are an indirect form of social/interpersonal contact with people with mental health problems, and this form of contact has been theorised, and demonstrated, to reduce stigma (Couture 2003; Pettigrew 2006; Corrigan 2012). Such narratives may also reduce stigma by increasing awareness of the variation amongst members of out‐groups and in‐groups, increasing social identity complexity, and increasing tolerance (Schmid 2009). Alternatively, narratives may act as 'mediated associations' in which an individual feels empathy towards the suffering of another without the other's physical presence, elicited through language (stories, films) or pictorial representation (e.g. photographs), with this empathy then being translated into a commitment to social justice (Kumagai 2008).

Our conceptualisation does not necessarily imply a linear mode of action with changes in prejudice leading to changes in discrimination. For example, a communication which imparts the message that it is unlawful to discriminate on the basis of mental health could change behaviour (discrimination) outcomes without necessarily changing attitudes (prejudice). Social marketing theory (Donovan 2003) states the importance of including a clear call to action. In England’s national anti‐stigma programme (Henderson 2009), the current call to action is ‘It’s time to talk’ including directions to keep in touch with people with mental health problems. As the loss of friends and being shunned are common reported experiences of discrimination, this is an example of how mass media messages may directly address one form of discrimination. Additionally mass media may change perceptions of social norms, with the change in social norms leading to behavior changes, leaving individual attitudes untouched (Wakefield 2010). Furthermore, subtle factors in communication can influence social behaviour without necessarily being mediated by conscious choice (Bargh 1996) and so mass media may affect behaviour directly. It is also recognised that changes in attitudes may not necessarily translate into changes in behaviour (Marcus 1998). Equalities and human rights legislation have a significant potential to reduce discrimination (Callard 2012). However research in this area is limited and it is not currently known how legislative approaches compare to mass media approaches in their effectiveness in reducing discrimination against people with mental health problems. Protest is another approach for countering discriminatory behaviour. Corrigan has compared protest and other approaches including indirect contact (one form of mass media intervention), and found that protest‐based interventions were rarely studied, and when they were they did not yield significant reductions in stigma, whilst indirect contact significantly reduced discriminatory intentions (Corrigan 2012).

Many variables are believed to influence the effectiveness of mass media interventions, including: whether an intervention is based on formative research; whether it has a theoretical basis; the degree of targeting; campaign intensity; the media channel (Noar 2006); and the 'ad creative' (the creative design and content of the intervention). In addition, whether the mass media element is part of a multi‐faceted campaign (Link 2001b) and which particular messages are conveyed (Clement 2010) are likely to be important. Reviews of mass media interventions in other fields have reported that the duration of campaigns appears to be important, with campaigns of longer duration being more effective (e.g. Friend 2002). Furthermore, interventions that are effective in reducing stigma in high‐income countries may not necessarily be effective if exported without modification to low‐ or middle‐income countries (Rosen 2003) for reasons relating to both available resources and culture. Within one country an anti‐stigma intervention may be received differently by different ethnic groups (Glasgow Anti Stigma Partnership 2007). We took many of these variables into consideration in planning the comparisons and subgroup analyses that were undertaken, as well as the data extraction for this review.

Why it is important to do this review

Stigma is highly prevalent and has serious adverse effects on the lives of people with mental ill health (as described above). Consequently there is a need to find effective ways to reduce mental health‐related stigma. Mass media interventions are one of the most commonly used types of intervention, and they are being carried out throughout the world (Sartorius 2005; Callard 2008). National programmes aiming to reduce mental health‐related stigma and containing mass media components are taking place in a number of countries, such as New Zealand (Vaughn 2004), England (Henderson 2009) and Scotland (Dunion 2005). Local and regional interventions are also widespread. Mass media interventions can be scaled‐up with relative ease to the population‐level and hence, if effective, are a feasible intervention for large‐scale change. If mass media interventions were to produce only a small magnitude of change, this may translate into important impacts at the population level (Noar 2006). Although other types of interventions, such as direct social contact (Couture 2003), have occasionally been used on a large scale (Corrigan 2006; Evans‐Lacko 2012a), this is unusual and presents greater implementation challenges than mass media approaches.

There is a recognised evidence gap in this field (Weiss 2006; Callard 2008). This systematic review synthesises what is currently known to enable future research to be appropriately focused. Such systematic investigation provides guidance for those who are planning initiatives, about whether mass media interventions are worthwhile; about optimal intervention design; and about any possible harm. As mass media interventions may be expensive (Austin 1998), evidence of ineffectiveness will free anti‐stigma resources for other approaches.

A number of non‐systematic reviews of mass media and other interventions to reduce mental health‐related stigma have been undertaken, (for example Warner 2001; Pinfold 2005; Rüsch 2005; Sartorius 2005; Warner 2005; Callard 2008; Hinshaw 2008; McDaid 2008; Thornicroft 2008). Recently four systematic reviews of interventions to reduce mental health‐related stigma have been conducted, but none focus on mass media interventions (Holzinger 2008; Schachter 2008; Yamaguchi 2011; Corrigan 2012). Our review adds to the growing body of systematic review evidence about the effectiveness of mass media interventions in other fields (Grilli 2002; Vidanapathirana 2005; Bala 2008; Brinn 2010). The systematic review of mass media anti‐stigma interventions in mental ill health is likely to create a greater understanding of this vital area, and to help to underpin the development of future population‐level interventions to combat mental health‐related stigma.

Objectives

The primary objective was to assess the effects of mass media interventions on reducing stigma related to mental ill health in terms of discrimination and prejudice compared to inactive controls. The secondary objective was to make comparisons of effectiveness based on the nature of the intervention (e.g. number of mass media components), the content of the intervention (e.g. type of primary message), and the type of media (e.g. print, internet).

Methods

Criteria for considering studies for this review

Types of studies

Two types of study were eligible: randomised controlled trials (RCTs), including cluster trials; and interrupted time series (ITS) analyses. In ITS studies the intervention required a defined start and end point, and at least three data points before the intervention was introduced and at least three after its end point.

RCTs were selected as these provide the strongest level of evidence on effectiveness. ITS analyses were included because this study design is commonly used to assess the effectiveness of mass media interventions (Grilli 2002; Vidanapathirana 2005). The specific criteria for ITS studies were based on Cochrane Consumers and Communication Review Group (CCCRG) study design guidance (Ryan 2009) which advocate using the criteria proposed by the Cochrane Effective Practice and Organisation of Care Review Group (EPOC) (EPOC undated) to minimise bias.

Types of participants

Participants were members of the general public or any of its constituent groups (e.g. occupational or sociodemographic groups or any other target group), including children. We excluded studies in which the whole sample are people with mental health problems. This is because a separate Cochrane review addressing this topic is registered with the Cochrane Schizophrenia Review Group.

Types of interventions

We included interventions if they met all of the following criteria:

1. It was a mass media intervention, defined as an intervention that uses a channel of communication intended to reach large numbers, and is not dependent on person‐to‐person contact. Such channels include newspapers, billboards, pamphlets, DVDs, television, radio, cinema, some web and mobile phone‐based media, street art and ambient media. Interventions may have been undertaken at international, national, regional or local level. Studies that used mass media interventions on a small scale in experimental contexts were also eligible for inclusion, as it is the nature of the intervention and its potential for scaling‐up that are the requisite factors. The mass media component(s) must have been substantial, in that it comprised more than 50% of the total intervention (e.g. in terms of time). Interventions with non‐mass media components were eligible, as long as this criterion was met. The mass media intervention may have used one, two or more types of mass media.

2. An intervention may have taken place at a single time point, may have been short‐term or sustained over a long period.

3. The content of the intervention may have taken any form including: factual material, fiction, persuasive material, personal narratives, slogans, symbols, images, quizzes and games.

4. Mental health was the subject (or one of the subjects) of the intervention. For the purpose of this review, mental health included all conditions listed in the Diagnostic and Statistical Manual of Mental Disorders (DSM‐IV‐TR) (APA 2000), including developmental disorders, dementia, learning disability and substance abuse. Interventions that did not specify a particular condition were also eligible, e.g. interventions that referred to psychological or emotional problems, mental well‐being, etc. Interventions that were not exclusive to mental ill health, but encompassed it, such as disability interventions, were eligible as long as outcomes were reported that related specifically to people with mental ill health.

5. The comparator was an inactive control, e.g. the control group received an intervention with no messages or other content likely to reduce mental health‐related stigma, or received no intervention.

There was no requirement for an intervention to have any intention to reduce stigma. However, we excluded media reports of violent acts committed by people with mental ill health, as these have no potential to reduce stigma. We also excluded clinical mental ill health education interventions directed at health or social care professionals.

Types of outcome measures

We did not exclude studies for failing to use validated outcome measures. However, we reported any validation of outcome measures.

The main outcomes (reported at Quality of the evidence) were: discrimination towards people with mental ill health; prejudice towards people with mental ill health; cost and unforeseen adverse effects.

Primary outcomes

There were two primary outcomes: (i) discrimination and (ii) prejudice; reflecting two of the three elements in Thornicroft’s conceptualisation of stigma (Thornicroft 2007). Discrimination, being a behavioural outcome, is more difficult to measure, but is of greater importance than prejudice for improving the lives of people with mental health problems; consequently data for this outcome are presented before those for prejudice outcomes. For the same reason, in our assessment of the strength of evidence (Quality of the evidence) we defined discrimination as a ‘critically important’ outcome and prejudice as an ‘important’ one.

To have been eligible for inclusion, a study must have included at least one of the discrimination or prejudice outcome measures.

1. Discrimination towards people with mental ill health, including: reports of discrimination personally experienced by people with mental ill health; observed discriminatory behaviour towards people with mental ill health, such as avoidance and negative interaction observed in experimental settings; and reported behavioural discrimination towards people with mental ill health. We had originally planned to include discriminatory behavioural intentions under discrimination, but elected to report these as under prejudice as an intention is more akin to an attitude than a behaviour.

2. Prejudice towards people with mental ill health, including: attitudes towards people with mental ill health; stereotyping of people with mental ill health; desire for social distance from people with mental ill health; emotional responses towards people with mental ill health; empathy for people with mental ill health; and implicit associations regarding people with mental ill health.

Secondary outcomes

The secondary outcomes were:

• knowledge (any type);

• cost of the mass media and comparator interventions (cost charged, or cost incurred if cost charged data were unavailable, in pounds sterling);

• reach, recall, and awareness of intervention(s);

• duration/sustainability of media effects;

• audience reactions to media content (generally and by specific groups within sample e.g. favourability and information/message communicated); and

• unforeseen adverse effects (other than increases in discrimination and prejudice).

Search methods for identification of studies

Electronic searches

We searched eleven electronic databases, each from its earliest date.

• Cochrane Central Register of Controlled Trials (CENTRAL, The Cochrane Library, Issue 7, 2011) (Appendix 1)

• MEDLINE (OvidSP),1966 to 15 August 2011 (Appendix 2)

• EMBASE (OvidSP),1947 to 15 August 2011 (Appendix 2)

• PsycINFO (OvidSP), 1806 to 15 August 2011 (Appendix 3)

• CINAHL (EBSCOhost) (nursing and allied health database) 1981 to 16 August 2011 (Appendix 4)

• ERIC (CSA) (educational database), 1966 to 16 August 2011 (Appendix 5)

• Social Science Citation Index (ISI), 1956 to 16 August 2011 (Appendix 6)

• OpenSIGLE (http://www.opengrey.eu/) (grey literature),1980 to 2005 (latest date of database, searched 18 August 2012) (Appendix 7)

• Worldcat Dissertations and Theses (OCLC), 1978 to 18 August 2011 (Appendix 8)

• metaRegister of Controlled Trials (http://www.controlled‐trials.com/mrct/mrct_about.asp), 1973 to 18 August 2011 (Appendix 9)

• Ichushi (Japanese medical database) (OCLC), 1903 to 11 November 2011 (Search strategies are available on request from sosei.yama@ncnp.go.jp).

A MEDLINE search strategy was developed (see Appendix 2) and tailored to the other databases. There were no language restrictions.

Peer review procedures raised concerns about the translation of searches including:

• the lack of exploded MeSH terms in the CENTRAL strategy; and

• limited translation of the MEDLINE strategy to EMBASE, PsycINFO and CINAHL databases.

The impact of these deficiencies is difficult to ascertain. We hope that by searching other resources (see below) we have mitigated the risk of having missed relevant studies. We welcome contact from any authors who believe their studies may be relevant to this review.

Searching other resources

Other search methods included: searching abstracts of World Psychiatric Association Stigma Conferences; reference checking of included studies and reviews; personal communication with experts in the field, including stigma researchers and media scientists; searching websites of governmental and non‐governmental organisations known to be running anti‐stigma campaigns in mental ill health; and citation forward checking from included studies using the Science Citation Index and the Social Science Citation Index via the Web of Science database.

Data collection and analysis

Selection of studies

Two authors independently screened the initial 20% of items. If agreement on whether to exclude studies between the two authors was greater than 95%, we planned for one author to screen the remaining references. This level of agreement was exceeded, so we proceeded in this way. Full papers were ordered for all items identified as potentially relevant by at least one author. Full papers were also ordered when there was insufficient information from the title and/or abstract to indicate possible relevance.

Two authors independently considered whether each full paper obtained met the inclusion criteria and disparities in inclusion decisions were resolved through discussion, with arbitration by a third author where necessary. Review authors did not contribute to inclusion decisions regarding studies in which they had been involved.

Data extraction and management

We used Endnote software to store and manage all located studies. We extracted data into data extraction tables, based on the CCCRG Data Extraction Template. The table format was piloted before use. The format is as follows:

Methodological details of study: aim of study; study design; details of cluster RCTs (number of clusters, size of each cluster, description of the clusters and the intra class correlation coefficient); details of ITS studies (number of time points, the length of time between points, the exact dates and duration of the intervention and the method of statistical analysis used); methods of recruiting participants; inclusion/exclusion criteria for participation; funding; statistical methods; power calculation; and consumer involvement in study design or intervention.

Assessment of risk of bias: Using standard tools (as detailed at Assessment of risk of bias in included studies).

Participants: description of sample measured; geographic location; setting; number; age; gender; ethnicity; and income level of participants' country (World Bank Index A, B or C).

Details of intervention: aim of intervention; content of intervention; type(s) of mass media used; number of mass media components; whether mass media component was combined with non‐mass media components; group(s) targeted by intervention; whether intervention involved personal narratives; whether celebrities were included; whether it was a fictional portrayal of mental ill health; type of message(s) in intervention (based on categories in Clement 2010); mental health condition(s) addressed; intervention providers (who designed the intervention, who funded it, who oversaw its delivery).

Details of control condition(s).

Details of co‐interventions in all groups (non‐mass media elements in interventions).

Delivery of intervention: stages, timing, frequency, duration (specifically and whether < 3 months or 3+ months), reach, recall, awareness.

Intervention quality and fidelity: whether intervention had a theoretical basis and details of theoretical basis; formative research undertaken in the development of intervention; evidence‐base for intervention; whether intervention was delivered as intended; quality information assessed by study authors, others, review team.

Outcomes: primary and secondary outcome measures (as identified by study authors); any validation of outcome measures; methods of assessing outcomes (e.g. phone survey); methods of follow‐up of non‐respondents; timing of outcome assessment (frequency and duration); adverse events.

Notes: contact with authors; if study was translated; if a duplicate publication; and other information.

Results (numerical data): effect estimates, standard errors (these may be calculated from other presented statistics). See also Measures of treatment effect.

Data were extracted independently by two authors. Disparities were resolved through discussion, with arbitration with a third author where necessary. Review authors did not contribute to data extraction of any studies in which they were involved.

We contacted study authors for further information when data relating to any of the fields in the data extraction table were missing.

Assessment of risk of bias in included studies

For RCTs we used the Cochrane Collaboration's 'Risk of bias' tool (Higgins 2011, section 8.5). For ITS studies we used the Cochrane 'Risk of bias' tool for ITS studies adapted using EPOC's criteria for ITS studies and input from the CCCRG (Ryan 2011). Two authors independently assessed the risk of bias. For 'Other sources of bias' we specifically considered: lack of evidence for reliability or validity of primary outcomes; for cluster trials with few clusters, whether there was a difference in baseline measures or participant characteristics; any evidence of counter‐discourse (e.g. high‐profile violence by a person with mental illness during study period); outcomes between audience members and non‐audience members of the media content not compared where relevant (e.g. in population‐level studies); and the risks of bias from other sources as listed in Higgins 2011 (section 8.15.1).

We had not specified methods for classifying levels of risk for outcome measures in our protocol, therefore, through discussion (SC, EB, SEL and FL) we established the following pragmatic criteria. We rated as ' high risk' measures developed by the study authors with no psychometric data reported; measures for which the authors reported a Cronbach's alpha of < 0.7; and un‐referenced measures. We rated as 'unclear' referenced measures with no psychometric data reported; referenced measures with no statement that the measure was reliable or valid; and validated measures being used for the first time in a different type of population. We rated as 'low risk' measures which study authors reported had a Cronbach's alpha of 0.7 or greater, or referenced the measure as being reliable or valid. Disparities in any 'Risk of bias' ratings were resolved through discussion, with arbitration with a third author where necessary, although in the event no arbitration was needed. We planned that review authors would not contribute to 'Risk of bias' assessment for any studies in which they were involved, but this situation did not arise. We incorporated the results of the 'Risk of bias' assessment into the review through narrative description about each of the 'Risk of bias' items, leading to an overall assessment of the risk of bias in the included studies. Studies considered at high risk of bias were removed as part of a sensitivity analysis (see Sensitivity analysis).

Measures of treatment effect

Subject to data availability, for RCTs with continuous outcome measures we had planned to report the mean differences with 95% confidence intervals (CIs), and for dichotomous outcome measures to calculate odds ratios with 95% CIs. However, as the same outcome was generally measured by different scales, we calculated standardised mean differences (SMDs) instead of mean differences.

In cluster RCTs, when the cluster size, number of clusters and the intra‐class correlation coefficient (ICC) (or estimate equivalent) could be successfully obtained for a study, we inflated the variances for clustering.

For ITS studies we planned to proceed as follows: where the risk of bias for all criteria was low, the study authors’ results would be used. If any ITS study failed to meet this criterion, raw data would be requested for reanalysis using autoregressive interrupted moving average (ARIMA) models as suggested in Ramsay (Ramsay 2003) when there are a large number of time points; otherwise by using time series regression as suggested by Grilli (Grilli 2002). When ARIMA models were used, we would obtain both point estimates and change in slope estimates for each study, as both of these are important in the interpretation of the intervention effect. When time series regression was used, regression coefficients would be used to measure intervention effects. In the event no ITS studies were included in the review.

We did not pre‐specify actions if data were skewed data. When this was the case the data were transformed into the logarithmic scale using methods described by Higgins and colleagues (Higgins 2008).

Unit of analysis issues

In cluster trials, where reported we used effect estimates and standard errors that were adjusted in the analysis for clustering, and combined the studies using the generic inverse variance method. If the analysis did not take account of clustering, we approximated the cluster adjusted effect size and standard error based on available data if the unadjusted effect estimate, the number or size of clusters and the ICC were provided. If the ICC could not be obtained then we used an estimate from similar studies.

In cross‐over trials, we planned to use the effect estimate and standard deviation based on a paired t‐test, and combine the studies using the generic inverse variance method (Higgins 2011, section 16.3). However, no appropriate cross‐over trials were identified (see Included studies).

If studies had more than two groups we combined all relevant experimental intervention groups of the study into a single group, and combined all relevant control intervention groups into a single control group (Higgins 2011, section 16.5.4). Where intervention arms fell into different subgroups each intervention arm was compared to the control group, and the possibility of meta‐analysis was only considered within each subgroup, thereby avoiding potential unit of analysis errors. Data from irrelevant intervention groups were ignored (e.g. live presentations). We had not anticipated studies having two control groups, so we made a post‐hoc decision to select the control group most similar to the intervention.

Dealing with missing data

We contacted study authors where any data were missing. Where studies did not state that results were reported using an intention‐to‐treat analysis for primary outcomes, we contacted study authors to request data to enable us to conduct such an analysis, and in the event of non‐response we analysed results as reported.

When there were missing summary data in a study, we contacted authors and asked them to provide the required summary data, or failing that, any data to derive the required summary data. If authors were unable to provide this, we attempted to derive the specific data from other reported statistics in the study. If we could not obtain such data, we analysed the particular study narratively.

Assessment of heterogeneity

Statistical measures of heterogeneity were ascertained visually, and using the Cochrane's Q and the I² statistic, with I² > 50% representing substantial heterogeneity (Higgins 2011, section 9.5.2). We also considered the clinical heterogeneity of the studies (for example in participants, interventions and outcomes) and methodological heterogeneity (such as in the quality of the studies, and in study design).

Assessment of reporting biases

Where we found at least 10 studies and an appropriate range of sample sizes, we assessed the possibility of reporting bias using funnel plots to examine the relationship between studies' risk of bias and effect size estimates. This was quantified using Egger's test of symmetry. Where reporting bias was identified, we investigated the impact in a sensitivity analysis.

Data synthesis

Whether a narrative synthesis or meta‐analysis was conducted, we planned to produce a 'Summary of findings' table from the included studies for each type of study design (i.e. RCT and ITS) using GRADEprofiler (GRADEpro) software. In the event the 'Summary of findings' table was produced using the template in RevMan, but still following the GRADE approach (Guyatt 2008).

For RCTs, for each comparison (mass media intervention versus control) we reported tables of summary statistics for each of the included studies. For each primary and secondary outcome, we reported outcome measure, follow‐up summary statistics, and effect estimates and their statistical significance. In the protocol (Clement 2011) we stated that we would also report baseline summary statistics, but this is no longer applicable now that SMDs rather than mean differences are being reported (see Measures of treatment effect). We also reported our assessment of risk of bias. For cluster randomised trials we noted whether there were unit of analysis issues. We also reported details concerning potential effect moderators (as specified under Subgroup analysis and investigation of heterogeneity) e.g. nature of the intervention, content of the intervention, type of media.

For ITS studies we had planned to follow the approach outlined by Brennan (Brennan 2009), and present results from these studies in tables for each comparison with summary statistics for each of the included studies, change in level of the outcome at the first point after the introduction of the intervention, post‐intervention slope minus the pre‐intervention slope, and information on effect modifiers. We would have also presented this graphically using, for example, scatter plots of change in level versus change in slope with combinations of statistical significance denoted by different symbols. In the event, we found no ITS studies.

In a narrative synthesis, for each comparison (e.g. mass media intervention versus control) we stated: the number of comparisons showing a positive direction of effect; the median effect size across all comparisons; the median effect size across comparisons without unit of analysis errors; and the number of comparisons showing statistically significant effects. This approach was recommended by Grimshaw 2003 as it “allows the reader to assess the likely effect size and consistency of effects across all included studies and whether these effects differ between studies, with and without unit of analysis errors”.

In the narrative synthesis and in any statistical synthesis, we synthesised first according to the different types of interventions (grouping similar interventions together), second according to the types of outcomes (with discrimination outcomes reported first, then prejudice outcomes, then secondary outcomes), and third according to the strength of evidence.

Preliminary scoping of the field indicated considerable heterogeneity in the types of intervention, participants and outcome measures, therefore we anticipated that we would be unlikely to find sufficient homogeneity to warrant meta‐analysis. However a review author group discussion (originally planned as a face‐to‐face meeting but altered to an email discussion, given the disparate locations of the authors) took place to judge the appropriateness of meta‐analysis in the light of the heterogeneity assessments. For any meta‐analysis undertaken we used a random‐effects model, as planned, as we had predicted there would be a high level of heterogeneity across the studies.

In the event of multiple outcomes reported in a study, the outcome selected for analysis was the primary outcome as defined by the authors of that particular study. If there was no specified primary outcome, or if a specific primary outcome could not be deduced from the study, we chose the outcome from which the power equation for the study was provided. In the case where this was not reported, we chose the outcome which had the median reported effect size (Grimshaw 2003). Where there was an even number of outcomes, we made a post‐hoc decision that, following Brennan 2009, we would select the outcome with the n/2 ranked effect size (using data from the final follow‐up point when there were two or more follow‐up points). A post‐hoc decision was also needed about which outcome to select when multiple outcomes were used in studies with median data. In these cases an adapted version of the methods proposed by Brennan 2009 was used whereby, after checking that the interquartile ranges were similar, we examined medians at the latest time point and selected the one ranked (n+1)/2 when there was an odd number of outcomes and the one ranked n/2 when there was an even number.

Subgroup analysis and investigation of heterogeneity

We planned to undertake the following subgroup analyses (by narrative methods and also by meta‐analysis if appropriate) to explore possible explanations for observed heterogeneity:

• Short‐term interventions (up to three months) versus long‐term interventions (three months or longer). This was not undertaken as all included studies had short‐term interventions.

• Studies in high‐income countries (band A, World Bank Index) versus middle‐/low‐income (band B and C) countries.

We intended to conduct the following comparisons:

Comparisons relating to nature of the intervention:

• Interventions with one mass media component versus those with two or more mass media components.

• Interventions in which the mass media component(s) was combined with non‐mass media components, versus interventions with a mass media component only.

Comparisons relating to the content of the interventions:

• Interventions involving personal narratives (indirect 'social contact') versus those not involving personal narratives. In the event a post‐hoc decision was made to use three subgroups here: first‐person narratives, third‐person narratives and no narratives, as we had not anticipated the middle group.

• Interventions with the primary message being biomedical, psychosocial, recovery‐oriented, 'see the person', high prevalence of mental disorders, anti‐dangerousness, valuing difference, social inclusion/human rights, continuum or negative impact of mental illness (Clement 2010). A post‐hoc decision was made not to use the 'see the person' message type as this message type only arose when interventions contained personal narratives and if we had categorised these as having a 'see the person' primary message, we would have missed messages contained in what the narrators said (or other aspects of the intervention). We also decided post‐hoc to include commonly‐used categories of primary message that were not in Clement 2010. We had not pre‐specified the method of deciding which message was primary, and decided this would be undertaken independently by two authors who would resolve disparities by discussion, and with arbitration if necessary.

• Interventions that included personal narratives by celebrities versus interventions that included personal narratives and included no celebrities.

• Interventions that included fictional narratives versus interventions with non‐fictional narratives.

Comparisons relating to the type of media:

• Interventions that used broadcast media (television, radio) versus print media versus cinema/recordings versus Internet/mobile phone versus other media. As no mobile phone, broadcast media or cinema interventions were found, we did not refer to these and they did not appear in the type of media subgroup analysis. There was just one intervention ‐ a CD‐ROM ‐ that fell in the 'other' category for media type, and we decided to group this in the Internet category, as Internet‐delivery would not have materially changed participants' experience of the intervention.

Sensitivity analysis

Where meta‐analysis was possible, we conducted sensitivity analysis to examine the effects of excluding studies at higher risk of bias.  If bias was discovered we used two methods as a sensitivity analysis:

1. removed the less precise studies, and

2. used the ‘trim and fill’ method. This method was only performed on the prejudice (immediate) outcome, for which there were more than 10 studies.

We had intended to test for small study effects of binary outcomes by performing the arcsine‐Thompson test, as this has been shown to perform well in simulations and it allows for substantial between‐study heterogeneity (Rücker 2008). However this was precluded because we found only two studies with binary primary outcomes (Yoshida 2002; Penn 2003, discrimination outcome) and these had very different timings of outcome (immediate and 9 months), rendering this test inappropriate.

We included a sensitivity check of a fixed‐effect model. A sensitivity analysis for plausible variations in estimated ICCs was performed when unit of analysis errors arose in cluster randomised trials and the ICCs were estimated for these studies from studies of similar populations, that is from university students (Campbell 2011).

As we found that three of the multi‐arm studies included arms that the study authors considered unlikely to reduce stigma (Reinke 2004; Brown 2010; Corrigan (submitted)), we undertook a post‐hoc sensitivity analysis to examine the effects of removing these studies.

Stakeholder participation

One of the authors of the review uses mental health services, has experience of mental health‐related stigma, and has close family members who have used mental health services, and draws on these perspectives in this review.

A consultation group was set up to provide additional relevant perspectives. The role of the consultation group members was to comment and provide feedback on the draft protocol, draft review and plain language summary. Members of this group who have worked in anti‐stigma campaigning/research were also included in the request for additional studies that may meet the inclusion criteria at the search stage. Those with personal experience of mental ill health were asked to give their view on the importance of the degree of change in stigma found. The group included the following members: a researcher from the Service User Research Enterprise, Institute of Psychiatry, Kings College London; the Deputy Director of Knowledge and Learning, Rethink (charity for people affected by severe mental illness); a medical doctor; an advertising executive; the Service User Lead for an organisation working to reduce mental health‐related stigma and discrimination, focusing particularly on employment; and the ex co‐chair of a mental health service user advocacy organisation. Four of these members were also stigma researchers.

The plain language summary was written by the review author with experience of mental health service use in collaboration with a person who has used mental health services but who is not involved in research.

In addition, the standard peer review process of the Cochrane Consumers and Communication Review Group includes review of the protocol and review by at least one consumer.

Results

Description of studies

See: Characteristics of included studies; Characteristics of excluded studies. '(a)' indicates information provided by study authors.

Results of the search

See Figure 1. The database searches yielded 22,895 records, as follows:

1.

Study flow diagram.

• CENTRAL (315);

• MEDLINE (3303);

• EMBASE (9530);

• PsycINFO (1803);

• CINAHL (401);

• ERIC (1782);

• Social Science Citation Index (3663);

• OpenSIGLE (46);

• Worldcat Dissertations and Theses (80);

• metaRegister of Controlled Trials (0); and

• Ichushi (1972).

We found a further 24 records from:

• reference checking of reviews (19);

• conference abstracts of World Psychiatric Association Stigma Conferences (1);

• citation checking from included studies (3); and

• communication with experts (1).

No records were identified from website checking, reference checking of included studies or communication with authors of included studies.

Two authors independently screened 20% of the 19,031 non‐duplicate records and achieved an agreement level of 99.9%. Consequently a single author screened the remainder of the records as per our protocol (Clement 2011). We deemed 131 records to be potentially relevant, and for each of these full‐text articles were obtained and assessed independently for inclusion by two authors. Disparities were resolved through discussion and 108 papers were excluded. We initially identified 23 studies (24 papers) as eligible for inclusion in the review, but one was excluded after contact with the author (Bayar 2009).

Included studies

We included 22 studies in the review. Three studies contained no analysable outcome data (Dias‐Vieira 2005; Han 2006; Varughese 2010), therefore 19 studies contribute to the qualitative synthesis. Meta‐analysis was appropriate for three studies and for one outcome only (see Effects of interventions). The 22 studies had a total of 4490 participants randomised to relevant study arms (median = 150, range = 46 to 739). For the 19 studies with analysable outcome data, data were available for a total of 1196 participants (five studies) for discrimination outcomes, 3176 (19 studies) for prejudice outcomes, 1213 (4 studies) for knowledge outcomes, 381 (3 studies) for audience reactions to the interventions, 416 (3 studies) for cost data, 727 (1 study) for awareness data, 1225 (5 studies) for duration of effects data, and 455 (2 studies) for statements about adverse effects.

We sought contact with authors of all included studies. Authors of nine studies responded with additional information, with one conducting additional analyses (Jorm 2010a) and one providing the data set (Yoshida 2002).

Study designs

The 22 included studies had a high level of clinical and methodological heterogeneity (described in this section), and were also statistically heterogeneous (see Effects of interventions). All included studies were randomised controlled trials, including three cluster trials (Yoshida 2002; Penn 2003; Coleman 2005) and one had a cross‐over design (Varughese 2010). In none of the cluster trials was there any adjustment for the design effect, although one study author (Yoshida 2002) provided his data set and the review team statistician (PW) analysed this accounting for clustering. Because there is no established 'wash‐out' period for stigma interventions, we concluded that the cross‐over design was inappropriate and contacted the author to request summary statistics for a post‐intervention comparison of those allocated to receive the intervention and control initially. As no reply was received, this study (Varughese 2010) had no analysable data, as did two other studies (Dias‐Vieira 2005; Han 2006).

Ten of the studies were multi‐arm studies having between two or more mass media intervention groups. Eleven of the studies had some level of involvement from people with mental health problems in the design of the study or intervention, although this was often indirect. Four studies had such a person as a study author and in each of these the person was also involved in developing the intervention itself (Reinke 2004; Kerby 2008; Matthews 2009; Jorm 2010a). A further three studies used or adapted interventions which had been developed with consumer involvement (Russell 1988; Bunn 2009; Brown 2010). Three used interventions endorsed by a consumer organisation (Woods 2002; Woods 2003; Woods 2005) and one drew upon messages endorsed by a consumer organisation in developing the intervention (Smith 2007). Eight of the studies were unpublished (seven were dissertations and one was a paper in submission). Twenty‐one studies were reported in English and one in Japanese (Yoshida 2002).

Settings and populations

The studies took place mainly in the United States (n = 15), or other high‐income countries, including two studies in the United Kingdom (Kerby 2008; Varughese 2010), and one study each in Canada (Matthews 2009), Australia (Jorm 2010a), Japan (Yoshida 2002) and Taiwan (Han 2006). One study took place in Russia (Finkelstein 2008), a country classified as 'upper middle income'. The majority (n = 13) of studies had samples comprised of college or university students studying psychology (usually introductory) or a variety of courses. Other studies involved students on professional courses: medical students (Kerby 2008; Bunn 2009); nursing students (Coleman 2005) and special education students (Finkelstein 2008). Four of the studies had general population samples (Yoshida 2002; Jorm 2010a; Varughese 2010; Corrigan (submitted)) and one involved employers (Russell 1988).

Intervention and control conditions

The interventions were all short‐term (defined a priori as up to three months). The majority (n = 16) involved a single presentation of a single‐component mass media intervention. However, in two studies participants could complete the multi‐component educational materials interventions over a number of sittings in up to four weeks (Finkelstein 2008; Jorm 2010a); in one study participants received three pamphlets mailed weekly (Russell 1988); in another they received material by email sent over four weeks (Smith 2007); in another they were handed three booklets (Yoshida 2002); and in one watched two films at one sitting (Kerby 2008). No interventions combined two or more types of mass media. In two studies the mass media component was combined with a non‐mass media component, that is a class discussion (Reinke 2004), or a meeting with people with mental health problems (Yoshida 2002). None of the interventions used broadcast media (television, radio); none were delivered through public cinemas, and no mobile phone interventions were found in the eligible studies. The interventions therefore fell into three of the predefined categories: Internet, recordings and print.

Internet interventions included an Internet‐delivered interactive computer education programme (Finkelstein 2008); web‐delivered newspaper articles (Corrigan (submitted); and educational adverts delivered by email (Matthews 2009). An educational CD‐ROM (Jorm 2010a) was classified as Internet, as this was the category it most closely fitted. Post‐hoc, we subdivided the recordings into 'audiovisual' and 'audio' because the audio recordings were very different from the audiovisual ones. The former were all DVDs or videos containing personal narratives (n = 7), educational material (Morgan Owusu 2002), a combination of the two, or a public service announcement (Demyan 2009). The latter were recordings of simulated hallucinations delivered through headphones. The print interventions included educational manuals (Finkelstein 2008; Jorm 2010a); a magazine article (Matthews 2009); brochures (Russell 1988; Yoshida 2002; Dias‐Vieira 2005); written text (Han 2006; Iobst 2008); and a photograph (Varughese 2010). Conditions covered were: schizophrenia/psychosis/serious mental illness (n = 8), a combination of conditions (n = 5), depression (Dias‐Vieira 2005; Han 2006); exam stress (Demyan 2009); Tourette's syndrome (Woods 2002; Woods 2003; Woods 2005); intellectual disability (Russell 1988; Varughese 2010); and autism (Iobst 2008). Reduction in stigma was not the main objective of all the interventions, for example some aimed to encourage healthcare‐seeking (Morgan Owusu 2002; Demyan 2009) and some had a primary educational aim (Coleman 2005; Jorm 2010a). Three studies included one intervention arm they believed was unlikely to reduce stigma (Reinke 2004; Brown 2010; Corrigan (submitted)).

The most common types of primary message included in the studies were recovery‐oriented messages (n = 7) and 'not to blame' messages (n = 4). Four studies had multiple primary messages. Intervention arms also included primary messages of the following types: biomedical (Coleman 2005; Han 2006; Iobst 2008), social inclusion (Russell 1988), 'seek professional care' (Morgan Owusu 2002; Demyan 2009; Jorm 2010a), 'negative impact of mental illness' (Reinke 2004; Corrigan (submitted), and 'this is hallucinatory experience' (Bunn 2009; Brown 2010). Some interventions had been subject to extensive developmental work (e.g. Jorm 2010a) and others were based on extensive theoretical considerations (e.g. Finkelstein 2008), whereas others were developed for the study (e.g. Smith 2007). For 12 studies the control was no intervention, 1 had a waiting list control (Jorm 2010a), and 9 used the same mass media containing material unrelated to mental health problems or stigma (e.g. a film about wildlife). One study (Penn 2003) had both a 'no intervention' and an irrelevant material control group, and for this study we used the latter as it was more closely matched and controlled better for non‐specific effects.

Outcomes

In the majority of studies outcomes were measured immediately post‐intervention only (n = 14); four had final follow‐up at 1 to 2 weeks, two at 1 to 2 months (Kerby 2008; Jorm 2010a); one at 6 months (Finkelstein 2008); and one at 9 months (Yoshida 2002). Discrimination outcomes were measured in only 5 of the 22 studies. Measures included the distance between chairs arranged by participants anticipating meeting a person with Tourette's syndrome (Woods 2002; Woods 2003; Woods 2005); written expression of interest in attending a focus group with people with schizophrenia and providing contact details (Penn 2003); and reported visits to community mental health facilities during the follow‐up period (Yoshida 2002). All studies measured prejudice outcomes with the majority (n = 12) using multiple measures for this outcome. In total 28 different measures were used, with the Social Distance Scale (Link 1987) being the most common, and used in 6 of the studies. Data on secondary outcomes were sparse, with four studies measuring knowledge (each of these assessed knowledge of mainstream models of mental illness), four measuring audience reactions to the interventions, one assessing awareness of the intervention (Yoshida 2002), and none assessing reach or recall. No studies assessed cost effectiveness or reported cost data, however two authors were able to provide some specific information about the costs of their interventions (Demyan 2009; Jorm 2010a).

Excluded studies

See Characteristics of excluded studies. The main reasons why the 108 potentially‐relevant studies were excluded were:

• not having an RCT or interrupted time series design (n = 38);

• there being no inactive comparator (n = 27);

• the intervention not being mass media (n = 19);

• the mass media component not comprising more than 50% of the intervention (n = 10); and

• there being no discrimination or prejudice outcome (n = 9).

One study (Bayar 2009) was initially included, however when we requested further details about randomisation methods the author informed us that alternation had been used, so we excluded this study as it did not meet the criteria for being an RCT. We located one simulated hallucinations intervention study (Kalyanaraman 2010) other than Bunn 2009 and Brown 2010, but this used a hand‐held headset to deliver both audio and visual simulated hallucinations and was excluded as, unlike headphone‐delivered interventions, it does not currently have the potential to be scaled up for use at the population level.

Risk of bias in included studies

For detection bias all studies were considered as being at high risk, because they had at least one primary outcome that was assessed by self‐report. For all other types of bias, a minority of studies were at low risk of bias. Selective reporting, incomplete outcome data and lack of participant blinding were the types of bias with fewest low risk studies, each including some high risk studies. High risk studies were also found for allocation concealment and other forms of bias (mainly relating to lack of validity of outcome measures). No studies had no high risk of bias ratings. See: Characteristics of included studies, Figure 2 and Figure 3.

2.

Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.

3.

Risk of bias summary: review authors' judgements about each risk of bias item for each included study.

Allocation

Thirteen of the 22 studies had an unclear risk of bias in relation to random sequence generation, generally because the particular method of generation was unspecified preventing us from being sure that the allocation sequence was truly random. The same number had an unclear risk for allocation concealment and a further two (Smith 2007; Brown 2010) were at high risk, both on the basis of further information provided by the authors. Only 6 of the 22 studies were at low risk on both indicators of selection bias.

Blinding

With mass media interventions participants will always know that they are receiving some sort of intervention and it will often be clear to them that they are experiencing something intended to alter their views or behaviour. Consequently the vast majority of studies (18/22) were at high risk for non‐blinding of participants. However four studies (Morgan Owusu 2002; Dias‐Vieira 2005; Demyan 2009; Matthews 2009) used cover stories to help blind participants by informing them that the purpose of the research was to rate or improve extraneous features of the mass media material (e.g. music, length), sometimes embedding the intervention and control within a battery of materials to further blind the participants (Demyan 2009; Matthews 2009). For the five studies with discrimination outcomes, two were at high risk of detection bias as measures were assessed by written self‐report (Yoshida 2002; Penn 2003), however three had an unclear risk of detection bias because these involved investigator measurements of the distance between two chairs and it is not stated whether or not the researcher was aware of group allocation when making this measurement (Woods 2002; Woods 2003; Woods 2005). All prejudice outcomes and the secondary outcomes were assessed by self‐completed questionnaire and therefore were considered to be at high risk of detection bias.

Incomplete outcome data

Fourteen studies had an unclear risk of attrition bias and two were considered to be at high risk. In Woods 2002 there were significant incomplete data for the discrimination measure as this was only assessed on the second half of the sample. This was also the case in Smith 2007 which had a post‐randomisation exclusion of, and no data available for, those who had previous contact with mental health services or had someone in their immediate family who had had such contact (n = 71). Only a minority (5/22) of studies were at low risk of attrition bias.

Selective reporting

For the majority of studies (17/22) there was an unclear risk of selective reporting as no study protocol or similar material was available to enable us to check that all variables measured had been reported. Two studies were deemed to be at high risk of selective reporting bias (Han 2006; Kerby 2008) as data for some measures were only partially reported. Three studies were at low risk because: their protocol appeared on a trials register (Jorm 2010a); the author provided the data set (Yoshida 2002); or the author provided a research ethics board application (Matthews 2009), and all measures in these materials were reported.

The group of studies including at least 10 studies and an appropriate range of sample sizes was the non‐clustered RCT studies with prejudice outcomes and immediate follow‐up. A funnel plot was created which was relatively symmetrical and therefore provided no evidence of publication bias. Likewise, Egger’s test of symmetry was not significant (P value = 0.444) providing no evidence of small study effects.

Other potential sources of bias

The most common potential other source of bias in the included studies was lack of reported evidence for the reliability and validity of outcome measures: two studies were rated as high risk for this (Morgan Owusu 2002; Kerby 2008), with an unclear risk in a further ten studies. Other high risks of bias came from baseline imbalance (Yoshida 2002; Reinke 2004; Coleman 2005); and evidence of counter‐discourse during the follow‐up period through the occurrence of serious crimes committed by people reported to have mental illness during the study period, which may have had a differential effect on those in the two study arms (Yoshida 2002). Finally two studies had an unclear risk of bias due to failure to adjust for study design effects in the analysis of cluster trials (Penn 2003; Coleman 2005).

See Figure 2; Figure 3

Effects of interventions

See: Table 1

See Table 1.

Primary outcomes

Narrative synthesis of effects of interventions

Discrimination

Main comparison

Five studies (n = 1196) assessed discrimination outcomes. Three studies had continuous outcomes, all of which were the distance placed between two chairs when a participant was anticipating a meeting with a person with Tourette's syndrome (Woods 2002; Woods 2003; Woods 2005, see Figure 4). One study reported a reduction in discriminatory behaviour (Woods 2002) with a standardised mean difference (SMD) of ‐0.85 (95% confidence interval (CI) ‐1.39 to ‐0.31), however the two other studies by the same author (Woods 2003; Woods 2005) involving the same type of population and methodology but with larger sample sizes, both failed to provide evidence of an effect of the intervention on discrimination: SMDs ‐0.25 (95%CI ‐0.51 to 0.02); ‐0.17 (95%CI ‐0.53 to 0.20). Meta‐analysis of these three studies was precluded due to statistical heterogeneity (I² = 0.57, see meta‐analysis section). Two studies had a dichotomous discrimination outcome: failure to register to attend a focus group which was to include people with schizophrenia (Penn 2003), and self‐report of not visiting sheltered workshops for people with mental illness during the follow‐up period (Yoshida 2002). These studies provided no evidence of effect: odds ratios (OR) 1.30 (95% CI 0.53 to 3.19); and 1.19 (95% CI 0.85 to 1.65). See Figure 5. Here meta‐analysis was inappropriate due to clinical and methodological heterogeneity (see meta‐analysis section).

4.

Forest plot of comparison: 1 Mass media vs. control (main comparison), outcome: 1.1 Discrimination (Immediate).

5.

Forest plot of comparison: 1 Mass media vs. control (main comparison): outcome 1.2 Discrimination.

We constructed a narrative synthesis table following Grimshaw 2003 (first four columns of Table 2). We added a fifth column to aid the interpretation of the findings, and classified the effect sizes as 'small'; 'medium' or 'large' derived from the work of Cohen (Cohen 1988). Further interpretation using transformation into a common scale is given in Summary of main results. Examination of Table 2 (top section, main comparison) indicates that overall there is no evidence to support or refute mass media interventions being able to reduce discrimination.

1. Narrative synthesis and interpretation: Discrimination.

Comparison	No. of comparisons with positive direct of effect1	Median effect size across all comparisons2	Median effect size across all comparisons without unit of analysis errors3	No. of comparisons showing statistically significant effects4	Interpretation of median effect size across all comparisons5
Mass media vs. control
Immediate follow‐up	3/4	SMD = ‐0.25 OR = 1.30	SMD = ‐0.25	1/4 (+)	Small/negligible
6 to 9 months follow‐up	0/1	OR = 1.19	OR = 1.19	0/1	Negligible
Mass media vs. control by if 2+ mass media components
2+ mass media components	0/1	OR = 1.19	OR = 1.19	0/1	Negligible
One mass media component	3/4	SMD = ‐0.25 OR = 1.30	SMD = ‐0.25	1/4 (+)	Small/negligible
Mass media vs. control by presence of narratives
First‐person narratives	3/4	SMD = ‐0.25 OR = 1.30	SMD = ‐0.25	1/4 (+)	Small/negligible
Third‐person narratives	0/1	OR = 1.19	OR = 1.19	0/1	Negligible
Mass media vs. control by type of primary message
Recovery‐oriented	0/1	OR = 1.19	OR = 1.19	0/1	Negligible
Not to blame	3/3	SMD = ‐0.25	SMD = ‐0.25	1/3	Small
Multiple	0/1	OR = 1.30	‐	0/1	Negligible
Mass media vs. control by fictional narratives
Fictional narratives	0/1	OR = 1.19	OR = 1.19	0/1	Negligible
Non‐fictional narratives	3/4	SMD = ‐0.25 OR = 1.30	SMD = ‐0.25	1/4 (+)	Small/negligible
Mass media vs. control by type of media
Audiovisual	3/4	SMD = ‐0.25 OR = 1.30	SMD = ‐0.25	1/4 (+)	Small/negligible
Print	0/1	OR = 1.19	OR = 1.19	0/1	Negligible

[1] Stigma‐reducing

[2] When there was an even number of effect sizes the median reported is the mean of the two middle values.

[3] This excludes cluster trials not adjusted for study design by the authors and without a study‐derived intra‐class correlation co‐efficient (Penn 2003)

[4] (+) means in stigma‐reducing direction, (‐) means in stigma increasing direction

[5] SMDs large ≥ 0.8; medium ≥ 0.5, small ≥ 0.2, negligible < 0.2; and for ORs large ≥ 4.3; medium ≥ 2.5, small ≥ 1.5, negligible < 1.5, based on Cohen’s (Cohen 1988) rule of thumb, and using a pragmatic decision about how to treat intermediate values

Subgroup comparisons

Several planned subgroup comparisons were not possible as all studies with discrimination outcomes were in the same subgroup. The findings from the subgroup analyses that were possible for this outcome are shown in Table 2. As in the main comparison, all subgroup effect sizes were found to be small/negligible apart from the 'not to blame' messages subgroup which had a small discrimination‐reducing effect size. See Data and analyses; Figure 4; Figure 5; Table 2.

Prejudice

Main comparison

All 19 studies with analysable data reported prejudice outcomes (n = 3176). The results of the main comparison can be seen in two forest plots (Figure 6 and Figure 7). In addition, for one study (Kerby 2008) with skewed outcomes, the medians (interquartile ranges) for the intervention and control groups post‐intervention were 13 ((10 to 15) n = 20) and 12 ((10 to 14) n = 22) respectively. At eight weeks follow‐up the data were 14 ((11 to 15) n = 20) and 12 ((10 to 14) n = 21) respectively. The study authors stated that there were no significant differences between the groups at any time point, but provided no statistical information. We constructed a narrative synthesis combining all these data for prejudice outcomes; see Table 3. Inspection of the forest plots (Figure 6 and Figure 7) and the top section of Table 3 indicates that there is evidence of benefit (reduction in prejudice) at all three follow‐up time points. For example, at immediate follow‐up the median effect size across all studies was SMD = ‐0.38. At 1 week to 2 months follow‐up, the all‐study median effect size was ‐0.38. The median effect size across all three studies with outcomes at 6 to 9 months was ‐0.49. Two studies (Russell 1988; Finkelstein 2008) stood out for having much larger prejudice‐reducing effect sizes: SMD ‐2.94 (95% CI ‐3.52 to ‐2.37) and ‐1.69 (95% CI ‐2.06 to ‐1.32) (see Figure 6), and one (Coleman 2005) for having a very large prejudice‐increasing effect (SMD 2.40, 95% CI 0.62 to 4.18) (see Figure 7). Only one meta‐analysis within one subgroup of studies (Finkelstein 2008; Jorm 2010a) ‐ non‐cluster trials with prejudice outcomes at 6 to 9 months ‐ had an I² value indicating reasonable statistical homogeneity. Meta‐analysis was warranted (see meta‐analysis section) and these studies had a combined affect size of SMD =‐0.36 (95% CI ‐0.63 to ‐0.10) demonstrating a reduction in prejudice.

6.

Forest plot of comparison: 1 Mass media vs. control (main comparison), outcome: 1.3 Prejudice.

7.

Forest plot of comparison: 1 Mass media vs. control (main comparison), outcome: 1.4 Prejudice.

2. Narrative synthesis and interpretation: Prejudice.

Comparison	No. of comparisons with positive direct of effect1	Median effect size across all comparisons2	Median effect size across all comparisons without unit of analysis errors3	No. of comparisons showing statistically significant effects4	Interpretation of median effect size across all comparisons ( Cohen 1988)5
Mass media vs. control
Immediate follow‐up	12/15	‐0.38	‐0.38	6/15 (+), 1/15 (‐)	Small
1 to 8 weeks follow‐up	3/5	‐0.38	‐0.38	2/5 (+)	Small
6 to 9 months follow‐up	3/3	‐0.49	‐0.49	1/3 (+)	Small‐to‐medium
Mass media vs. control by income of country
Upper middle income	1/1	‐1.69	‐1.69	1/1	Large
High income	14/18	‐0.42	‐0.42	7/18 (+) 1/18 (‐)	Small
Mass media vs. control by if 2+ mass media components
2+ mass media components	4/6	‐0.49	‐0.49	3/6 (+)	Small‐to‐medium
One mass media component	12/13	‐0.34	‐0.34	5/13 (+) 1/13 (‐)	Small
Mass media vs. control by if combined with non‐mass media
With non‐mass media	2/2	‐0.42	‐0.42	0/2	Small
Mass media alone	13/17	‐0.43	‐0.43	8/17 (+) 1/17 (‐)	Small
Mass media vs. control by presence of narratives
First‐person narratives	8/10	‐0.51	‐0.51	6/10 (+) 1/10 (‐)	Medium
Third‐person narratives	4/5	‐0.03	‐0.03	2/5 (+)	Negligible
No narratives	5/7	‐0.27	‐0.27	3/7 (+) 1/7 (‐)	Small
Mass media vs. control by celebrity narratives
Celebrity narratives	1/1	‐0.23	‐0.23	0/1	Small
Non‐celebrity narratives	11/15	‐0.48	‐0.44	7/15 (+) 1/15 (‐)	Small
Mass media vs. control by fictional narratives
Fictional narratives	2/3	‐0.42	‐0.42	1/3 (+)	Small
Non‐fictional narratives	10/12	‐0.47	‐0.47	6/12 (+) 1/12 (‐)	Small
Mass media vs. control by type of message
Biomedical	1/2	0.99	‐0.42	1/2 (+) 1/2 (‐)	Large
Social inclusion/human rights	1/1	‐2.94	‐2.94	1/1 (+)	Large
Recovery‐oriented	5/5	‐0.30	‐0.30	1/5 (+)	Small
Negative impact of mental illness	1/2	‐0.13	‐0.13	0/2	Negligible
Seek professional care	2/3	‐0.43	‐0.43	2/3 (+)	Small
Not to blame	3/3	‐0.51	‐0.51	3/3 (+)	Medium
Hallucinatory experience	1/2	0.22	0.22	1/2 (‐)	Small
Multiple messages	3/4	‐0.70	‐0.71	1/4 (+)	Medium
Mass media vs. control by type of media
Internet	3/4	‐0.30	‐0.30	2/4 (+)	Small
Audiovisual	7/10	‐0.47	‐0.47	5/10 (+) 1/10 (‐)	Small
Audio	1/2	0.22	0.22	1/2 (‐)	Small
Print	6/6	‐0.46	‐0.46	4/5 (+)	Small

[1] Stigma‐reducing

[2] Excluding study with median and IQR data only (Kerby 2008). When there was an even number of effect sizes the median reported is the mean of the two middle values.

[3] This excludes cluster trials not adjusted for study design by the authors and without a study‐derived intra‐class correlation co‐efficient (Penn 2003; Coleman 2005)

[4] (+) means in stigma‐reducing direction, (‐) means in stigma increasing direction. Excluding study with median and IQR data only (Kerby 2008)

[5] SMDs large ≥ 0.8; medium ≥ 0.5, small ≥ 0.2, negligible < 0.2, based on Cohen’s (Cohen 1988) rule of thumb, and using a pragmatic decision about how to treat intermediate values

Subgroup analyses

See Data and analyses. We were unable to undertake the planned subgroup analysis of short‐term (up to three months) versus long‐term (three months or longer) interventions, as no intervention in the review lasted longer than four weeks. All the studies with analysable outcome data (n = 19) had a prejudice outcome and contributed to the narrative synthesis for prejudice (Table 3). Forest plots are not reported for the subgroup analyses as this would create an excessive number of figures, although all were created and examined and used to derive the data in Table 3.

All but one study were undertaken in high‐income countries and these had median SMD of ‐0.42, indicating a small effect size (Cohen 1988). The Kerby 2008 study is excluded from all the subgroup analyses as it used medians and therefore no SMD was calculable. The individual effect sizes for the studies contributing to the subgroup analyses can be seen in Figure 6 and Figure 7 unless otherwise stated. In contrast to the small effect in high‐income countries, a large effect size (SMD ‐1.69, 95% CI ‐2.06 to ‐1.32) was found in the one study undertaken in an upper‐middle income country (Russia) (Finkelstein 2008).

Five studies (Russell 1988; Yoshida 2002; Smith 2007; Finkelstein 2008; Jorm 2010a) had two or more mass media components. In each case the multiple components were within one media type (e.g. three printed booklets) rather than using two or more different types of media in one intervention. The multiple component interventions had a higher median effect size than those with one mass media component (median SMDs ‐0.49 and ‐0.34 respectively). Two studies combined mass media with non‐mass media elements, one by adding a 5‐minute facilitated discussion after a 10‐minute DVD presentation (Reinke 2004), and the other by inviting participants to attend a community event where they would meet people with mental illness and hear about their experiences, and receive information about mental health services (Yoshida 2002). These interventions had a similar small effect size to purely mass media interventions (median SMDs ‐0.42 and ‐0.43 respectively).

The nine interventions which included first‐person narratives and provided SMD data had a medium median effect size (‐0.51); whereas for five studies with third‐person narratives the median effect size was negligible at ‐0.03; and for the seven interventions with no narratives there was a small effect size (median SMD = ‐0.27). Data and studies contributing to these medians can be seen in Figure 8 for non‐cluster trials. For cluster trials the data can be seen in Figure 7 with two studies (Penn 2003; Coleman 2005) containing first‐person narratives and one (Yoshida 2002) containing third‐person narratives. Only one of the narratives involved a celebrity (an emailed newspaper article about Nobel Prize‐winning mathematician John Nash who was diagnosed with schizophrenia) (Corrigan (submitted) intervention A) and this had a small effect size (SMD ‐0.23, 95% CI ‐0.63 to 0.16), whereas the interventions with narratives by/about people who were not celebrities had a median SMD of ‐0.48. For data and studies contributing to this median, see Figure 9 and all studies in Figure 7. When we compared fictional and non‐fictional narratives, there was little difference with median SMDs for each group respectively being ‐0.42 and ‐0.47. Data and studies contributing to these medians can be seen in Figure 10 for non‐cluster trials. For cluster trials the data can be seen in Figure 7 with two studies (Penn 2003; Coleman 2005) containing non‐fictional narratives and one (Yoshida 2002) containing fictional narratives.

8.

Forest plot of comparison: 5 Mass media vs. control by presence of narratives, outcome: 5.1 Prejudice (at earliest follow‐up time point).

9.

Forest plot of comparison: 6 Mass media vs. control by celebrity narratives, outcome: 6.1 Prejudice (at earliest follow‐up time point).

10.

Forest plot of comparison: 7 Mass media vs. control by fictional narratives, outcome: 7.1 Prejudice (at earliest follow‐up time point).

As the interventions were categorised into eight different subgroups for type of message, there were few (one to three) interventions within most subgroups. One subgroup ‐ 'recovery‐oriented' messages ‐ had effect size data from five interventions (Yoshida 2002; Reinke 2004 interventions A and B combined; Matthews 2009; Brown 2010 intervention A; Corrigan (submitted) intervention A) and found a small prejudice‐reducing median SMD (‐0.30). There was a large reduction in prejudice for a single study with a social inclusion/human rights message (SMD ‐2.94, 95% CI ‐3.52 to ‐2.37) (Russell 1988). Medium prejudice‐reducing effects were found for interventions with multiple primary messages (median SMD = ‐0.70) (Penn 2003; Smith 2007; Finkelstein 2008) and those with 'not to blame' messages (median SMD = ‐0.51) (Woods 2002; Woods 2003; Woods 2005). 'Seek professional care' messages had small (median SMD = ‐0.43) prejudice‐reducing effects (Morgan Owusu 2002; Demyan 2009; Jorm 2010a); and negligible effects (median SMD = ‐0.13) were found for 'negative impact of mental illness' messages (Reinke 2004 intervention C; Corrigan (submitted) intervention B). Some types of message were increased levels of prejudice. A small prejudice‐increasing effect (median SMD = 0.22) was found for 'this is hallucinatory experience' messages (Bunn 2009; Brown 2010 intervention B); and a large (median SMD = 0.99) prejudice‐increasing effect for biomedical messages (Coleman 2005; Iobst 2008 interventions A and C combined). The studies and data contributing to the medians for the messages subgroup analysis can be seen in Figure 11 for non‐cluster trials. For cluster trials the data can be seen in Figure 7 with Penn 2003 having multiple primary messages; Coleman 2005 having a biomedical primary message, and Yoshida 2002 having a recovery‐oriented primary message.

11.

Forest plot of comparison: 8 Mass media vs. control by type of message, outcome: 8.1 Prejudice (at earliest follow‐up time point).

When the interventions were examined by type of media, audiovisual and print interventions were found to have similar median SMDs of ‐0.47 and ‐0.46 respectively. A small prejudice‐reducing effect was also found for Internet interventions (median SMD = ‐0.30). For the two audio interventions (Bunn 2009; Brown 2010 intervention B) the median effect size represented a small increase in prejudice at 0.22. The studies and data contributing to these medians can be seen in Figure 12 for non‐cluster trials. For cluster trials the data can be seen in Figure 7 with two (Penn 2003; Coleman 2005) being audiovisual and one (Yoshida 2002) being a print intervention.

12.

Forest plot of comparison: 9 Mass media vs. control by type of media, outcome: 9.1 Prejudice (at earliest follow‐up time point)).

Meta‐analysis

See Data and analyses. We examined the values of the Q (Chi²) statistic and I² index for studies with forest plots for overall effects by follow‐up time point and for each subgroup in the subgroup analyses for both primary outcomes. No meta‐analysis was appropriate for the secondary outcomes due to the disparate and sparse nature of the data. There was substantial statistical heterogeneity (I² > 50%) (Higgins 2011, section 9.5.2) in all groups apart from the following:

1. Discrimination outcome: main analysis, cluster trials, dichotomous outcomes:.Tau² = 0.00; Chi² = 0.03, df = 1 (P = 0.85); I² = 0% (Yoshida 2002; Penn 2003)

2. Prejudice outcome: main analysis, non‐cluster trials, 6 to 9 months follow‐up: Tau² = 0.01; Chi² = 1.50, df = 1 (P = 0.22); I² = 33% (Finkelstein 2008; Jorm 2010a)

3. Prejudice outcome: Third person narratives: Tau² = 0.03; Chi² = 5.67, df = 3 (P = 0.13); I² = 47% (Iobst 2008; Demyan 2009; Matthews 2009; Corrigan (submitted))

4. Prejudice outcome: Non‐celebrity narratives: Tau² = 0.00; Chi² = 0.48, df = 1 (P = 0.49); I² = 0% (Matthews 2009; Corrigan (submitted), intervention B)

5. Prejudice outcome: 'Not to blame' messages: Tau² = 0.00; Chi² = 0.27, df = 2 (P = 0.87); I² = 0% (Woods 2002; Woods 2003; Woods 2005)

Although there was statistical homogeneity regarding the dichotomous discrimination outcomes, the two studies (Yoshida 2002; Penn 2003) were very dissimilar (different media, primary messages, and number of components and one intervention being fictional and the other non‐fictional), and were also heterogenous methodologically, one assessing outcomes immediately post‐intervention and the other only at nine months follow‐up. Consequently, meta‐analysis was considered inappropriate.

For prejudice outcomes in the main analysis there was also one group of studies for which I² was < 50%. (non‐cluster trials, Finkelstein 2008; Jorm 2010a at 6 to 9 months follow‐up). These studies were considered to be clinically and methodologically homogenous, both being three‐arm studies of Internet versus print versus control multi‐component with the Internet arms involving first‐person non‐fictional narratives. Therefore meta‐analysis was deemed appropriate and the combined effect size was SMD = ‐0.36 (95% CI ‐0.63 to ‐0.10) demonstrating a reduction in prejudice.

We concluded that meta‐analysis was not warranted for the studies in the first subgroup with I² below 50% (third‐person narratives, prejudice outcome) due to substantial clinical heterogeneity. For example the third‐person narrative interventions portray disparate conditions (autism, psychosis and exam stress), varied media types (Internet, audiovisual and print) and different types of primary message (biomedical, seek help, recovery‐oriented, and multiple).

There is more clinical similarity in the second subgroup above (non‐celebrity narratives, prejudice outcome) in that both address psychosis and are print journalism (delivered in print format or by the Internet), although their primary messages are very different, with one being recovery‐oriented and the other focusing on the negative impact of mental illness. The former is a magazine article recounting the story of a young woman successfully recovering from an episode of psychosis with the help of mental health services. The latter is a newspaper piece telling the story of a man who stabbed himself to death in prison during an acute psychotic episode whilst guards looked on, and focuses on poor mental healthcare provision. We therefore concluded that meta‐analysis was not appropriate here due to clinical heterogeneity.

For the third subgroup with I² < 50% (interventions with 'not to blame' primary messages) the studies are highly homogenous with identical outcomes, interventions, and population types and similar methodologies, and all were conducted by the same author. For the prejudice but not the discrimination outcome, statistical homogeneity was also confirmed by visual inspection of the forest plot and by the heterogeneity statistics cited above. The combined effect size was SMD = ‐0.52 (95% CI ‐0.71 to ‐0.33) demonstrating, in line with the findings of the individual studies, a reduction in prejudice, but providing a more precise estimate of the effect.

Sensitivity analysis

Much of the sensitivity analysis planned was precluded since so few data were appropriate for meta‐analysis.

In the meta‐analysis of prejudice outcomes in the studies by Woods (Woods 2002; Woods 2003; Woods 2005) we examined the effects of removing Woods 2002 as this was at high risk of attrition (the studies did not differ on any other type of risk). This marginally increased the combined effect size from ‐0.52 (95% CI ‐0.71 to ‐0.33) to ‐0.55 (95% CI ‐0.76 to ‐0.33). Removing the less precise studies (Woods 2002; Woods 2005) also increased the effect size marginally from ‐0.52 (95% CI ‐0.71 to ‐0.33) to ‐0.56 (95% CI ‐0.83 to ‐0.30). Using a fixed‐effect model did not alter the findings with the combined effect size remaining at ‐0.52 (95% CI ‐0.71 to ‐0.33).

We undertook a sensitivity analysis to investigate the impact of using estimated ICCs in the cluster trials which did not account for the design effect in their analysis and for which no raw data were available (Penn 2003; Coleman 2005). The SMD between intervention and control arms was insignificant in both studies after adjusting standard errors for clustering. Any extra adjustments to penalise the studies further would result in the outcomes still remaining insignificant at the 5% significance level. Effect estimates of these studies unadjusted for the clustered data remained insignificant.

Removing the multi‐arm studies which included arms that the study authors considered unlikely to reduce stigma (Reinke 2004; Brown 2010; Corrigan (submitted)) in a post hoc sensitivity analysis changed the median SMD for prejudice outcomes at immediate follow‐up from ‐0.38 to ‐0.44 (range unchanged). As both values represent a small effect size using Cohen's (Cohen 1988) rule of thumb, we considered that the findings were not substantially altered by this removal.

Secondary outcomes

Four studies (n = 1213) (Morgan Owusu 2002; Yoshida 2002; Finkelstein 2008; Jorm 2010a) examined knowledge and each used a different type of outcome and statistic (see Table 4). A narrative synthesis (Grimshaw 2003) of the knowledge data shows that for all seven interventions in the four studies the direction of effect was positive (increasing knowledge) and for five of the seven interventions the effect was statistically significant.

3. Mass media vs. control: Knowledge.

Study ID	Measure	Intervention arm	Intervention group summary statistic	N(I)		Control group summary statistic	N(C)	Effect estimate/test statistic and P value/95% CI
Finkelstein 20081	Psychiatric Knowledge Survey	A (Internet education)	Immediate follow‐up: 90.5% items correct Six‐months follow‐up: 65.6% items correct	69		43.9% items correct	48	Pre‐post Χ2 P value < 0.0001
		B (Printed documents)	Immediate follow‐up: 56.3% items correct Six‐months follow‐up: 49.6% items correct	76		43.9% items correct	48	Pre‐post Χ2 P value < 0.0001
Jorm 2010a2	Four knowledge items, beliefs about depression treatments selected	A (CD‐ROM)	65.8%		73	52.4%	82	OR for pre‐post intervention interactions = 2.11 (95% CI 0.61 to 7.3)
		B (manual)	67.9%		83	52.4%	82	OR for pre‐post intervention interactions = 3.64 (95% CI 1.01 to 13.09)
Morgan Owusu 20023	Seven knowledge items, item 1 (what is mental illness) selected	A (standard video)	96.6% (29/30)	30		76.7% (23/30)	30	OR = 8.83 (95% CI 1.01 to 76.96)
		B (culturally specific video)	96.6% (29/30)	30		76.7% (23/30)	30	OR  = 8.83 (95% CI 1.01 to 76.96)
Yoshida 20024	Knowledge of Mental Illness scale	Booklet	3.22 (1.48)	301		3.03 (1.41)	391	Adjusted mean difference = 0.20, 95% CI ‐0.06 to 0.46, P = 0.133)

[1] Outcome measure used in two previous studies (Jorm 1997; Griffiths 2004); control group same for both intervention arms; statistic from paper, OR or SMD not calculable.

[2] Four items, if recognised depression/schizophrenia from vignettes of person portrayed as meeting DSM‐IV‐TR criteria for the conditions DSM (Jorm 1997); two items developed for study: beliefs about treatments for schizophrenia/depression (% of all type treatments deemed by authors to be helpful endorsed by respondent; outcome for review as specified per protocol (Brennan 2009); data from earliest follow‐up time point (1 month). N assumed from consort diagram as not stated.

[3] Items developed for study, dichotomous (correct or not); item 1 selected as knowledge outcome for review as specified per protocol [Grimshaw 2003]

[4] Six‐item scale developed for study; continuous scale; linear regression model fitted including a random effect for clustering by sampling design (19 areas, average observations per cluster 36); from authors dataset; results presented are unadjusted means (SD) by arm along with the adjusted mean difference between arms, corresponding 95% confidence interval and P value.

None of the publications provided any information about cost, although three authors of studies involving 416 participants provided intervention cost data on request. In an audio visual intervention the costs were estimated to be 100 US dollars (equivalent to £64 GBP) for the telecommunications students who developed the intervention (Demyan 2009). For a mental health first‐aid intervention the purchase costs were 35,000 Australian dollars (£22,404 GBP) for 250 e‐learning CDs at 140 dollars per CD and 7140 Australian dollars (£4570 GBP) for 238 manuals at 30 dollars per manual (Jorm 2010a). A third author stated on request that the only costs involved were for printing (Matthews 2009). No data were presented on cost effectiveness.

No studies examined reach or recall. One study (Yoshida 2002) assessed awareness of the mass media intervention with data on 727 participants and found that at nine months follow‐up 33% (106/321) of the intervention group reported 'having seen the booklets', as did 6.6% (27/406) of the control group.

Regarding the duration/sustainability of media effects, in 14 studies outcomes were measured immediately post‐intervention only. Five studies with analysable outcome data (n = 1225) had longer follow‐up and so could provide some information about duration or sustainability of effects. Discrimination was assessed at 9 months post‐intervention in one study (Yoshida 2002) and there was no evidence of effect at this time point (see Figure 5). It can be seen from Table 3, and as reported in the prejudice main comparison section, that there is evidence of prejudice reduction 1 week to 2 months, and 6 to 9 months after the interventions. It is also informative to consider studies which assessed outcomes at two post‐intervention time points. One study (Finkelstein 2008) assessed prejudice immediately post‐intervention and at six months post‐intervention. This study reported a large reduction in prejudice at immediate follow‐up and a medium reduction at six months (see Figure 6). One study assessed effects at one and six months post‐intervention (Jorm 2010a) and found a medium effect size at one month which was reduced to a small one at six months. One study (Kerby 2008) assessed prejudice immediately post‐intervention and at 8 weeks, and, although no data were available to calculate SMDs, in the paper authors reported a within‐group comparison of baseline to immediate follow‐up of z = ‐2.78 P = 0.005 and stated that the same test for immediate to 8 weeks follow‐up found 'no significant differences', suggesting a lasting effect. Brown 2010 measured prejudice immediately post‐intervention and one week later, and our analyses, which combined interventions, found no evidence of effect at either time point, although these analyses combined two interventions which had different effects. In the paper (Brown 2010) the authors reported that at one week follow‐up the prejudice‐reducing effects of the filmed contact (intervention A) partially persisted, however for simulated hallucinations (intervention B) at one week there was a small effect size indicating a continued increase in prejudice. However, the latter finding, as with Kerby 2008, should be viewed with caution as it is a within‐group comparison.

Three studies (n = 381) presented numerical data on the audience reactions to the interventions (Yoshida 2002; Finkelstein 2008; Jorm 2010a). These data were solely on favourability and none addressed information or message communicated. See Table 5. Reactions were positive to all five interventions in the three studies.

4. Audience reactions to interventions.

Finkelstein 2008	Intervention A (Internet education) %, n = 69	Intervention B (printed articles) %, n = 76
Liked educational experience in general (yes/probably yes)	97.1	89.3
Topic is important (yes/probably yes)	100	97.4
Content is useful (y) (yes/probably yes)	98.6	90.7
Content is interesting (y) (yes/probably yes)	100	86.8
Content is unpleasant (no/probably no)	81.2	73.3
Content is difficult to understand (no/probably no)	89.9	61.8
Liked way information is presented (y) (yes/probably yes)	87.0	60.5
Jorm 2010a	Intervention A (CD‐ROM) %, n = 63 to 64	Intervention B (manual) %, n = 62 to 66
Read most/all of material	95.3	100.0
Material was easy/very easy to understand	96.9	97.0
Learnt a great deal/a fair bit from material	90.6	90.1
Material was useful/very useful	93.8	98.4
Will use intervention in future	73.0	90.3
Would probably/definitely recommend intervention to others	96.9	98.4
Yoshida 2002 (N = 106)1,2	Mean (SD)
Favourability rating (1 to 5)	3.67 (0.85)
Looking forward to next issue of booklet rating (1 to 5)	3.67 (1.04)
Recommendation to others rating (1 to 5)	3.29 (0.78)

[1] Among participants who were aware of the intervention

[2] High rating indicates a positive audience reaction

Only two studies (n = 455) made any mention of unforeseen adverse effects (other than increases in discrimination and prejudice). One (Jorm 2010a) stated ‘Given that this was an educational intervention with a non‐clinical sample, there was no formal enquiry about adverse events. Informally, no adverse events were reported'. Another (Finkelstein 2008) stated ‘No adverse effects, such as an increase in stigma as a result of the intervention, were identified’.

Discussion

Summary of main results

Our review identified 22 eligible studies with a total of 4490 participants, and 19 of these studies had primary outcome data. All studies were randomised controlled trials (RCTs) and were highly heterogeneous. Participants included the general population, employers, and students training for health/educational roles, but the majority (n = 13) involved other types of students. Interventions spanned Internet, audiovisual, audio and print media. Discrimination was assessed in only five studies, prejudice in all. Data on secondary outcomes were relatively sparse. Outcomes were most often assessed only immediately post‐intervention (n = 14), although three studies had follow‐up beyond six months (Yoshida 2002; Finkelstein 2008; Jorm 2010a).

For discrimination outcomes, one study (Woods 2002) found evidence of reduced discrimination but this was not replicated in two larger nearly identical studies by the same author (Woods 2003; Woods 2005), which found no evidence of effect, as did two further studies (Yoshida 2002; Penn 2003). The effect size was small or negligible for discrimination, and we conclude that these findings are compatible with mass media having either a positive effect, a negative effect or no effect on discrimination.

Our review showed that, overall, mass media interventions reduce prejudice and that the size of this effect can be considered small‐to‐medium (Cohen 1988) with standardised mean differences (SMDs) of ‐0.38, ‐0.38 and ‐0.49 for the three follow‐up time‐point categories (see Table 3). To further aid interpretation, we transformed effect sizes into the equivalent number of points on Link's Social Distance Scale (SDS) (Link 1999) using a population standard deviation of 0.59 for social distance in relation to schizophrenia using the SDS from the observational study, General Social Survey 1996, USA (Link 1999). On this scale, respondents can score a minimum of 1 and a maximum of 4. The overall median effect sizes for prejudice outcomes immediately post‐intervention, at 1 week to 2 months and at 6 to 9 months are the equivalent of reductions of 0.22, 0.22 and 0.29 points on the SDS. We can further extrapolate from data about SDS scores for different mental health conditions (Link 1999) that the overall effects of mass media interventions found in our review are similar to reducing the level of prejudice from that associated with symptoms of schizophrenia (mean SDS 2.75) to the level associated with symptoms of major depression (mean SDS score 2.54) as the difference between the two is 0.21. The four members of the stakeholder group and review team with declared personal experience of mental ill health were asked to rate the importance of this degree of reduction, and three rated it as 'quite important' and one as 'slightly important'.

This overall finding of a small‐to‐medium reduction in prejudice conceals the considerable variation in the effects of different individual interventions. The prejudice subgroup findings help to explain some of this variation. The clearest pattern of evidence is for the presence of first‐person narratives, which were found to be effective in reducing prejudice. Interventions with two or more components tended to reduce prejudice more than those with only one. Other findings are tentative due to the small number of interventions in subgroups..

The individual studies with large effect sizes merit comment. The two studies (Russell 1988; Finkelstein 2008) with large prejudice‐reducing effects took place in atypical settings. The former was dated 1988 and was a much earlier study than the others; the latter was the only study in a non‐high‐income country. It is possible that there may have been higher baseline prejudice in these contexts, making it easier to achieve large change. Alternatively the large effects may be due to factors inherent in the interventions, the former using three mailings of postal booklets with social inclusion/human rights messages and the latter a computer‐assisted educational programme containing personal narratives. The study with a large prejudice‐increasing effect (Coleman 2005) was the only first‐person narrative intervention to have a primarily biomedical message and the film’s title, 'Fires of the Mind: Dark Voices: Schizophrenia', emphasises acute illness. This highlights the importance of the type of narrative content.

Overall completeness and applicability of evidence

Completeness of evidence

There were a number of areas where the evidence was incomplete, thereby limiting the external validity of the review findings. Although two types of study design were eligible for inclusion, we located very few ITS studies and none that met the inclusion criteria of having defined start and end points and at least three time points before and after the intervention. This is surprising, as ITS designs are common in studies evaluating other types of mass media intervention (Grilli 2002; Vidanapathirana 2005). Only five included studies examined our primary outcome of discrimination, but all included studies examined prejudice, our other primary outcome. Discrimination, being a behavioural outcome, is more difficult to measure, but is of greater importance than prejudice for improving the lives of people with mental health problems. Data on all secondary outcomes were sparse. We found no studies conducted in lower‐middle, middle or low income countries, and none in which the populations were children or adolescents. The populations studied did not cover the full range of target groups who may stigmatise people with mental ill health. Four involved the general public and a further four targeted students training for health‐related professions. However, none targeted practising health professionals, and only one study included employers despite these groups being identified as common sources of mental health‐related stigma (Schulze 2007; Brohan 2010).

None of the studies tested long‐term (over three months) interventions and none investigated interventions combining more than one type of mass media, and yet these approaches are typical of several national and regional anti‐stigma programmes, e.g.Vaughn 2004; Dunion 2005; Henderson 2009. These programmes also often combine mass media with non‐mass media components. Although we only included studies where the mass media component comprised more than 50% of the intervention, we only located two studies combining mass media and non‐mass media, so the evidence remains relatively incomplete in this regard. Some types of mass media were not investigated by any of the included studies, specifically television, radio, cinema and mobile phone media. Only one study included a celebrity narrative (Corrigan (submitted), intervention A), whereas this is relatively common in national mass media campaigns (e.g. Vaughn 2004; Henderson 2009). No studies addressed stigma arising from mental ill health at the same time as addressing stigma due to other attributes.

The data were incomplete in several studies, as many authors did not provide important details, e.g. about randomisation methods (see Characteristics of included studies); attrition (see Figure 2); and about numbers of participants contributing data to each study arm (e.g. see footnotes to Figure 6). Furthermore, means and standard deviations were not always reported in papers and sometimes had to be requested from the study authors, or estimated or derived from other sources.

Applicability and transferability of evidence

The findings cannot necessarily be transferred to other contexts, populations and interventions not covered in the review (described in the section above on the completeness of the evidence). There are also issues around feasibility and resources. Some interventions are likely to be too costly to implement in many settings, although not enough is known about costs to state which are most likely to be problematic to implement for this reason. The majority of the print and Internet interventions were text‐based, and required some degree of literacy, which may preclude their use with some populations (Clement 2009). The availability of technology is a further factor which may limit the transferability of the findings. As anti‐stigma interventions are often led by people with personal experience of mental health‐related stigma or are conducted in partnership with them, their views will frequently shape the nature of mass media interventions used. For example, mental health professionals and people with mental ill health differ in their views about which types of messages should be included in anti‐stigma interventions (Clement 2010). The values and preferences of people with mental health problems may also influence whether mass media interventions are used at all; they may prefer face to face interventions as these provide greater opportunities for personal empowerment and employment (Clement 2012).

Quality of the evidence

See Table 1. We had pre‐specified in the protocol (Clement 2011) that the main outcomes for assessment of the quality of the evidence were: discrimination towards people with mental ill health; prejudice towards people with mental ill health; cost; and unforeseen adverse effects. We made a post‐hoc decision to categorise discrimination as a critically‐important outcome and the remainder as important. Although all of the studies were randomised controlled trials which are considered to provide a high quality of evidence due to their design, for each outcome in this review we considered study limitations, inconsistency of results, the indirectness of the evidence, imprecision or other considerations, and downgraded the quality where appropriate (Guyatt 2008).

Critically‐important outcomes

Discrimination

Discrimination was assessed in five of the included studies. The evidence is affected by study limitations (risk of bias) as the majority (30/35) of 'Risk of bias' items were rated as unclear or high. We did not downgrade for inconsistency as the studies were fairly consistent in showing no evidence of effect. The discrimination measures were somewhat indirect, in that although four studies assessed behaviour in situations in which participants believed they would be interacting with a person with mental ill health, no observation of an actual interaction was made, and one study used participants' reports of their behaviour in real‐life settings as its measure (Yoshida 2002). Also, for each type of discrimination outcome the behaviour is not an indicator of discrimination in every instance. Consequently we downgraded the evidence for indirectness. The studies were not considered imprecise as only a minority (2/5) had outcome data from less than 100 participants (Woods 2002; Penn 2003). No other limits were noted (lack of validity of outcome measures was already included in our 'Risk of bias' ratings). Consequently the quality of evidence for discrimination was downgraded by two levels to 'low' for this outcome.

Important outcomes

Prejudice

All included studies assessed prejudice. There were study limitations due to risk of bias, as the majority (105/133) of the 'Risk of bias' indicators were classified as unclear or high risk. Overall the results were fairly consistent for 16 of the 19 studies with analysable outcome data (see Figure 6 and Figure 7); therefore the evidence was not downgraded for inconsistency. The evidence was limited by indirectness of study populations. with generic student samples commonly used as proxies for the general population. Imprecision was not a large problem as only a minority (5/19) of studies had outcome data from less than 100 participants. No other limits were noted. Consequently the quality of evidence for prejudice was also downgraded by two levels to 'low'.

Cost

No studies included any information on cost or cost effectiveness in papers. Three study authors provided data on the costs of the interventions on request (Demyan 2009; Matthews 2009; Jorm 2010a). These studies were not considered to be compromised by study limitations as the majority (12/21) of 'Risk of bias' items were rated as low risk, and there were only four high risk ratings. There was inconsistency of results, with costs varying widely. The data did not suffer from indirectness, but there was imprecision in the costs for two (Demyan 2009; Matthews 2009) of the three studies. Other considerations also limited the quality of the evidence such as the lack of cost‐reporting in papers and lack of cost‐effectiveness data. Consequently we downgraded the quality of the evidence by three levels to 'very low'.

Unforeseen adverse outcomes

Two studies contained statements about unforeseen adverse outcomes (Finkelstein 2008; Jorm 2010a). These studies were considered to be compromised by study limitations as the majority (9/14) of 'Risk of bias' items were rated as unclear or high risk. There was consistency of results and the data did not suffer from indirectness. However there was imprecision, in that one study did not formally assess adverse outcomes and neither specified their methods. No other limits were noted. Consequently we downgraded the quality of the evidence by two levels to 'low'.

Potential biases in the review process

We searched 11 databases, including grey literature databases and a non‐English language database. Limited translation of the MEDLINE strategy to other databases (notably CINAHL, CENTRAL, EMBASE and PsycINFO) will have impaired sensitivity and specificity. The impact of these shortcomings is difficult to determine; it may have resulted in the loss of relevant studies and influenced the results of the review. Resource limitations precluded the conduct of new searches before publication. However the electronic searching was supplemented by checking of references, citations and websites and communication with experts, which yielded eight further studies, all unpublished. It is therefore feasible that all relevant studies have been identified. We welcome contact from any authors who believe their studies may be relevant to this review.

The restriction to RCTs and ITS studies meant that we were unable to consider data from other types of evaluation of mass media interventions such as qualitative evaluations and before and after studies. Although RCTs are the highest form of evidence for evaluating interventions, the RCTs in our review were subject to a number of biases and other limitations (see Figure 2; Quality of the evidence).

As 10 of the 22 studies had 2 or more active arms, in our main analysis we combined these, as per our protocol (Clement 2011). However the arms were often quite different and had different effects. This had the effect of making our overall findings more conservative than they might otherwise have been. A further issue was the multiplicity of prejudice outcome measures evident in 12 of the studies. In most of these, no primary outcome was defined by the study author and no power calculation was undertaken (our second basis for selecting the outcome for analysis). Consequently we used methods which select the outcome for analysis as being the one with the median effect size (Grimshaw 2003; Brennan 2009). This obscures larger effects found for some outcome measures.

Our pre‐specified classification of all attitudinal and emotional outcomes into the single category of prejudice precluded the examination of possible differential effects on different types of prejudice. For example, some interventions may increase empathy but also increase social distance (Brown 2010 intervention B), or may reduce blame but increase assumptions about poor prognosis (Penn 2003).

It is possible that particular combinations of characteristics of a mass media intervention are important to its effectiveness, e.g. type of message within, for example, a personal narrative, using a particular medium. A hierarchical subgroup analysis might have been illuminating in this regard, but would have required substantially more data than were available in this review.

Agreements and disagreements with other studies or reviews

Our findings are broadly in line with other studies and reviews, whilst making an important additional contribution to the existing knowledge base. Several systematic reviews have investigated the effects of anti‐stigma interventions which have included mass media interventions. A systematic review of 71 outcome studies using contact, education or protest‐based approaches to reduce mental health‐related stigma (Corrigan 2012) has recently reported findings relevant to our review. Corrigan and colleagues' review did not specifically address the effectiveness of mass media interventions of all types, but did compare outcomes for face‐to‐face and video‐based contact, the latter being the equivalent of the mass media first‐person narrative interventions in our review. They found that both approaches were beneficial (Corrigan 2012). The effect sizes found for video‐based contact were smaller than those found in our review, with a mean d = ‐0.296 for attitudes and ‐0.197 for behavioural intentions (Corrigan 2012) compared to our finding of a median effect size of ‐0.51 on prejudice. This difference may be attributable to Corrigan's findings being based on all types of study design, not just RCTs. Corrigan and colleagues found video‐based contact to be less effective than face‐to‐face contact (Corrigan 2012), although two RCTs have found the delivery modes to be equivalent (Reinke 2004; Clement 2012).

A systematic review of school‐based interventions to reduce mental health‐related stigma noted a dearth of RCT evidence, poor methods, considerable clinical heterogeneity, and inconsistent or null results (Schachter 2008). It recommend the development, implementation and evaluation of a curriculum which fosters the development of empathy and, in turn, an orientation toward social inclusion and inclusiveness. These effects may be achieved largely by bringing especially but not exclusively the youngest children into direct, structured contact with an infant, and likely only the oldest children and youth into direct contact with individuals experiencing mental health difficulties. Similarly, a narrative review of educational interventions for young people concluded that direct contact with people with mental health problems seems to be key in reducing stigmatisation, while the components of education and video‐based contact conditions are still arguable (Yamaguchi 2011). A systematic review of target group‐oriented interventions to reduce mental health‐related stigma did not report any findings on the efficacy of mass media interventions, although some of the studies included in the review did have mass media elements (Holzinger 2008).

These reviews report that discrimination outcomes are rarely measured (Yamaguchi 2011; Corrigan 2012). Evaluations of national anti‐stigma programmes which have mass media components have sometimes examined reports of discrimination experienced by people with mental ill health, and have found reductions in discrimination (Wyllie 2011; Henderson 2012). These studies did not use the types of study design eligible for inclusion in this review, but their findings suggest that anti‐stigma programmes may have the potential to reduce discrimination. We found very little information about costs. McCrone and colleagues have applied economic modelling to data from Scotland's anti‐stigma campaign (McCrone 2010). They report that If the campaign caused 10% of changed attitudes then it was estimated to cost £35 per one less person who felt that people with mental health problems were dangerous, and £186 per one less person who felt the public needs protection from people with mental health problems.

The problematic nature of biomedical messages apparent in our review has also been demonstrated in two recent systematic reviews on public attributions for mental illness and levels of stigmatisation (Angermeyer 2011; Schomerus 2012), and in a consensus development study (Clement 2010). The latter study recommended the use of ‘see the person’ messages, and our finding about the benefits of interventions echoes this. A systematic review of simulated hallucination interventions to reduce stigma (Ando 2011) was in accordance with this review's finding that such interventions can be problematic with regard to stigma, especially when not combined with pre‐ and post‐simulation activities. However, the review's qualitative findings from studies which had included a pre‐intervention DVD developed by a researcher with personal experience of hearing voices, and a post‐simulation debriefing discussion, suggest that when delivered in this way, simulated hallucinations may have a number of beneficial effects.

When we compare our review findings with systematic review evidence about the effectiveness of mass media interventions in other fields, we find that studies on promoting HIV testing (Vidanapathirana 2005) and increasing health service utilisation (Grilli 2002) provided more consistent evidence for the effectiveness of mass media interventions than did the studies in our review. Our preliminary findings about the benefits of multi‐component interventions echo Brinn and colleagues' finding that the more effective interventions tended to be more intensive (Brinn 2010). Our review is also in agreement with one on mass media and HIV testing (Vidanapathirana 2005) in highlighting the need for more research on different types of media, cost effectiveness and on the characteristics of messages.

Authors' conclusions

Implications for practice.

This review provides evidence that mass media interventions may reduce prejudice in the immediate, short and medium term. Overall, the extent to which they do this may be considered small‐to‐medium, roughly equivalent to reducing the level of prejudice from that associated with symptoms of schizophrenia to the level associated with symptoms of major depression, and mainly rated as ‘quite important’ by consumers. Some types of intervention had larger beneficial effects. Consequently there is justification for continuing to use mass media as one strategy for countering the stigma associated with mental ill health. Also, as mass media interventions have the potential to reach large numbers of people, even small benefits may have important effects at the population level. Mass media interventions were able to increase knowledge and were well received in the small number of studies which assessed these outcomes.

This review has implications for the types of mass media intervention that are likely to be most beneficial. On the basis of our subgroup finding the use of first‐person narratives may be promising. Social inclusion/human rights messages may be effective in reducing prejudice but this result is based on only one study. We found that the portrayal of acute symptoms or bio‐medical messages may contribute to increasing prejudice. Therefore anti‐stigma interventions should avoid including such messages or should use them with caution.

As overall the findings from the five trials with discrimination outcomes were mixed: with effects showing a reduction, increase or evidence of no effect of mass media on discrimination, it is not possible to draw implications for practice about whether, or how, mass media interventions reduce discriminatory behaviour.

These implications must be considered as somewhat uncertain due to limitations in the quality of the available evidence (see Quality of the evidence). As discussed in Overall completeness and applicability of evidence, the evidence reviewed here cannot necessarily be generalised to all contexts, populations or mass media interventions. Furthermore, in making decisions about the use of mass media to reduce mental health‐related stigma, policy makers will need to consider: the resources and technologies available in the setting in question; literacy levels; and the values and preferences of relevant stakeholders.

Implications for research.

The key research question of whether mass media interventions reduce discrimination remains unresolved by this review, due to the absence of evidence of effect, and limitations in quality of the evidence. The low quality of the evidence meant that the findings for prejudice were uncertain. We were not able to assess cost‐effectiveness, reach, recall and awareness, and adverse events due to scarcity of evidence. Given the large number of mass media anti‐stigma interventions in use worldwide; the serious impact of stigma on individuals and societies, and the consequent need to find the best ways to address it; the high cost of some mass media interventions and importance of assessing cost effectiveness as well as cost (McCrone 2010); and the possibility of some interventions increasing prejudice; it is imperative that research is undertaken to fill these evidence gaps.

Research is particularly needed in the following more neglected areas: in low‐ and middle‐income countries; on children and adolescents; on non‐student populations such as employers and health professionals; on television, radio, cinema and mobile phone media; on interventions combining more than one type of mass media; on interventions repeated over time and those lasting more than a month. Clearly, the content of interventions is crucial to their success. More research is needed to build on what is known about effective messages for reducing mental health‐related stigma. This review found evidence to support the effectiveness of first‐person narratives, but we need to know more about the optimum characteristics of the narrator(s) and the narratives shared (Clement 2012). Little is known about the other forms of indirect social contact such as written autobiography and pictorial representation. Further research is also needed on celebrity (Corker 2011), and on fictional narratives.

Future studies should build in outcome assessment beyond immediately post‐intervention and preferably to six months follow‐up or longer. More studies should include discrimination as an outcome, and more validation of such behaviour measures is needed alongside the development of more direct discrimination outcomes. Given the multitude of prejudice outcomes and their conceptual differences, it would be helpful for future research to develop a toolkit of validated measures that assess key stigma domains. Future synthesis might usefully include a number of prejudice outcomes such as social distance, blame, empathy, coercion beliefs, and prognosis beliefs. From a broader perspective, research could also usefully address the potential impact of public anti‐stigma campaigns on the anticipated discrimination or internalised stigma of people with mental health problems (Evans‐Lacko 2012b). This review has not addressed whether reductions in discrimination and prejudice result in greater social inclusion of people with mental health problems, but clearly this is a vital question.

We advocate the greater use of robust study designs such as randomised controlled trials (RCTs) and interrupted time series studies. When RCTs are used, researchers should address the common risks of bias found in this review. Cluster designs may sometimes be appropriate, but analyses need to take the design effect into account. Finally, power calculations should be undertaken to ensure adequate sample sizes.

Appendices

Appendix 1. Cochrane Central Register of Controlled Trials search strategy

CENTRAL, The Cochrane Library, Issue 7, 2011

stigma OR discrimination OR prejudi* OR social perception OR social distance OR human rights

AND

mental* OR  psych* OR depress* OR schizo* OR bipolar OR anxiety OR substance OR alcohol OR dementia OR intellectual disabil* OR learning disabil* OR retardation OR anorex* or bulimi* OR obsessi* OR phobi* OR panic

AND

media OR communication OR television OR radio OR film OR cinema OR movie OR newspap* OR internet OR video OR DVD OR publication OR advert* OR social market* OR campaign* OR messag* OR narrative OR social contact OR audio* OR virtual OR health promotion OR online

315 records found

Appendix 2. MEDLINE and EMBASE search strategy

MEDLINE (OvidSP),1966 to 15 August 2011

EMBASE (OvidSP) 1947 to 15 August 2011

1. stereotyping/

2. (stereotyp* or stigma* or label* or negative image* or ignoran* or misconception* or misperception* or literacy or ((public* or community or social or popular) adj perception*)).tw.

3. social perception/

4. public opinion/

5. prejudice/

6. exp attitude/

7. ((public* or community or social or popular) adj attitude*).tw.

8. (((negative or positive or chang*) adj3 attitude*) or prejudice* or hostil* or intoleran*).tw.

9. social distance/

10. rejection psychology/

11. human rights/

12. (rights or discriminat* or marginali* or rejecting behavior or injustice* or (social adj (distance or justice or rejection or acceptance or exclusion or inclusion))).tw.

13. or/1‐12

14. mental health/

15. mental health services/

16. exp mental disorders/

17. mentally ill persons/

18. ((mental* or psychiatr* or psychological* or developmental* or learning or substance*) adj (ill* or disorder* or disease* or distress* or disab* or problem* or health* or well‐being or wellbeing or patient* or treatment or retardation)).tw.

19. ((chronic* or severe* or serious* or persistent) adj (mental* or psychiatr* or psychological*)).tw.

20. (emotional adj3 (disorder* or problem*)).tw.

21. (psychos#s or psychotic* or schizo* or depression or depressive or bipolar or mania or manic or obsessi* or panic or phobic or phobia or anorexi* or bulimi* or borderline or narcissis* or personality adj1 disorder or self injur* or self harm or dementia or substance abuse).tw.

22. or/14‐21

23. exp mass media/

24. (mass communication or media or broadcast* or radio or television or cinema or film* or movie* or trailer* or journalis*).tw.

25. serial publications/

26. (newspaper* or magazin* or newsletter* or press).tw.

27. journalism/

28. publishing/

29. communications media/

30. telecommunications/

31. electronic mail/

32. (electronic mail* or email* or e‐mail* or webmail* or mailing list* or discussion list* or listserv*).tw.

33. cellular phone/

34. (((mobile or cell* or wireless) adj (phone* or telephone*)) or text messag* or texting or texted or sms or mms).tw.

35. tape recording/

36. optical storage devices/

37. multimedia/

38. (audio* or video* or cassette* or tape* or dvd* or compact dis* or cd or cds or multimedia or multi media).tw.

39. internet/

40. (internet or web or website* or online or blog* or weblog* or podcast* or portal* or e‐communication* or electronic communication* or computer program* or computer mediated).tw.

41. video games/

42. video recording/

43.(apps or facebook or twitter or tweet or bebo or youtube or myspace or chatroom or chatroom or viral message or viral advert or wiki* or virtual*).tw.

44. software/

45. hypermedia/

46. user computer interface/

47. computer assisted instruction/

48. books/

49. pamphlets/

50. (pamphlet* or booklet* or leaflet* or flyer* or brochure* or print* media or print* material* or publication*).tw.

51. publications/

52. government publications as topic/

53. information dissemination/

54. (information adj2 (distribut* or disseminat*)).tw.

55. advertising as topic/

56. public relations/

57. persuasive communication/

58. famous persons/

59. ((famous adj (person* or people)) or celebrit*).tw.

60. social marketing/

61. (campaign* or message* or advert* or marketing or public relation* or publicity or public information or (communication adj (program* or strateg*)) or positive framing or (rais* adj2 awareness)).tw.

62. virtual or indirect or record* or film* or audio*) adj10 (social contact or testimony* or stor* or account* or experience* or narrative* or play or theat*)

63. Health promotion /

64. ((community or broadbased or broad based or public) adj3 education program*).tw.

65. (poster* or billboard* or  ribbon* or button* or badge* or visual art* or street art* or  (promotion* adj (item* or material*)) or festival* or entertainment).tw.

66. or/ 24‐66

67. 13 and 23 and 67

68. randomized controlled trial.pt.

69. controlled clinical trial.pt.

70. random*.tw.

71. placebo*.tw.

72. trial.tw.

73. groups.ab.

74. evaluation studies.pt.

75. evaluat*.tw.

76. follow up studies/

77. prospective studies/

78. (experiment* or intervention*).tw.

79. (pre test or pretest or post test or posttest).tw.

80. (preintervention or postintervention).tw.

81. time series.tw.

82. time point*.tw.

83. or/69‐83

84. exp animals/ not humans.sh.

85. 84 not 85

86. 68 and 86

3303 records found in MEDLINE

9530 records found in EMBASE

Appendix 3. PsycINFO search strategy

OvidSP, 1806 to 15 August 2011

1. stereotyping/

2. (stereotyp* or stigma* or label* or negative image* or ignoran* or misconception* or misperception* or literacy or ((public* or community or social or popular) adj perception*)).tw.

3. Stigma/

4. social perception/

5. public opinion/

6. prejudice/

7. exp attitude/

8. ((public* or community or social or popular) adj attitude*).tw.

9. (((negative or positive or chang*) adj3 attitude*) or prejudice* or hostil* or intoleran*).tw.

10. social distance/

11. rejection psychology/

12. human rights/

13. (discriminat* or marginali* or rejecting behavior or injustice* or (social adj (distance or justice or rejection or acceptance or exclusion or inclusion))).tw.

14. or/1‐13

15. mental health/

16. mental health services/

17. exp mental disorders/

18. mentally ill persons/

19. ((mental* or psychiatr* or psychological*) adj (ill* or disorder* or disease* or distress* or disab* or problem* or health* or well‐being or wellbeing or patient* or treatment or retardation)).tw.

20. ((chronic* or severe* or serious* or persistent) adj (mental* or psychiatr* or psychological*)).tw.

21. (emotional adj3 (disorder* or problem*)).tw.

22. (((psychos#s or psychotic* or schizo* or depression or depressive or bipolar or mania or manic or obsessi* or panic or phobic or phobia or anorexi* or bulimi* or borderline or narcissis* or personality) adj1 disorder) or self injur* or self harm or dementia or substance abuse).tw.

23. or/15‐22

24. exp mass media/

25. (mass communication or media or broadcast* or radio or television or cinema or film* or movie* or trailer* or journalis*).tw.

26. serial publications/

27. (newspaper* or magazin* or newsletter* or press).tw.

28. journalism/

29. publishing/

30. communications media/

31. telecommunications/

32. electronic mail/

33. (electronic mail* or email* or e‐mail* or webmail* or mailing list* or discussion list* or listserv*).tw.

34. cellular phone/

35. (((mobile or cell* or wireless) adj (phone* or telephone*)) or text messag* or texting or texted or sms or mms).tw.

36. tape recording/

37. optical storage devices/

38. multimedia/

39. (audio* or video* or cassette* or tape* or dvd* or compact dis* or cd or cds or multimedia or multi media).tw.

40. internet/

41. (internet or web or website* or online or blog* or weblog* or podcast* or portal* or e‐communication* or electronic communication* or computer program* or computer mediated).tw.

42. Video games/

43. video recording/

44. (apps or facebook or twitter or tweet or bebo or youtube or myspace or chatroom or chatroom or viral message or viral advert or wiki* or virtual*).tw.

45. software/

46. hypermedia/

47. user computer interface/

48. computer assisted instruction/

49. books/

50. pamphlets/

51. (pamphlet* or booklet* or leaflet* or flyer* or brochure* or print* media or print* material* or publication*).tw.

52. publications/

53. government publications as topic/

54. information dissemination/

55. (information adj2 (distribut* or disseminat*)).tw.

56. advertising as topic/

57. public relations/

58. persuasive communication/

59. famous persons/

60. ((famous adj (person* or people)) or celebrit*).tw.

61. social marketing/

62. (campaign* or message* or advert* or marketing or public relation* or publicity or public information or (communication adj (program* or strateg*)) or positive framing or (rais* adj2 awareness)).tw.

63. ((virtual or indirect or record* or film* or audio*) adj10 (social contact or testimon* or stor* or account* or experience* or narrative* or play or theat*)).tw.

64. Health promotion/

65. ((community or broadbased or broad based or public) adj3 education program*).tw.

66. (poster* or billboard* or ribbon* or button* or badge* or visual art* or street art* or (promotion* adj (item* or material*)) or festival* or entertainment).tw.

67. or/24‐66

68. 14 and 23 and 67

69. randomized controlled trial.pt.

70. controlled clinical trial.pt.

71. random*.tw.

72. placebo*.tw.

73. trial.tw.

74. groups.ab.

75. evaluation studies.pt.

76. evaluat*.tw.

77. follow up studies/

78. prospective studies/

79. (experiment* or intervention*).tw.

80. (pre test or pretest or post test or posttest).tw.

81. (preintervention or postintervention).tw.

82. time series.tw.

83. time point*.tw.

84. or/69‐83

85. exp animals/ not humans.sh.

86. 84 not 85

87. 68 and 86

1808 records found

Appendix 4. CINAHL search strategy

EBSCOhost, 1981 to 16 August 2011

S55. s38 AND s54

S54. s50 NOT s53

S53. s51 NOT S52

S52. (MH "Human")

S51. (MH "Animals")

S50. s39 or s40 or s41 or s42 or s43 or s44 or s45 or s46 or s47 or s48 or s49

S49. (MH "Multiple Time Series")

S48. "preintervention or postintervention"

S47. "experiment* or intervention*"

S46. (MH "Comparative Studies")

S45. (MH "Crossover Design")

S44. (MH "Prospective Studies") OR "follow up stud*"

S43. "evaluation stud*"

S42. "trial or groups"

S41. "random* or placebo*"

S40. (MH "Clinical Trials") OR "controlled clinical trial"

S39. (MH "Randomized Controlled Trials")

S38. s13 and s21 and s35

S37. "randomized controlled trial or controlled clinical trial or random* or palcebo* or trial or groups or evaluation studies or evaluat* or follow up studies or prospective studies or cross over studies or comparative study"

S36. s13 and s21 and s35

S35. s22 or s23 or s24 or s25 or s26 or s27 or s28 or s29 or s30 or s31 or s32 or s33 or s34

S34. TX poster* or billboard* or ribbon* or button* or badge* or visual art* or street art* or promotion* or festival* or entertainment

S33. TX community or broadbased or broad based or public N3 education program*

S32. (MH "Health Promotion") OR "health promotion" OR (MH "Mental Health Promotion (Saba CCC)")

S31. TX campaign* or message* or advert* or marketing or public relation* or publicity or public information or communication or program* or strateg* or public figure* or persuasive communication or social marketing

S30. (MH "Selective Dissemination of Information") OR (MH "Consumer Health Information")

S29. (MH "Government Publications") OR (MH "Public Opinion")

S28. TX pamphlet* or booklet* or leaflet* or flyer* or brochure* or print* media or print* material* or publication*

S27. TX Blog* or apps or facebook or twitter or tweet or bebo or youtube or myspace or chatroom or chatroom or viral message or viral advert or wiki* or virtual* or software or hypermedia or user‐computer interface or computer assisted instruction

S26. TX internet or web or website* or online or on line or blog* or weblog* or podcast* or portal? or e‐communication* or electronic communication* or computer program* or computer mediated

S25. TX audio* or video* or cassette* or tape* or dvd* or compact dis* or cd or cds or multimedia or multi media or computer storage devices or optical disks or audiorecording or videorecording

S24. TX mobile or cell* or wireless N2 phone* or telephone*

S23. TX electronic mail* or email* or e‐mail* or webmail* or mailing list* or discussion list* or listserv* or mobile or cell* or wireless phone* or wireless telephon* or text messag* or texting or texted or SMS or MMS

S22. TX mass communication or media or broadcast* or radio or television or cinema or film* or movie* or trailer* or journalis* or serial publications or newspaper* or magazin* or newsletter* or press

S21. S14 or S15 or S16 or S17 or S18 or S19 or S20

S20. TX psychos?s or psychotic* or schizo* or depression or depressive or bipolar or mania or manic or obsessi* or panic or phobic or phobia or anorexi* or bulimi* or borderline or narcissis* or (personality N1 disorder) or self injur* or self harm or dementia

S19. (MH "Affective Disorders") OR (MH "Affective Disorders, Psychotic") OR "emotional disorder"

S18. "psychiatric illness"

S17. (MH "Mental Retardation") OR "mental retardation" OR (MH "Mentally Disabled Persons")

S16. (MH "Attitude to Mental Illness") OR (MH "Mentally Disabled Persons") OR (MH "Psychosocial Aspects of Illness") OR "mentally ill persons"

S15. (MH "Mental Disorders") OR "mental disorders" OR (MH "Behavioral and Mental Disorders (Non‐Cinahl)")

S14. (MH "Mental Health") OR "mental health" OR (MH "Mental Health Services")

S13. S1 or S2 or S3 or S4 or S5 or S6 or S7 or S8 or S9 or S10 or S11 or S12

S12. "(rights or discriminat* or marginali* or rejecting behavior or injustice* or social) N5 (distance or justice or rejection or acceptance or exclusion or inclusion)"

S11. (MH "Human Rights") OR "human rights"

S10. "rejection psychology"

S9. "social distance"

S8. "(negative or positive or chang*) N5 (attitude* or prejudice*or hostil* or intoleran*)"

S7. "(public* or community or social or popular) N5 (attitude*)"

S6. (MH "Attitude") OR "attitude"

S5. (MH "Prejudice") OR "Prejudice"

S4. (MH "Public Opinion") OR "public opinion"

S3. "social perception"

S2. (stereotyp* or stigma* or label* or negative image* or ignoran* or misconception* or misperception* or literacy or public* or community or social or popular) N5 (perception*)

S1. (MH "Stereotyping")

401 records found

Appendix 5. ERIC search strategy

CSA, 1966 to 16 August 2011

1. stereotyping/

2. stereotyp* or stigma* or label* or negative image* or ignoran* or misconception* or misperception* or literacy or public* perception  or community perception or social perception or popular perception

3. social perception or public opinion or prejudice or attitude or hostil* or intoleran* or social distance or rejection psychology or human rights

4. discriminat* or marginali* or rejecting behavio?r or injustice* or social distance or social justice or social rejection or social acceptance or social exclusion or social inclusion

5. or/1‐4

6. mental health or mental health services or mental disorders or mentally ill persons or mental* or psychiatr* or psychological* or well‐being or wellbeing or patient* or treatment or retardation

7. psychos* or psychotic* or schizo* or depression or depressive or bipolar or mania or manic or obsessi* or panic or phobic or phobia or anorexi* or bulimi* or borderline or narcissis* or personality or self injur* or self harm or dementia or substance abuse

8. or/6‐7

9. mass media or communication or mass communication or media or broadcast* or radio or television or cinema or film* or movie* or trailer* or journalis* or serial publications or newspaper* or magazin* or newsletter* or press or journalism or publishing or communications media or telecommunications or electronic mail or electronic mail* or email* or e‐mail* or webmail* or mailing list* or discussion list* or listserv*

10. cellular phone or mobile or cell* or wireless or phone* or telephone*or text messag* or texting or texted or sms or mms or tape recording or optical storage devices or multimedia or audio* or video* or cassette* or tape* or dvd* or compact dis* or cd or cds or multimedia or multi media

11. internet or web or website* or online or blog* or weblog* or podcast* or portal* or computer program* or computer mediated or video recording or apps or facebook or twitter or tweet or bebo or youtube or myspace or chatroom or chatroom or viral message or viral advert or wiki* or virtual* or software or hypermedia or user computer interface or computer assisted instruction

12. pamphlet* or booklet* or leaflet* or flyer* or brochure* or print* or material* or publication* or information dissemination or advertising or public relations or famous persons or celebrit* or social marketing or campaign* or message* or advert* or marketing or public relation* or publicity or public information or positive framing or community or broadbased or broad based or public education or program or poster* or billboard* or ribbon* or button* or badge* or visual art* or street art* or promotion* or festival* or entertainment

13. or/9‐12

14. 5 and 8 and 13

15. randomized controlled trial or controlled clinical trial or random or placebo* or trial or groups or evaluation studies or evaluat* or follow up studie or prospective studies or cross over studies or comparative study or time series or time point*

16. animals NOT humans

17. 15 NOT 16

18. 14 and 17

1782 records found

Appendix 6. Social Science Citation Index search strategy

ISI, 1956 to 16 August 2011

1. stereotyping

2. (stereotyp* or stigma* or label* or negative image* or ignoran* or misconception* or misperception* or literacy or ((public* or community or social or popular) near/1 perception*))

3. “social perception”

4. “public opinion”

5. prejudice

6. attitude

7. ((public* or community or social or popular) near/1 attitude*)

8. (((negative or positive or chang*) near/3 attitude*) or prejudice* or hostil* or intoleran*)

9. “social distance”

10. “rejection psychology”

11. “human rights”

12. discriminat* or marginali* or “rejecting behaviour” or injustice* or (social near/1 (distance or justice or rejection or acceptance or exclusion or inclusion)))

13. or/1‐12

14. “mental health”

15. “mental health services”

16. “mental disorders”

17. “mentally ill persons”

18. ((mental* or psychiatr* or psychological*) near/1 (ill* or disorder* or disease* or distress* or disab* or problem* or health* or well‐being or wellbeing or patient* or treatment or retardation))

19. ((chronic* or severe* or serious* or persistent) near/1 (mental* or psychiatr* or psychological*))

20. (emotional near/3 (disorder* or problem*))

21. (((psychos?s or psychotic* or schizo* or depression or depressive or bipolar or mania or manic or obsessi* or panic or phobic or phobia or anorexi* or bulimi* or borderline or narcissis* or personality) near/1 disorder) or self injur* or “self harm” or dementia or “substance abuse”)

22. or/14‐21

23. “mass media”

24. (“mass communication” or media or broadcast* or radio or television or cinema or film* or movie* or trailer* or journalis*)

25. “serial publications”

26. (newspaper* or magazin* or newsletter* or press)

27. journalism

28. publishing

29. “communications media”

30. telecommunications

31. “electronic mail”

32. (electronic mail* or email* or e‐mail* or webmail* or mailing list* or “discussion list*” or listserv*)

33. “cellular phone”

34. (((mobile or cell* or wireless) near/1 (phone* or telephone*)) or “text messag*” or texting or texted or sms or mms)

35. “tape recording”

36. “optical storage devices”

37. multimedia

38. (audio* or video* or cassette* or tape* or dvd* or compact dis* or cd or cds or multimedia or “multi media”)

39. internet

40. (internet or web or website* or online or blog* or weblog* or podcast* or portal* or e‐communication* or “electronic communication*” or “computer program*” or “computer mediated”)

41. “video recording”

42. (apps or facebook or twitter or tweet or bebo or youtube or myspace or chatroom or chatroom or “viral message” or “viral advert” or wiki* or virtual*)

43. software

44. hypermedia

45. “user computer interface”

46. “computer assisted instruction”

47. books

48. pamphlets

49. (pamphlet* or booklet* or leaflet* or flyer* or brochure* or print* media or print* material* or publication*)

50. publications

51. “government publications as topic”

52. “information dissemination”

53. (information near/2 (distribut* or disseminat*))

54. “advertising as topic”

55. “public relations”

56. “persuasive communication”

57. “famous persons”

58. ((famous near/1 (person* or people)) or celebrit*)

59. “social marketing”

60. (campaign* or message* or advert* or marketing or “public relation*” or publicity or public information or (communication adj (program* or strateg*)) or “positive framing” or (rais* near/2 awareness))

61. ((community or broadbased or “broad based” or public) near/3 “education program*”)

62. (poster* or billboard* or ribbon* or button* or badge* or “visual art*” or “street art*” or (promotion* near/1 (item* or material*)) or festival* or entertainment)

63. or/23‐62

64. 13 and 22 and 63

65. “randomized controlled trial”

66. “controlled clinical trial”

67. “random*”

68. “placebo*”

69. “trial”

70. “groups”

71. “evaluation studies”

72. “evaluat*”

73. “follow up studies”

74. “prospective studies”

75. “cross over studies”

76. “comparative study”

77. (experiment* or intervention*)

78. (“pre test” or pretest or “post test” or posttest)

79. (preintervention or postintervention)

80. “time series”

81. “time point*”

82. or/65‐81

83. animals/ not humans

84. 82 not 83

85. 64 and 84

3663 records found

Appendix 7. OpenSIGLE search strategy

http://www.opengrey.eu/) 1980 to 2005, searched 18 August 2011, 2005 date of final entry to database

(stigma OR discrimination OR stereotype* OR social perception OR public opinion OR attitude)

AND

(mental health OR mental health disorders OR mentally ill persons OR mental* or psychiatr* OR psycholog* OR eating disorder OR psycho* OR bipolar OR substance abuse OR anxiety) AND (mass media OR media OR communication OR radio OR television OR cinema OR film OR newspap* OR advertising)

46 records found

Appendix 8. Worldcat search strategy

OCLC, 1978 to 18th Augsut 2011

stigma OR discrimination OR prejudi* OR social perception OR social distance OR human rights

AND

(mental* OR  psych* OR depress* OR schizo* OR bipolar OR anxiety OR substance OR alcohol OR dementia OR intellectual disabil* OR learning disabil* OR retardation OR anorex* or bulimi* OR obsessi* OR phobi* OR panic) AND

(media OR communication OR television OR radio OR film OR cinema OR movie OR newspap* OR internet OR video OR DVD OR publication OR advert* OR social market* OR campaign* OR messag* OR narrative OR social contact OR audio* OR virtual OR health promotion OR online)

AND

random* OR trial OR time series OR time point

80 records found

Appendix 9. metaRegister of Controlled Trials search strategy

http://www.controlled‐trials.com/mrct/mrct_about.asp, 1973 to 18 August 2011

stigma

AND

mental health

AND

mass media

0 records found

Data and analyses

Comparison 1. Mass media vs. control: main comparison.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Discrimination (Immediate)	3		Std. Mean Difference (IV, Random, 95% CI)	Totals not selected
2 Discrimination	2		Odds Ratio (Random, 95% CI)	Totals not selected
2.1 Immediate	1		Odds Ratio (Random, 95% CI)	0.0 [0.0, 0.0]
2.2 6 ‐ 9 months	1		Odds Ratio (Random, 95% CI)	0.0 [0.0, 0.0]
3 Prejudice	15		Std. Mean Difference (IV, Random, 95% CI)	Totals not selected
3.1 Immediate	12		Std. Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]
3.2 1 week ‐ 2 months	4		Std. Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]
3.3 6 ‐ 9 months	2		Std. Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]
4 Prejudice	3		Std. Mean Difference (Random, 95% CI)	Totals not selected
4.1 Immediate	2		Std. Mean Difference (Random, 95% CI)	0.0 [0.0, 0.0]
4.2 6 ‐ 9 months	1		Std. Mean Difference (Random, 95% CI)	0.0 [0.0, 0.0]

1.1. Analysis.

Comparison 1 Mass media vs. control: main comparison, Outcome 1 Discrimination (Immediate).

1.2. Analysis.

Comparison 1 Mass media vs. control: main comparison, Outcome 2 Discrimination.

1.3. Analysis.

Comparison 1 Mass media vs. control: main comparison, Outcome 3 Prejudice.

1.4. Analysis.

Comparison 1 Mass media vs. control: main comparison, Outcome 4 Prejudice.

Comparison 2. Mass media vs. control by income of country.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Prejudice (at earliest follow‐up time point)	15		Std. Mean Difference (IV, Random, 95% CI)	Totals not selected
1.1 Upper‐middle income	1		Std. Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]
1.2 High income	14		Std. Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]

2.1. Analysis.

Comparison 2 Mass media vs. control by income of country, Outcome 1 Prejudice (at earliest follow‐up time point).

Comparison 3. Mass media vs. control by number of mass media components.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Prejudice (at earliest follow‐up time point)	15		Std. Mean Difference (IV, Random, 95% CI)	Totals not selected
1.1 Multiple mass media components	4		Std. Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]
1.2 One mass media component	11		Std. Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]

3.1. Analysis.

Comparison 3 Mass media vs. control by number of mass media components, Outcome 1 Prejudice (at earliest follow‐up time point).

Comparison 4. Mass media vs. control by whether combined with non‐mass media.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Prejudice (at earliest follow‐up time point)	15		Std. Mean Difference (IV, Random, 95% CI)	Totals not selected
1.1 Mass media combined with non‐mass media	1		Std. Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]
1.2 Mass media alone	14		Std. Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]

4.1. Analysis.

Comparison 4 Mass media vs. control by whether combined with non‐mass media, Outcome 1 Prejudice (at earliest follow‐up time point).

Comparison 5. Mass media vs. control by presence of narratives.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Prejudice (at earliest follow‐up time point)	15		Std. Mean Difference (IV, Random, 95% CI)	Totals not selected
1.1 First‐person narratives	7		Std. Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]
1.2 Third‐person narratives	4		Std. Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]
1.3 No narratives / none stated	7		Std. Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]

5.1. Analysis.

Comparison 5 Mass media vs. control by presence of narratives, Outcome 1 Prejudice (at earliest follow‐up time point).

Comparison 6. Mass media vs. control by celebrity narratives.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Prejudice (at earliest follow‐up time point)	11		Std. Mean Difference (IV, Random, 95% CI)	Totals not selected
1.1 Celebrity	1		Std. Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]
1.2 Non‐celebrity	11		Std. Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]

6.1. Analysis.

Comparison 6 Mass media vs. control by celebrity narratives, Outcome 1 Prejudice (at earliest follow‐up time point).

Comparison 7. Mass media vs. control by fictional narratives.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Prejudice	11		Std. Mean Difference (IV, Random, 95% CI)	Totals not selected
1.1 Fictional narrative	2		Std. Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]
1.2 Non‐fictional narrative	9		Std. Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]

7.1. Analysis.

Comparison 7 Mass media vs. control by fictional narratives, Outcome 1 Prejudice.

Comparison 8. Mass media vs. control by type of message.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Prejudice (at earliest follow‐up time point)	15		Std. Mean Difference (IV, Fixed, 95% CI)	Totals not selected
1.1 Biomedical	1		Std. Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]
1.2 Social inclusion / human rights	1		Std. Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]
1.3 Recovery‐oriented	4		Std. Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]
1.4 Negative impact of mental illness	2		Std. Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]
1.5 Seek professional care	3		Std. Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]
1.6 Not to blame	3		Std. Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]
1.7 This is hallucinatory experience	2		Std. Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]
1.8 Multiple primary messages	2		Std. Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]

8.1. Analysis.

Comparison 8 Mass media vs. control by type of message, Outcome 1 Prejudice (at earliest follow‐up time point).

Comparison 9. Mass media vs. control by type of media.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Prejudice (at earliest follow‐up time point))	15		Std. Mean Difference (IV, Random, 95% CI)	Totals not selected
1.1 Internet (also includes CD‐ROM)	4		Std. Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]
1.2 Recorded ‐ audiovisual	7		Std. Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]
1.3 Recorded ‐ audio	2		Std. Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]
1.4 Print	5		Std. Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]

9.1. Analysis.

Comparison 9 Mass media vs. control by type of media, Outcome 1 Prejudice (at earliest follow‐up time point)).

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Brown 2010.

Methods	Aim of study: To test the hypothesis that individuals who view a film presenting people with schizophrenia will report decreased levels of stigma and those individuals who listen to a simulation of auditory hallucinations will report increased levels of stigma immediately following the intervention and one week later Study design: RCT Recruitment: From university introductory psychology courses Inclusions/exclusion criteria: Excluded self‐reported hearing or uncorrectable vision impairments (a) Informed consent: Yes Ethical approval: Yes Funding: None (a) Consumer involvement: Indirect involvement in development of intervention B (people who experience auditory hallucinations were involved in developing the simulated hallucination intervention from which the segment used in the present study was taken)
Participants	Description: Undergraduates at a state university recruited from introductory psychology courses Geographic location: Iowa, USA (assumed from author affiliation) Income level of country: High Setting: University  Number: Eligible 143 (a), total randomised 143, (intervention A 40, intervention B 58, control 45), lost to follow‐up T1 0, T2 7; included in analyses 143 (prejudice 143) Age: range 18 to 24, mean (SD) 19.3 (1.2) Gender: 58.0% female Ethnicity: 92.9% Caucasian Other social/demographic details: Main majors: business (20.3%), education (14.7%) and biology (11.9%). 5.6% were majoring in psychology
Interventions	Aim of intervention: Intervention A: to provide personal portrayals of schizophrenia and potentially dispel negative attitudes or misperceptions towards those with schizophrenia. Intervention B: to simulate a credible experience of auditory hallucinations Content of intervention: Intervention A: film segment showing three individuals psychiatrically stable and diagnosed with schizophrenia discussing their symptoms, difficulties, treatment and accomplishments. Intervention B: audio segment of voice and non‐voice sounds simulating both derogatory and neutral/benevolent auditory hallucinations Content of control: No intervention Co‐interventions in all groups: None Delivery: Intervention A: one film, duration 16 minutes. Intervention B: one simulation, duration 16 minutes, delivered at conversational level (60 decibels) Providers: Intervention A: film segment taken from film 'Living with Schizophrenia' produced by Treatment Resources for Understanding Schizophrenia Therapy, 2001.  Intervention B simulation extraction from 'Hearing Voices that are Distressing: A Training and Simulation Experience' (Deagan, The National Empowerment Centre Inc, 1996) Type of mass media: Intervention A: audiovisual recording (film). Intervention B: audio recording (simulated hallucinations) Number of mass media components: Intervention A: one. Intervention B: one Combined with non‐mass media component: No Contains personal narrative: Intervention A: yes (first‐person). Intervention B: no Celebrities involved: No Fictional portrayals: No Primary message: Intervention A: recovery‐oriented. Intervention B: 'this is hallucinatory experience' Mental health condition: Schizophrenia
Outcomes	Primary outcome measures: Discrimination: none. Prejudice: Social Distance Scale (SDS); Affect Scale (AS) Secondary outcome measures: None Measure(s) selected for analysis if multiple measures per outcome: SDS (following Brennan 2009) Methods of assessing outcome measures: Self‐complete questionnaire Validity and reliability of primary outcome measures: Study paper stated 'These measures have documentation of strong reliability and validity' with four references provided. Cronbach’s alpha for the two scales using data from this study were reported as 0.75 for SDS and 0.81 for AS Methods of follow‐up for non‐respondents: None described Timing of outcome assessment: Immediate and one week
Notes	Contact with author: Yes Power calculation: No (a)
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Quote: 'participants were randomly assigned' Quote 'I used a computer generated table of random number (ranging from 1 to 3) to represent the 3 intervention conditions. No restrictions implemented.' (a)
Allocation concealment (selection bias)	High risk	Quote: 'participants were randomly assigned' Quote 'No' (a) (Author response to question about concealment)
Incomplete outcome data (attrition bias) All outcomes	Low risk	Comment: Response rate at 1 week is very high (95.1%), however, it is unclear how this varied by intervention group and what the reasons were
Selective reporting (reporting bias)	Unclear risk	Comment:  No protocol mentioned.  Study reports that two outcomes were assessed and both are reported
Other bias	Low risk	Comment: Low risk for lack of evidence for reliability or validity of primary outcomes N/A for difference in baseline measures or participant characteristics in cluster trials with few clusters Low risk for evidence of counter‐discourse in follow‐up period N/A for outcomes between audience members and non‐audience not compared Low risk for items listed as potential other sources of bias in Higgins 2011 (section 8.15.1)
Blinding of participants and personnel (performance bias) All outcomes	High risk	Comment: Blinding of participants and intervention providers not possible
Blinding of outcome assessment (detection bias) All outcomes	High risk	Comment: Outcome assessment was by self‐complete questionnaire

Bunn 2009.

Methods	Aim of study: To test the hypothesis that after experiencing what it is like to have auditory hallucinations medical students will have increased empathy for the challenges of the psychiatric patient Study design: RCT Recruitment: Invited to participate on first day of psychiatry rotation Inclusion/exclusion criteria: None stated Informed consent obtained: Yes Ethical approval: Yes Funding: None stated Consumer involvement: Indirect involvement in development of intervention (people who experience auditory hallucinations were involved in developing the simulated hallucination intervention from which the segment used in the present study was taken)
Participants	Description: Medical students on their psychiatry rotation Geographic location: Utah, USA Income level of country: High Setting: University Number: Eligible 150, total randomised 150 (intervention 100, control 50), included in analysis 150 (prejudice 150) Age: Not stated Gender:  84 (56%) male, 66 (44%) female Ethnicity: 90% Caucasian, 4% African American, 3% Asian, 3% other ethnic background Other social/demographic details: 65% had taken a psychology class prior to medical school; 80% had a friend or family member with a psychiatric disorder
Interventions	Aim of intervention: To increase empathy and understanding about mental illness Content of intervention: Simulation of auditory hallucinations, with concurrent tasks (constructing geometric designs with toothpicks, participating in a modified Mini Mental State examination, interacting with peers) Content of control: No intervention Co‐interventions in all groups: None Delivery: 40 minutes simulation of auditory hallucinations delivered via headphones Providers: Created by Patricia Deegan of the National Empowerment Center Inc. as part of the ‘Hearing Voices that are Distressing’ curriculum Type of mass media: Audio recording (simulated hallucinations) Number of mass media components: One Combined with non‐mass media component: No Contains personal narrative:  No Celebrities involved: No Fictional portrayals: No Primary message:  'This is hallucinatory experience' Mental health condition: Psychosis
Outcomes	Primary outcome measures: Discrimination: none. Prejudice: Jefferson Scale of Physician Empathy – student version (JSPE‐S) Secondary outcome measures: None Methods of assessing outcome measures: Self‐complete questionnaire Validity and reliability of primary outcome measures: Study paper stated ‘The JSPE‐S is a validated instrument’ with one reference given to support this, no psychometric data provided Methods of follow‐up for non‐respondents: N/A Timing of outcome assessment: Immediate
Notes	Contact with author: Yes Power calculation: None stated
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Quote: ‘Fifty students [of a cohort of 150] were randomly selected for the comparison group and completed the same procedures as the test participants … but without listening to the simulated auditory hallucinations’
Allocation concealment (selection bias)	Unclear risk	Quote: ‘Fifty students [of a cohort of 150] were randomly selected for the comparison group and completed the same procedures as the test participants … but without listening to the simulated auditory hallucinations’
Incomplete outcome data (attrition bias) All outcomes	Low risk	Comment: The authors state that 150 were invited to participate and report data on 150
Selective reporting (reporting bias)	Unclear risk	Comment: No protocol mentioned. One outcome was described in the measures section and that one was reported in the results section
Other bias	Unclear risk	Comment: Unclear risk for lack of reliability and validity of outcome measures. Study paper stated ‘The JSPE‐S is a validated instrument’ with one reference given to support this, no psychometric data provided. One reference given to support validation, no psychometric data provided N/A for difference in baseline measures or participant characteristics in cluster trials with few clusters N/A for evidence of counter‐discourse in follow‐up period N/A for outcomes between audience members and non‐audience not compared Low risk for items listed as potential other sources of bias in Higgins 2011 (section 8.15.1)
Blinding of participants and personnel (performance bias) All outcomes	High risk	Comment: Blinding of participants and intervention providers was not possible
Blinding of outcome assessment (detection bias) All outcomes	High risk	Comment: Outcome assessment was by self‐complete questionnaire

Coleman 2005.

Methods	Aim of study: To examine the effects of age and exposure to educational material on schizophrenia on stigma in student nurses Study design: Cluster RCT Size and description of clusters: Participants were randomly assigned, by class section, to either treatment or control. Size of clusters not specified Recruitment: From college course Inclusion/exclusion criteria: None stated Informed consent: Yes Ethical approval: Not stated, although 'they were treated in accordance with the ethical principles of the American Psychological Association' Funding: None stated Consumer involvement: None stated
Participants	Description: Second year nursing students in a Community College School of Nursing Geographic location: Suffolk County, New York, USA Income level of country: High Setting: Community College Number: Total randomised 240 (intervention 119, control 121), included in analysis 240 (prejudice 240) Age: mean 35.4, (SD for whole group not stated, 9.2 and 8.7 for intervention and control groups) Gender: 15.43 male Ethnicity: not stated Other social/demographic details: For intervention and control groups respectively (no overall data available): Mean (SD) years of education 15 (1.4), 15.3 (1.4); previous work or experience with the mentally ill 3.36%, 0%; planned future work with the mentally ill 48.73%, 41.32%
Interventions	Aim of intervention:  To educate about schizophrenia Content of intervention: Video entitled ‘Fires of the mind, dark voices: Schizophrenia’‐ an educational documentary. Covers the causes, treatments and outcomes for persons with schizophrenia. Contains interviews with individuals diagnosed with schizophrenia and their families, provides first‐hand accounts of the effects of the disorder on their family dynamics, personal relationships, and overall level of functioning. Provides education on the biological causes and treatments of schizophrenia, social consequences of the symptomatology, and various treatment options including psychopharmacology, social skills training, and outpatient cognitive‐behaviour therapy, giving a comprehensive look at the causes, treatments and outcomes for individuals with chronic schizophrenia. Aimed to provide information in an objective, straightforward manner and by design avoids emotional content and tone Content of control:  Educational video entitled ‘What is diabetes' from the American Association of Diabetes Educators patient education video series, covering the cause, treatments and maintenance of health, diet, exercise and effective ways to cope with the effects diabetes can have on an individual’s health. It did not discuss any psychological components of diabetes nor did it offer any advice on coping or treatment of any psychological effect of diabetes Details of co‐interventions in all groups: None Delivery of intervention: Duration of intervention video one hour, not stated for control video Providers: Intervention: Produced by the US television Discovery Channel.  Control produced by American Association of Diabetes Educators Type(s) of mass media: Audiovisual (video, television documentary) Number of mass media components: One Combined with non‐mass media component: No Contains personal narrative:  Yes Celebrities involved:  No Fictional portrayals: No Primary message: Biomedical Mental health condition: Schizophrenia
Outcomes	Primary outcome measures: Discrimination: none. Prejudice: Custodial Mental Illness Ideology Scale (CMI); three subscales of the Opinions about Mental Illness Scale (OMI): Authoritarianism (OMI‐A), Benevolence (OMI‐B) and Social Restrictiveness (OMI‐SR). Two of the OMI subscales Mental Hygiene Etiology (OMI‐C) and Interpersonal Etiology (OMI‐E) focus on aetiology beliefs rather than stigma and so were not considered stigma outcomes Secondary outcome measures: None Measure(s) selected for analysis if multiple measures per outcome: CMI (as authors based hypothesis on this) Methods of assessing outcome measures: Self‐complete questionnaire Validity and reliability of primary outcome measures: CMI: Spearman‐Brown corrected reliability estimate of 0.85 and test re‐test reliability was in the mid 0.80s. Two of the OMI subscales have low internal consistency (C and E) but neither were considered to be stigma outcomes (see above). The other OMI scales had internal consistency as follows: Authoritarianism 0.77 to 0.80, Benevolence 0.70 to 0.72; Social Restrictiveness 0.71 to 0.76 Methods of follow‐up for non‐respondents:  N/A Timing of outcome assessment: Immediate
Notes	Contact with author: Attempted, unsuccessful Power calculation: None stated Unpublished: Dissertation
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Quote: ‘Participants were randomly assigned, by class section, to either treatment or control’
Allocation concealment (selection bias)	Unclear risk	Quote: ‘Participants were randomly assigned, by class section, to either treatment or control’
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Comment: Follow‐up was immediate and no statement was given about any missing data
Selective reporting (reporting bias)	Unclear risk	Comment:  No protocol mentioned.  Study reports that one scale and five subscales were assessed and these are reported.
Other bias	High risk	Comment: Unclear risk for lack of evidence for reliability or validity of primary outcomes. CMI: Spearman‐Brown corrected reliability estimate of 0.85 and test re‐test reliability was in the mid 0.80s. Two of the OMI subscales have low internal consistency (C and E) but neither were considered to be stigma outcomes (see above). The other OMI scales had internal consistency as follows: Authoritarianism 0.77 to 0.80, Benevolence 0.70 to 0.72; Social Restrictiveness 0.71 to 0.76, however these data were from old citations and internal consistency for the current study population was not reported. High risk due to it being a cluster trial with the number of clusters being unstated and with some baseline differences apparent e.g. 17.64 vs 13.22% male and 48.73 vs 41.32 planned future work with mentally ill. Also 3.36% vs 0% of treatment and control groups respectively had previous work experience and experience generally with people with mental illness and in discussion it is stated that this is significant. N/A for evidence of counter‐discourse in follow‐up period N/A for outcomes between audience members and non‐audience not compared Unclear risk as the analysis did not take clustering into account, or did not state that this was done or how Low risk for other items listed as potential other sources of bias in Higgins 2011 (section 8.15.1)
Blinding of participants and personnel (performance bias) All outcomes	High risk	Comment: Blinding of participants and personnel not possible
Blinding of outcome assessment (detection bias) All outcomes	High risk	Comment: Outcome assessment was by self‐complete questionnaire

Corrigan (submitted).

Methods	Aim of study: To investigate whether readers who complete a news story about serious mental illness and recovery will show significant decreases in their stigmatising attitudes as well as enhancement of affirming beliefs, and whether a story highlighting dysfunctional and failing mental health and related public systems will have unintended negative consequences on stigma eradication Study design: Three‐arm RCT Recruitment: From volunteers section of a community website Inclusion/exclusion criteria: None (a) Informed consent: Yes Ethical approval: Yes (a) Funding: Center on Adherence and Self‐determination Consumer involvement: No (a)
Participants	Description: General public Geographic location: Chicago, Illinois, USA Income level of country: High Setting: Online Number: Total randomised 150 (intervention A 50, intervention B 50, control 50) (a) included in analysis 150 (prejudice 150) Age: mean (SD) 32.7 (7.53) Gender: 66.7% male Ethnicity: Not stated Other social/demographic details: Married/in long‐tem relationship 71.4%, single 23.8%, divorced or separated 4.8%; bachelor’s degree or equivalent 65.2%, associate’s degree or some college 8.1%, graduate education 6.2%, high school diploma or less 1.9%
Interventions	Aim of intervention:  Not stated Content of intervention Intervention A: Story on recovery, ‘Beautiful minds can be recovered’, which combined the story of Nobel Laureate John Nash with the empirical evidence from long‐term follow‐up research that people with serious mental illness recover. Intervention B: story on dysfunctional systems, ‘Sometimes the State’s dead must teach’, which told the story of Billy Owens, a convict with mental illness who stabbed himself to death in a prison cell surrounded by guards untrained to address his symptoms Content of control:  Dental care story, ‘Maintaining good dental care habits’, which made no mention of mental health issues Co‐interventions in all groups: None Delivery: Stories were read online, Intervention A 926 words, intervention B 645 words, control 519 words (calculated from stories provided by author) Providers: Stories were selected from a set of 20 completed by experienced reporters and published in major newspapers, published in New York Times (recovery story), the Oregonion 2003 (dysfunctional systems story) and EverydayHealth.com (dental health care story) Type of mass media:  Internet (text delivered through a research website) Number of mass media components: One Combined with non‐mass media component: No Contains personal narrative:  Yes (third‐person) Celebrities involved:  Intervention A: yes (John Nash). Intervention B: no Fictional portrayals: No Primary message: Intervention A: recovery‐oriented. Intervention B: negative impact of mental illness Mental health condition:  Serious mental illness/psychosis
Outcomes	Primary outcome measures: Discrimination: none. prejudice: Attribution Questionnaire, short form, three subscales: Dangerousness (AQ‐9(d)); Blame (AQ‐9(b)); and Coercion (AQ‐9(c)); Stigma Through Knowledge test (STKT); Recovery Affirmation Scale (RAS); Empowerment Affirmation Scale (EAS); Empowerment Scale (adapted for use with general public); Self‐Determination Affirmation Scale (SDAS) Secondary outcomes: none Measure selected for analysis if multiples measures per outcome: RAS (following Grimshaw 2003) Methods of assessing outcome measures: Online self‐complete questionnaires Validity and reliability of primary outcome measures: AQ‐9 has strong reliability, construct validity and sensitivity to change after anti‐stigma programme participation, strong reliability, construct validity and sensitivity to change (one reference cited). Three scores were gleaned from single items (no psychometric information was provided for these); STKT no psychometrics given but one reference to previous use; RAS, modified for this study to be a measure of how the public views the recovery of people with serious mental illness, Cronbach's alpha = 0.77; EAS modified for this study, alpha = 0.86; SDAS developed for this study, alpha = 0.80 Methods of follow‐up for non‐respondents: N/A Timing of outcome assessment: Immediate
Notes	Contact with author: Yes Power calculation: No (a) Unpublished: Paper currently submitted (Journal of Nervous and Mental Disease, provided by author), not yet peer reviewed
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Quote: ‘Research participants were randomly assigned to one of three conditions’ Quote ‘We used Qualtrics, an alternative online survey to SurveyMonkey. Qualtrics has an option for facilitating an RCT’ (a)
Allocation concealment (selection bias)	Low risk	Quote: 'Web based allocation'
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Comment: There is no mention of incomplete data although no Ns are given in the results table
Selective reporting (reporting bias)	Unclear risk	Quote: ‘Three scores were gleaned from single items of the AQ‐9 for this study’ Comment:  It is unclear from this quotation whether this may indicate possible selective reporting. No protocol available to review authors (a)
Other bias	Low risk	Comment: Low risk for lack of evidence for reliability or validity of primary outcomes N/A for difference in baseline measures or participant characteristics in cluster trials with few clusters N/A for evidence of counter‐discourse in follow‐up period N/A for outcomes between audience members and non‐audience not compared Low risk for items listed as potential other sources of bias in Higgins 2011 (section 8.15.1)
Blinding of participants and personnel (performance bias) All outcomes	High risk	Comment: Blinding of participants and intervention providers not possible
Blinding of outcome assessment (detection bias) All outcomes	High risk	Comment: Outcome assessment was by self‐complete questionnaire

Demyan 2009.

Methods	Aim of study: To examine the effects of a public service announcement style video intervention on attitudes towards therapy and intentions to seek therapy Study design: RCT Recruitment: Recruited from psychology courses Inclusion/exclusion criteria: None stated Informed consent: Yes Ethical approval: Yes (a) Funding: No (a) Consumer involvement: No (a)
Participants	Description: University students studying psychology Geographic location: Ohio, USA (a)  Income level of country: High Setting: University Number: Total randomised 221 (intervention 100, control 121 (a)); lost to follow up 1 (assumed from author data), included in analysis 220 (a) (prejudice 220) Age:  range 18 to 55 , mean 20.24  (SD not reported) Gender: 57.5% female, 42.5% male Ethnicity: Caucasian 83%, African American 8.8%, Hispanic 1.3%, Asian 1.3%, Native American 1.3%, other 3.5% Other social/demographic details: 36% freshmen 21% sophomores, 23% juniors, 20% seniors; 66.7% had never sought psychological assistance, 31.6% had, 1.8% did not respond
Interventions	Aim of intervention:  To improve attitudes to seeking professional psychological help and intentions to seek help Content of intervention: Pro‐psychotherapy public service announcement (PSA) style video. Video featuring a female student feeling overwhelmed by college work. Voiceovers say meeting a therapist can help with life’s hurdles, people who see a therapist have better outcomes than those who try to fix things on their own, therapy is confidential, therapy is common and effective, you’d be surprised who is benefiting from it right now. Student is shown making appointment, looking happier, shaking hands with therapist after session. Content of control:  Four 10 minute music videos with other PSAs between them (content unspecified, assumed not about stigma or mental health problems). Details of co‐interventions in all groups: Four ten‐minute music videos and three brief tasks in which students gave their opinion on music in advertising between the showing of the videos during the intervention and control session and two music opinion‐giving brief tasks at follow‐up Delivery of intervention: Two minute video shown with four 10 minute music videos and other PSAs (unspecified), assume intervention PSA was presented once Details of providers: Study author described the specific aims of the study to students in an advanced telecommunications script‐writing course.  They developed scripts and the one that best addressed the aims was selected by the course tutor Type of mass media: Audiovisual (video, pubic service announcement) Number of mass media components: One Combined with non‐mass media component: No Contains personal narrative: Yes (third‐person) Celebrities involved: No Fictional portrayals: Yes Primary message: Seek professional care Mental health condition: ‘Life’s hurdles’ (encompassing mild problems and larger, more debilitating psychopathology) and stress
Outcomes	Primary outcome measures: Discrimination: none. Prejudice: Stigma Scale for Receiving Psychological Help (SSRPH) Secondary outcomes: Cost of intervention provided by author Methods of assessing outcome measures Self‐complete questionnaire Validity and reliability of primary outcome measures SSRPH Cronbach's alpha = 0.72 (original study), and 0.65 in present study. Has adequate inter‐item reliability, good construct validity and negatively correlates with the Attitudes Towards Seeking Professional Psychological Help Methods of follow‐up for non‐respondents: None stated Timing of outcome assessment: One week
Notes	Contact with author: Yes Power calculation: No (a) Unpublished: Dissertation, paper in press with Journal of Counseling Psychology (a), data extracted from dissertation
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Quote: ‘students were randomly broken into two groups’ Quote: 'Half of the informed consent forms had small, black dots on them (from a sharpie marker).  The forms were shuffled.  After the informed consent process I had participants check to see whether their form had the black dot or not. Those with the black dot then moved to a room across the hall to complete the control condition of the study.' (a)
Allocation concealment (selection bias)	Low risk	Quote: ‘students were randomly broken into two groups’ Quote: 'Half of the informed consent forms had small, black dots on them (from a sharpie marker).  The forms were shuffled.  After the informed consent process I had participants check to see whether their form had the black dot or not. Those with the black dot then moved to a room across the hall to complete the control condition of the study.' (a)
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Comment: 15.5% across both groups did not attend 1 week follow‐up (a) , no information available on whether drop‐out was selective.
Selective reporting (reporting bias)	Unclear risk	Comment:  No protocol mentioned. All outcomes reported in methods had data reported in results.
Other bias	Low risk	Comment: Low risk for lack of evidence for reliability or validity of primary outcomes N/A for difference in baseline measures or participant characteristics in cluster trials with few clusters Low risk for evidence of counter‐discourse in follow‐up period N/A for outcomes between audience members and non‐audience not compared Low risk for items listed as potential other sources of bias in Higgins 2011 (section 8.15.1)
Blinding of participants and personnel (performance bias) All outcomes	Low risk	Comment: Cover story used, participants told their role was to rate the effects of music in advertising and were shown 4 10‐minute music performance videos with 2‐minute public service announcements in between, one of which was the intervention
Blinding of outcome assessment (detection bias) All outcomes	High risk	Comment: Outcome assessment was by self‐complete questionnaire

Dias‐Vieira 2005.

Methods	Aim of study: To test the hypothesis that receiving information about postpartum depression will result in (i) less stigmatising attitudes about a target with the disorder; less stigma towards mental illness in general; and decreased stigma towards receiving treatment for depression. Other hypotheses relating to effects on help‐seeking attitudes, and the moderating effects of target’s symptom severity and value placed on motherhood Study design:  RCT Recruitment: From university classes, informed that they were being asked to participate in research focused on improving the transition to parenthood (to analyse brochures to help people prepare for their lives as new parents and to gain a better understanding about people’s beliefs about new mothers which might help improve the brochures) Inclusion/exclusion criteria: Age 18+ Informed consent:  Yes Ethical approval: Not stated Funding: None stated Consumer involvement: None stated
Participants	Description: Undergraduate university students from seven upper level psychology courses, two introductory psychology courses, one introductory communications, and two advanced communications Geographic location: Rhode Island, New England USA Income level of country: High Setting: University Number: Total randomised 507, post randomisation exclusions 8, included in analysis 0 (no analysable data) Age: range 18 to 58, mean (SD) 19.74 (3.01) Gender: 159 (31%) male, 348 (69%) female Ethnicity: 87.2% White, 3.4% Asian American or Pacific Islander, 4.3% Black or African American, 3.0% Hispanic or Latino, 0.2% Native American, 0.8% Multiracial, 1.2% other. 12.1% did not provide details of ethnic background Other social/demographic details: 99% did not have children, 87% would like to in future. 11% unsure and 2% didn't want to
Interventions	Aim of intervention: To reduce the stigma associated with postpartum depression and increase healthcare seeking (assumed from hypotheses) Content of intervention: Brochure about postpartum depression covering symptomatology and prevalence and designed to counter some of the negative stereotypes about mothers who have postpartum depression. Entitled ‘What to expect when you are expecting: Information about the emotions of new mothers’.  Subheadings: Do women automatically feel happy after having a baby?; What are the baby blues?; What is postpartum depression? (gives prevalence and symptoms); What to do if you are experiencing postpartum depression (realise not alone and not a bad mother and recognising symptoms is first step towards improving things for you and your family); The good news about postpartum depression (effective therapies, talk therapy, medication or both, if untreated can have negative consequences for mother, baby and rest of family, the sooner a woman gets help the sooner she will start to feel better and enjoy her time with her family). Informed that brochure had been created by a leading physicians group to increase credibility. Content of control: Brochure  about pregnancy and preparing for the baby’s arrival that did not address postpartum depression entitled ‘Preparing for life with your new baby: what to do and when’. Subheadings: When should I start preparing for my baby’s arrival?; Do I need a helper list?; How do I prepare my home for baby’s arrival?; What will I need to take to hospital?; Don’t forget your partner or labour coach. Informed that brochure had been created by a leading physicians group to increase credibility. Details of co‐interventions in all groups: None Delivery of intervention: Survey packets (which included the intervention and control brochures) were distributed to groups that ranged in size from 5 to 35 students. Brochure was two pages long Providers: Produced by study author Type of mass media used in intervention: Print (text, brochure) Number of mass media components: One Combined with non‐mass media component: No Contains personal narrative: No Celebrities involved in intervention: No Fictional portrayals: No Primary message: Mixed Mental health condition: Postpartum depression
Outcomes	Primary outcome measures: Discrimination: none. Prejudice: Semantic Differential (postpartum depression) (SD‐PPD); two subscales of Semantic Differential: Understandability (SD‐U) and Evaluation (SD‐E); Stigma Scale for Receiving Psychological Help (SSRPH), modified for depression Secondary outcome measures: None Measure selected for analysis if multiple measures per outcome: SD‐PPD (powered on this measure) Methods of assessing outcome measures: Self‐complete questionnaire Validity and reliability of primary outcome measures: Internal consistency (Cronbach's alpha) using data from study participants: SD‐PPD alpha = 0.87; SD‐U alpha = 0.65; SD‐E alpha = 0.74; SSRPH alpha = 0.78 Methods of follow‐up for non‐respondents: N/A Timing of outcome assessment: Immediate
Notes	Contact with author: Sought, unsuccessful Power calculation: Yes Unpublished: Dissertation
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Quote: ‘Participants were given a survey packet from a group of packets that had previously been randomised in accordance with the respective brochure and vignette conditions’
Allocation concealment (selection bias)	Unclear risk	Quote: ‘Participants were given a survey packet from a group of packets that had previously been randomised in accordance with the respective brochure and vignette conditions’
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Quote: ‘Out of the 507 participants 19 had some missing data (16 missing 1‐3 qs and 3 missed between 11 and 13 items). Overall it seems that the study’s missing data was randomly distributed… Means calculated from the available data were used to replace any missing values…while mean replacement is not ideal, in this case where less than 1% of the data was missing, the impact of mean imputation was deemed to be minimal’ Comment: Also excluded 8 participants post‐randomisation as they were judged to have understood the true intention of the study. No information given on which groups they were in or whether excluded participants were spread equally between the two groups
Selective reporting (reporting bias)	Unclear risk	Comment: No protocol mentioned.  All outcomes in study methods have data reported for them
Other bias	Unclear risk	Comment: Unclear risk for reliability or validity of primary outcomes. One scale, SD‐U, had slightly low Cronbach's alpha (0.65) N/A for difference in baseline measures or participant characteristics in cluster trials with few clusters N/A for evidence of counter‐discourse in follow‐up period N/A for outcomes between audience members and non‐audience not compared Low risk for items listed as potential other sources of bias in Higgins 2011 (section 8.15.1)
Blinding of participants and personnel (performance bias) All outcomes	Low risk	Comment: Cover story used, participants told their role was to help improve the quality of materials to aid the transition to parenthood.
Blinding of outcome assessment (detection bias) All outcomes	High risk	Comment: Outcome assessment was by self‐complete questionnaire

Finkelstein 2008.

Methods	Aim of study: To assess if computer‐assisted anti‐stigma education can be effective in reducing the level of psychiatric stigma in a sample of special education students and if the effect is durable when assessed six months after the intervention Study design: RCT Recruitment: Enrolled into the study from special education university course Inclusion/exclusion criteria: None stated Informed consent: Not stated Ethical approval: Yes Funding: None stated Consumer involvement: None stated
Participants	Description: University students in their second year of a five year university programme for special education teachers (who teach children with learning disabilities, mental health problems, speech, hearing and vision disorders). They had just completed a one year, eight‐credit psychiatry course Geographic location: St Petersberg, Russia Income level of country: Upper middle income Setting: University (online) Number: Total randomised 193(intervention A 69, intervention B 76, control 48); lost to follow T1 0, T2 32; included in analysis 193 (prejudice 193, knowledge 193, audience reactions 145) Age: mean and 95% CI  19.1 ± 1.7 Gender: 99% female and 1% male (2 persons) Ethnicity: Not stated Other social/demographic details: Self‐reported mental health problems for two weeks or more in the past 36.3%; underwent psychiatric treatment 4.7%; considered themselves as currently having a severe psychiatric disease 1%; had relatives with psychiatric diseases 18.1%; current work included rendering services for people with psychiatric disorders 19.2%
Interventions	Aim of intervention:  To reduce stigma Content of intervention: Intervention A: computer‐assisted education system (CO‐ED) comprising educational messages followed by multiple choice questions, addressing cognitive, emotional and behavioural aspects of stigma, aiming to induce strong emotional response in learners and to provide information about the most common misperceptions. Covered: what is stigma, how widespread are psychiatric disorders, causes and treatments, are psychiatric patients dangerous, personal story of a person with schizophrenia able to live a fruitful and active life, myths and reality, mass killing of psychiatric patients in Nazi Germany, and what can I do. Intervention B: selection of reading materials (WHO brochure, article by Russian psychiatrist and article by British psychiatrist). Covered: what is stigma, epidemiology, causes of stigma in society, common misperceptions, stigma and discrimination, consequences of stigma, main ways to help psychiatric patients, and different approaches for reducing stigma. Content of control:  No intervention Details of co‐interventions in all groups: None Delivery: Delivered via the Internet during one visit but no information given on time taken to complete computer programme Providers: Intervention A (computer programme) was created at the University of Maryland (Chronic Disease Informatics Group) in collaboration with the St Petersberg State University.  Intervention B was a selection of existing materials (brochure on stigma and discrimination against the mentally ill by the European arm of the World Health Organisation), a Russian academic article (Gurovich 2001) and a British academic article (Lawrie 1999) Type of mass media: Intervention A: Internet (computer programme, web‐based). Intervention B: Print (written texts) Number of mass media components: Multiple Combined with non‐mass media component: No Contains personal narrative: Intervention A: yes. Intervention B: no Celebrities involved: No Fictional portrayals: No Primary message: Interventions A and B multiple messages Mental health condition: Severe mental disorders (schizophrenia, bipolar disorder and major depression)
Outcomes	Primary outcome measures: Discrimination: none. Prejudice: Bogardus Scale of Social Distance (BSSD) with vignettes on psychiatric disease and severe skin disorder, the latter to control for the Hawthorn effect; three subscales of the Community Attitudes to Mental Illness (CAMI): authoritarianism (CAMI‐A), benevolence (CAMI‐B) and social restrictiveness (CAMI‐SR) Measure selected for analysis if multiple measures per outcome: BSSD (study powered on this) Secondary outcomes: Knowledge: Psychiatric Knowledge Scale. Audience reactions: individual items developed by authors. Adverse events: authors stated ‘No adverse effects, such as an increase in stigma as a result of the intervention, were identified’ Methods of assessing outcome measures: Self‐complete questionnaire for all except acceptability which was assessed by semi‐structured interview. Validity and reliability of primary outcome measures: BSSD: ‘Reliability and validity [have been] established’ (one reference cited); ‘Cronbach alpha of the scale was reported to be 0.92’. CAMI: Internal consistency ranges from 0.68‐0.80 for the different subscales (cited from literature). Methods of follow‐up for non‐respondents: Self‐complete questionnaires completed in class at both time points Timing of outcome assessment: Immediate and 6 months
Notes	Contact with authors: Sought, unsuccessful Power calculation: Yes Additional paper: Finkelstein 2007 is a conference report, and reports data that is a subset of the data in Finkelstein 2008 (main paper for study)
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Quote 'simple randomisation has been used'
Allocation concealment (selection bias)	Unclear risk	Quote: ‘Subject allocation to the study group was carried out by research staff who were not aware about the baseline characteristics of the study subjects and did not have any prior relationships with the study subjects. Therefore the group allocation bias could not be exercised even at the subconscious level. Since the baseline characteristics of the study subjects were not available at the time of enrollment and participant assignment, no allocation concealment was utilized'
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Quote: ‘All study subjects completed the first visit including the pre/post questionnaires and the prescribed intervention… We had loss to follow‐up due to  student turnover and absence from classes. We believe that this loss to follow‐up was not associated with our intervention and was completely random.  The samples at visit one and visit two were not statistically different in relation to age, gender and psychosocial history in all study groups. Comment: All the losses seem to have come from the intervention groups (none lost from control) – which appears to suggest it is associated with the interventions
Selective reporting (reporting bias)	Unclear risk	Comment:  No protocol mentioned. Outcomes that the study says were assessed are reported.
Other bias	Low risk	Comment: Low risk for lack of evidence for reliability or validity of primary outcomes N/A for difference in baseline measures or participant characteristics in cluster trials with few clusters Low risk for evidence of counter‐discourse in follow‐up period N/A for outcomes between audience members and non‐audience not compared Low risk for items listed as potential other sources of bias in Higgins 2011 (section 8.15.1)
Blinding of participants and personnel (performance bias) All outcomes	High risk	Comment: Blinding of participants not possible.  No personnel involved in providing the intervention
Blinding of outcome assessment (detection bias) All outcomes	High risk	Quote: ‘The main study outcomes were represented by objective parameters eliminating possibility of subjectivity in assessment. Finally, we employed blind outcome assessment methodology to prevent risk of expectation bias of particular findings. This was achieved by coding group assignments in the study database and employing a statistician for the study analysis who was unaware which group was expected to the affected by the intervention’ Comment: Although the precautions above were used, as outcome assessment was by self‐complete questionnaire, there remains a high risk of detection bias

Han 2006.

Methods	Aim of study: To test the hypotheses that biological attribution education and destigmatising education would elevate willingness to seek professional help for depression and that combining the two types of education would have the greatest effect Study design: RCT Recruitment: From three universities Inclusion/exclusion criteria: None stated Informed consent: Yes Ethical approval: Not stated Funding: None stated Consumer involvement: None stated
Participants	Description: University students from three universities studying introductory psychology, nursing, nutrition, Chinese literature or Japanese language Geographic location: Northern Taiwan Income level of country: High Setting: University Number: Total randomised 299 (intervention A 75, intervention B 76, Intervention C 72, control 76, lost to follow up 56, included in analysis 0 (no analysable data) Age: range 18‐36, mean and 95% CI 20.3 ± 2.18 Gender: 73% female Ethnicity: not stated Other social/demographic details:  31% had religious beliefs; 48% (calculated) studying introductory psychology
Interventions	Aim of intervention: To increase willingness to seek professional help for depression Content of intervention: Intervention A: Biological information comprising of five short paragraphs explaining the biological aetiology and related information on depression (genes, neurotransmitters, endocrine systems and physiological characteristics of depression); Intervention B: Destigmatisation information comprising of five short paragraphs to reduce the psychological blameworthy attitude toward depressed people (do not have lower ability, do not have weaker willpower, are not close‐minded and reluctant to think positively, do not hold a lazy attitude towards improving themselves, depression is not a fault caused by the depressed person).  Intervention C: Combined comprising the 10 paragraphs above Content of control: No intervention Co‐interventions in all groups: None Delivery: Participants read the material, which took 5 to 10 minutes for Interventions A and B (assumed 10 to 20 minutes for Intervention C). Providers: Developed by study authors who were from Departments of Psychology and Psychiatry. Information was taken from textbooks on abnormal psychology and general psychiatry and research evidence, psychological theories and clinical observations were included as evidence Type(s) of mass media used in intervention  Print (written texts) Number of mass media components: One Mass media component combined with non‐mass media component in intervention:  No Contains personal narrative:  No Celebrities involved: No Fictional portrayals: No Primary message: intervention A: biomedical. Intervention B: Not to blame. Intervention C: biomedical Mental health condition: Depression
Outcomes	Primary outcome measures: Discrimination: none. Prejudice: Psychological Blame Scale (PBS) Secondary outcome measures: None Methods of assessing outcome measures Self‐complete questionnaires Validity and reliability of primary outcome measures  PBS was developed for study. Cronbach’s alpha: 0.70, test‐retest reliability 0.76, exploratory factor analysis found one single factor loading accounting for 49.3% of the total variance Methods of follow‐up for non‐respondents:  None specified Timing of outcome assessment: Two weeks post‐intervention
Notes	Contact with author: Sought, unsuccessful Power calculation:  None reported
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Quote: ‘all the participants were randomly assigned into one of four groups’
Allocation concealment (selection bias)	Unclear risk	Quote: ‘all the participants were randomly assigned into one of four groups’
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Decided in agreement meeting that groups not stated so unclear Quote: ‘the authors found a significant difference of gender percentages between participants who joined and did not join the second wave of data collection (i.e. 23% vs 47% were male respectively). There was also a significant difference of the percentages having religious beliefs… (34% vs 15%)’ Comment: Paper did not report in which groups these differences occurred
Selective reporting (reporting bias)	High risk	Comment:  No protocol available. Study reports that one primary and two secondary outcomes were assessed, data for the primary outcome is reported in full, but only a t‐test and P value are reported for the secondary outcomes (one of which was the stigma outcome for this review)
Other bias	Low risk	Comment: Low risk for lack of evidence for reliability or validity of primary outcomes N/A for difference in baseline measures or participant characteristics in cluster trials with few clusters Low risk for evidence of counter‐discourse in follow‐up period N/A for outcomes between audience members and non‐audience not compared Low risk for items listed as potential other sources of bias in Higgins 2011 (section 8.15.1)
Blinding of participants and personnel (performance bias) All outcomes	High risk	Comment: Blinding of participants and intervention providers not possible.
Blinding of outcome assessment (detection bias) All outcomes	High risk	Comment: Outcome assessment was by self‐complete questionnaire

Iobst 2008.

Methods	Aim of study: To determine the influence of different types of information on adults’ acceptance of a child with autism Study design: RCT Recruitment: From university classes Inclusion/exclusion criteria: Post‐randomisation exclusion: Those who failed manipulation check, that is they did not view the boy in video as having autism) Informed consent: Yes Ethical approval: Yes Funding: None stated Consumer involvement: None stated
Participants	Description University students, freshmen and sophomores, field of study unspecified Geographic location: Cincinnati, USA (assumed from author affiliation) Income level of country: High Setting: University Number: Total randomised 288, total randomised to groups relevant to this review 144, excluded post randomisation from groups relevant to this review 7, included in analysis 137 (prejudice 137) Age: mean 19 years and 10 months, SD 2 years and 10 months Gender: 50% female, 50% male Ethnicity: Caucasian 75%, African American 19%, Hispanic 1%, Asian 3% other 3% Other social/demographic details: 6% were parents, 69% were often or always around children, 15% were often or always around children with special needs
Interventions	Aim of intervention: To increase adults acceptance of a child with autism Content of intervention:  Intervention A: vignette about a boy with focusing on neuropsychological explanations for his behaviour. Intervention B: vignette about same boy with autism which just describes his behaviour. Intervention C: combination of material in A and B. Also three interventions irrelevant to this review involving vignettes about a 'neuro‐typical' boy. Content of control  No intervention Details of co‐interventions in all groups: Read an introductory vignette which names the child, describes his home town and activity preferences and states that his family may be moving to the area and he will be going to school in the neighbourhood.  Viewed a one minute video of a boy portraying autism as a stimulus for the outcome rating scale Delivery: Asked to read vignettes. Interventions A and B were each one paragraph long, C two paragraphs with explanatory information presented first Providers: Vignettes were developed for the study Type of mass media : Print (written text) Number of mass media components One Mass media component combined with non‐mass media component: No Contains personal narrative: Yes (third‐person) Celebrities involved:  No Fictional portrayals: Yes Primary message: Intervention A: biomedical. Intervention B: no message. Intervention C: biomedical Mental health condition: Autism
Outcomes	Primary outcome measures: Discrimination: None. Prejudice: Ratings of the Child Questionnaire (ROCQ) with a stimulus video of a 12 year old boy acting as if he had autism Secondary outcome measures: None Methods of assessing outcome measures: Self‐complete questionnaire Validity and reliability of primary outcome measures: Authors stated that ROQC has good psychometric properties and has been used in several studies assessing adult’s acceptance of a child with special needs (five references given, covered cerebral palsy, prematurity, general, cancer). No references or comment that it has been used to rate a child with autism. The stimulus video had been used in a previous study and involved a boy who had a brother with autism and was rated by professionals as being a realistic portrayal Methods of follow‐up for non‐respondents: N/A Timing of outcome assessment: Immediate
Notes	Contact with author: Sought, unsuccessful Power calculation: None stated
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Quote: ‘Prior to the beginning of the study, participants were randomly assigned to view one of the videotapes (autism or typical boy, latter not relevant to review). They were also randomly assigned to vignette condition… An equal number of males and females were assigned to each videotape and vignette condition’
Allocation concealment (selection bias)	Unclear risk	Quote: ‘Prior to the beginning of the study, participants were randomly assigned to view one of the videotapes (autism or typical boy, latter not relevant to review). They were also randomly assigned to vignette condition… An equal number of males and females were assigned to each videotape and vignette condition’
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Comment: Seven participants who failed manipulation check (did not realise boy in video had autism) were excluded post‐randomisation. Paper does not state how this was spread across groups.  Unlikely to be related to stigma
Selective reporting (reporting bias)	Unclear risk	Comment: No protocol. All outcomes in measures section were reported in results
Other bias	Unclear risk	Comment: Lack of evidence for reliability or validity of primary outcomes. No references or comment in paper to demonstrate that measure had previously been used to rate a child with autism N/A for difference in baseline measures or participant characteristics in cluster trials with few clusters N/A for evidence of counter‐discourse in follow‐up period N/A for outcomes between audience members and non‐audience not compared Low risk for items listed as potential other sources of bias in Higgins 2011 (section 8.15.1)
Blinding of participants and personnel (performance bias) All outcomes	High risk	Comment: Blinding of participants and personnel not possible
Blinding of outcome assessment (detection bias) All outcomes	High risk	Comment: Outcome assessment was by self‐complete questionnaire

Jorm 2010a.

Methods	Aim of study: To evaluate the effects of mental health first aid training on the public via an e‐learning CD, compared to either the information in a printed mental health first aid manual or receiving no training Study design: RCT Recruitment: Through online advertisements on community and research notice boards (related and unrelated to mental health), flyers in university libraries, community and mental health centres, adverts in local newspapers and newsletters promoting mental health.  The adverts called for participants in an evaluation of mental health first aid educational materials aimed at adult members of the general public Inclusion/exclusion criteria: Adults (18+) from the general community in Australia, with access to a computer with a CD drive and the Internet. Exclusion: unable to read English, living overseas, completed face‐to‐face adult Mental health First Aid course within past 2 years, intending to take face‐to‐face course, working in clinical psychology area, too busy, recovering from illness (from Australia and New Zealand Clinical trial Registry (ACTRN12608000188336) Informed consent: This was inferred by study authors by virtue of completing the form electronically (signature not required) and returning it to the trial administrator (a) Ethical approval: Yes Funding: Study funded by Australian Rotary Health, a National Health and Medical Research Council Fellowship, and the Colonial Foundation. Intervention was supported by the Australian Department of Employment and Workplace Relations Consumer involvement: Yes. Mental health consumer advocate with experienced recurrent major depression is a study author is, and co‐developed intervention.
Participants	Description: General public Geographic location: Australia Income level of country: High Setting: Intervention A: location with computer access  Intervention B: any Number: Assessed for eligibility 297, excluded 35, total randomised 262 (intervention A 90, intervention B 88, control 84), lost to follow up T1 24, T2 33, included in analysis 238 (prejudice 234, knowledge 238, audience reactions 130) Age  range not stated, mean (SD) 40 (12) Gender  81% female   Ethnicity 91% Australian. The majority of these 9%, 13 people, had English speaking country citizenship (mainly Britain and New Zealand). There were a further 6 people with Asian and 3 people with European country citizenship (a) Other social/demographic details 56% had a bachelor or higher degree, 37% were currently studying; 91% were Australian citizens, 88% had English as their first language, 79% lived in major cities, 18% in regional areas and 3% in remote areas, 52% reported having a history of mental health problems
Interventions	Aim of intervention: To teach members of the general public to recognise the early signs of mental illness and to provide initial help to a person developing a mental disorder or in a mental health crisis until appropriate professional help is received or the crisis resolves Content of intervention: Intervention A Computer (CD‐ROM) mental health first aid training, presents information with interactive exercises and includes video clips of people who have mental health problems (anxiety disorder, depression, bipolar disorder and schizophrenia); action plans (assess risk of suicide or harm, listen non‐judgementally, give reassurance and relevant information, encourage appropriate professional help, encourage self‐help strategies); interactive case studies applying the action plan; knowledge quiz; and links to online resources.  Intervention B Mental health first aid printed manual with the same content, but without the video clips and interactive case studies. Content of control:  Waiting list control Details of co‐interventions: in all groups None Delivery: Participants were sent weekly emails for a month to help pace them through the CD or manual giving them 4 weeks to complete the materials. Average time to complete not specified. Providers: Interventions developed by Betty Kitchener (mental health consumer advocate with experience of recurrent major depression and academic researcher) and Antony Jorm (Professor of Psychology) Type of mass media: Intervention A: Interactive CD‐ROM (classified as Internet for purposes of this review); Intervention B Print (educational manual) Number of mass media components: Multiple Combined with non‐mass media component in intervention: No Contains personal narrative: Intervention A yes (first‐person), Intervention B: no Celebrities involved: No Fictional portrayals: No Primary message: Seek professional care Mental health condition: Various (including covers depression, bipolar disorder, anxiety disorders, psychosis and substance abuse)
Outcomes	Primary outcome measures: Discrimination: none. Prejudice: Subscales of Personal and Perceived Stigma scale: Personal Stigma Scale (schizophrenia) (PersonalSS(s)), Personal Stigma Scale (depression) (PersonalSS(d)), Perceived Stigma Scale (schizophrenia) (PerceivedSS(s)), Perceived Stigma Scale (depression) (PerceivedSS(d)); Social Distance Scale (schizophrenia vignette) (SDS(s)); Social Distance Scale (depression vignette) (SDS(d) Secondary outcome measures: Knowledge: Recognition depression/schizophrenia from four vignettes (cited as used in previous studies); beliefs about treatments for schizophrenia/depression matching those of health professionals (two items, developed for study). Audience reactions: individual items developed by study authors. Cost: costs of interventions provided by authors. Adverse events: No formal enquiry, statement that 'informally, no adverse events were reported' Measures selected for analysis if multiple measures per outcome: Prejudice: SDS(s) (following Brennan 2009). Knowledge: Beliefs about depression treatment item (following Brennan 2009) Methods of assessing outcome measures Online self‐complete questionnaires Validity and reliability of primary outcome measures: Personal and perceived attitude scales (one reference cited), present study replicated the two factor solution found in previous study about scale. Social Distance Scale (two references cited) Methods of follow‐up for non‐respondents: None stated Timing of outcome assessment: 1 month and 6 months
Notes	Contact with author: Yes Power calculation: Yes Additional data taken from Australia and New Zealand Clinical trial Registry (ACTRN12608000188336)
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Quote:  ‘A computerised random number generator … was used to carry out a simple randomisation procedure, where a randomly assigned variable (1‐3) was assigned to each participant’s ID number to determine which condition they were allocated to’
Allocation concealment (selection bias)	Low risk	Quote: ‘The PI carried out the random number generation using participants ID numbers (but with no other knowledge of participant’s details). The allocation sequence was concealed from the PI until it was time to advise participants of their random assignment and send educational materials as required; that is after baseline data was collected’
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Quote: ‘There was a difference in sample retention … may have been due to the promise that they would receive ... the CD at the end of the trial, whereas the CD group already had the intervention that was the focus of the study. Despite the differential retention rate, the drop‐out rate was relatively small and there was no evidence that this biased the findings’
Selective reporting (reporting bias)	Low risk	Comment:  Protocol available in trial registry and checked. All outcomes in protocol reported in this paper
Other bias	Low risk	Comment: Low risk for lack of evidence for reliability or validity of primary outcomes N/A for difference in baseline measures or participant characteristics in cluster trials with few clusters Low risk for evidence of counter‐discourse in follow‐up period N/A for outcomes between audience members and non‐audience not compared Low risk for items listed as potential other sources of bias in Higgins 2011 (section 8.15.1)
Blinding of participants and personnel (performance bias) All outcomes	High risk	Comment: Blinding of participants not possible.  Blinding of personnel N/A as postal intervention
Blinding of outcome assessment (detection bias) All outcomes	High risk	Comment: Outcome assessment was by self‐complete questionnaire

Kerby 2008.

Methods	Aim of study: To determine the effects of an anti‐stigma intervention based on films produced by mental health service users and combining both education and stereotype disconfirmation elements, on medical students' attitudes to both serious mental illness and psychiatry Study design: RCT Recruitment: From medical school Inclusion/exclusion criteria: None stated Informed consent: Authors stated 'written informed consent was not required as the project was an assessment of an educational intervention'. Students had the opportunity not to take part (36 declined to participate).  Ethical approval: Yes Funding: None stated Consumer involvement: One of the study authors is a person with experience of mental illness (inferred from his appearance in one of the first‐person narrative films). Both intervention films were made by or in collaboration with, people with personal experience of mental illness
Participants	Description:  4th year medical undergraduates at the start of their psychiatry attachment Geographic location: Nottingham, UK (assumed from author affiliation) Income level of country: High Setting: Medical School Number: Eligible 82, refused to take part 36, total randomised 46 (intervention 23, control 23, lost to follow up T1 4, T2 5, included in analysis 42 (prejudice 42) Age: mean 21 Gender: 74% female Ethnicity: 80% White European Other social/demographic details: 60% had previous contact with a person diagnosed with a serious mental illness
Interventions	Aim of intervention: To challenge stereotypes and mention positive aspects of serious mental illness, convey first‐hand experience to challenge the stereotype that psychosis is a condition opaque to understanding Content of intervention: Two anti stigma films: 'A Human Experience' and 'A Day in the Mind of ...' The first adopts a talking head documentary style approach and revolves around three mental health professionals (a teacher/researcher, a Mental Health Act Commissioner and a psychiatrist) discussing their experiences of being diagnosed with a serious mental illness (psychosis, schizophrenia and severe depression, all of which required hospitalisation) and in particular their experiences of stigma and discrimination. The film challenges particular stereotypes beliefs, including dangerousness, inability to work and maintain relationships, and mentions positive aspects of serious mental illness (importance in forging personal identity, sense of overcoming adversity, celebration of difference, deepening of lived experience. The second adopts a first‐person perspective. Its narrative focuses on the subjective experience of psychosis over the course of a typical day and attempts of convey the first‐hand experience of being diagnosed with a serious mental illness, challenging the stereotype of psychosis as a condition opaque to understanding. Content of control:  Documentary film unrelated to mental illness or psychiatry matched for visual format Co‐interventions in all groups: None Delivery: Anti‐stigma films 15 and 12 minutes (combined = 27 minutes), control film 25 minutes. Watched films on the second day of their psychiatry attachment. Providers: 'A Human Experience' was produced  Mark Smith (2005) in collaboration with service users from Rethink (UK Mental Health charity for severe mental illness). 'A day in the mind of …' was produced by Graeme Green (2005) and made by service users at Framework Housing Association. Type of mass media: Audiovisual (documentary films) Number of mass media components in intervention: Two Mass media component combined with non‐mass media component: No Contains personal narrative: Yes (first‐person) Celebrities involved:  No Fictional portrayals: No Primary message: Multiple messages Mental health condition: Serious mental illness (schizophrenia, psychosis, severe depression)
Outcomes	Primary outcome measures: Discrimination: None. Prejudice: Attitudes Toward Serious Mental Illness Scale – Adolescent version (ATourette's SyndromeMI‐AV), Dangerousness Scale (DS), Social Distance Scale (SDS), Behavioural Intentions towards people with mental illness (BI). We did not consider Attitudes to Psychiatry as a stigma measure as it encompasses more than stigma Secondary outcomes: None Measure selected for analysis if multiple measures per outcome: DS (following Grimshaw 2003) Methods of assessing outcome measures Self‐complete questionnaire Validity and reliability of primary outcome measures: ATourette's SyndromeMI‐AV: one reference cited, no information on validity or reliability, authors state that it measures attitudes ‘in young people’ but unclear how appropriate an adolescent version is for medical students. DS: paper states that had good internal consistency. SDS paper states that had good internal consistency. BI scale unspecified, direction not of scale specified, no information on psychometrics Methods of follow‐up for non‐respondents:  None stated Timing of outcome assessment: Immediate and at 8 weeks
Notes	Contact with author: Sought, unsuccessful Power calculation: None reported
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Quote: ‘they were randomly allocated using a concealed randomisation method’
Allocation concealment (selection bias)	Low risk	Quote: ‘they were randomly allocated using a concealed randomisation method’
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Comment: Figure 1 shows that 5 people were lost to follow‐up but the 'n =' header at the top of the results section refers to the total number randomised. There is no information about imputation
Selective reporting (reporting bias)	High risk	Comment: A behavioural intentions measure is listed in measures section, but, unlike the other measures, there is no statistical test information given in the results section for this measure
Other bias	High risk	Comment: High risk for lack of evidence for reliability or validity of primary outcomes: ATourette's SyndromeMI‐AV one reference given, no information on validity or reliability, reports that it measures attitudes ‘in young people’ but unclear how appropriate an adolescent version is for medical students; DS, stated that had good internal consistency; SDS stated that had good internal consistency; BI scale not described, direction not specified, no information on psychometrics N/A for difference in baseline measures or participant characteristics in cluster trials with few clusters Low risk for evidence of counter‐discourse in follow‐up period N/A for outcomes between audience members and non‐audience not compared Low risk for items listed as potential other sources of bias in Higgins 2011 (section 8.15.1)
Blinding of participants and personnel (performance bias) All outcomes	High risk	Comment: Blinding of participants and personnel was not possible.
Blinding of outcome assessment (detection bias) All outcomes	High risk	Quote: Statistical analyses were undertaken by an independent researcher masked to allocation status and all participants were asked not to reveal their group allocation Comment: Although the precautions above were taken the assessment was by self‐report questionnaire consequently the study was rated as high risk

Matthews 2009.

Methods	Aim of study: Can reading a non‐fictional, transporting, non‐threatening narrative persuade readers to reduce their stigma toward mental illness? (also is the effect of the intervention greater for participants who score higher on the transportation scale; and is greater familiarity with mental illness associated with lower stigma regardless of condition?) Study design: RCT Recruitment: Presentations in classes at university Inclusion/exclusion criteria: None stated Informed consent: Yes Ethical approval:Yes Funding: None (a) Consumer involvement: Yes, in study design (a)
Participants	Description: University students on introductory journalism and business communication courses in two universities Geographic location: Nova Scotia, Canada (a) Income level of country: High Setting: University Number: Total randomised 61 (intervention 31, control 30), withdrawn 1, included in analysis 60 (prejudice 60) Age: range 18 to 49, mean (SD)  22.4 (7.9) Gender: 37 females (61.7%), 23 males (38.3%) Ethnicity: 48 (80%) Caucasian; 6 (10%)  Black; 3 (5%) Asian; 3 (5%) other Other social/demographic details: 31 at one university and 29 at the other
Interventions	Aim of intervention: To reduce stigma Content of intervention: Early psychosis story, about a young woman with early psychosis, was designed and written specifically to avoid evoking fear: friends and family were not afraid of the young woman; biogenetic explanations were removed from the original published version; outcomes were positive; and personal information was included in an effort to promote familiarity. The story was based on a young woman who was one of the subjects of a video created by the Nova Scotia Early Psychosis Unit. The story used in this study was a shortened version of a magazine feature entitled “Mending minds” originally published in a regional health magazine in 2008 by the study author, with photograph (smiling, attractive individual), contains information about the character’s life, family, response to treatment, as well as information about the treatment for early psychosis Content of control:  Eye disorders story, a profile of the eye care centre at the QEII Health Sciences Centre in Halifax, Nova Scotia. It was originally published in a regional health magazine with photographs Details of co‐interventions in all groups: Diabetes story (to disguise true purpose) and, as stimulus for outcome measure, schizophrenia story (three‐sentence vignette for measure, disguised as story, with illustration of a man with a neutral expression) Delivery of intervention: Both intervention stories were approximately 800 words in length, participants were asked to read them in class Provider: Developed by study author, by shortening a magazine article of her own previously published in a regional health magazine, based story of person in video created by the Nova Scotia Early Psychosis Unit. Type of mass media: Print (magazine article, text with photographs) Number of mass media component: One Mass media component combined with non‐mass media component: No Contains personal narrative: Yes (third‐person) Celebrities involved: No Fictional portrayals: No primary message: Recovery oriented Mental health condition: Early psychosis
Outcomes	Primary outcome measures: Discrimination: none. Prejudice: Standardized Stigmatization Questionnaire (SSQ) Secondary outcome measures: Cost: Cost of intervention information provided by author Methods of assessing outcome measures Self‐complete questionnaire Validity and reliability of primary outcome measures: SSQ: Cronbach's alpha from study 0.87 Methods of follow‐up for non‐respondents N/A Timing of outcome assessment: Immediate
Notes	Contact with author: Yes Power calculation: None (a) Unpublished: Dissertation
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Quote: ‘Participants were randomly assigned to control vs intervention conditions’ Quote ‘table of random numbers’ (a)
Allocation concealment (selection bias)	Unclear risk	Quote: ‘Participants were randomly assigned to control vs intervention conditions’
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Comment: One person withdrew after debriefing which could have been related to outcomes
Selective reporting (reporting bias)	Low risk	Comment: All outcomes in measures section of IRB application were reported in results section of dissertation (a)
Other bias	Low risk	Comment: Low risk for lack of evidence for reliability or validity of primary outcomes N/A for difference in baseline measures or participant characteristics in cluster trials with few clusters N/A for evidence of counter‐discourse in follow‐up period N/A for outcomes between audience members and non‐audience not compared Low risk for items listed as potential other sources of bias in Higgins 2011 (section 8.15.1)
Blinding of participants and personnel (performance bias) All outcomes	Low risk	Comment: Cover story used, intervention/control stories embedded with two other stories, participants told purposed was to rate length and design of all three stories
Blinding of outcome assessment (detection bias) All outcomes	High risk	Comment: Outcome assessment was by self‐complete questionnaire

Morgan Owusu 2002.

Methods	Aim of study: To investigate whether a culturally specific intervention for Black college students is a more effective approach for reaching those who never utilise mental health services, with hypothesis two being that they will express more positive opinions about mental illness Study design: RCT Recruitment: From psychology classes and university pre‐professional organisations and social clubs Inclusion/exclusion criteria: Inclusion: Black college students. Exclusion: students with previous therapeutic experience of clinical psychiatry history Informed consent: Yes Ethical approval: Not stated Funding: None stated Consumer involvement: None stated
Participants	Description: Black university students from several colleges from psychology classes, clubs and organisations, freshmen to seniors Geographic location: New York, USA (from author affiliation) Income level of country: High Setting: University Number: Total randomised 90 (intervention A 30, intervention B 30, control 30) included in analysis 90 (prejudice 90, knowledge 90) Age:  range 17 to 23 Gender: 41 (45.6%) male Ethnicity: All Black Other social/demographic details: Single 88 (97.8%), married 1 (1.1%) divorced 1 (1.1%); freshman 15 (16.7%), sophomore 24 (26.7%) junior 26 (28.9%), senior 23 (25.6%) (% calculated)
Interventions	Aim of intervention: To increase use of mental health services and produce more positive attitudes toward mental illness Content of intervention: Intervention A Standard psycho‐educational video presentation. Covered definition of mental illness, prevalence of mental illness, description of two specific disorders (depression/manic depression and schizophrenia), common treatments for mental illness, description of psychotherapy and counselling and resources available on campus for emotional difficulties, with each topic introduced by a question and followed by an answer that explained more about the topic. Intervention B Psycho‐educational video presentation culturally specific to Blacks, same presenter (ethnicity unspecified) and content as intervention A but it acknowledged and emphasised cultural factors in mental illness and psychological problems Content of control:  Psycho‐educational video presentation unrelated to mental health problems, on study skills Co‐interventions in all groups: None Delivery: Watched at college in groups of around 10. Length of videos unspecified; Intervention B appears to have been longer as it had additional material. Asked to spend two minutes listing thoughts after viewing interventions. Providers: Scripts were based on the videotape presentation used in a previous study Type of mass media: Audiovisual (video) Number of mass media components: One Mass media component combined with non‐mass media component: No Contains personal narrative: No Celebrities involved: No Fictional portrayals: No Primary message: Seek professional care Mental health condition: Mental illness
Outcomes	Primary outcome measures: Discrimination: None. Prejudice: Subscales of the Opinions about Mental Illness questionnaire (OMI): Authoritarianism (OMI‐A); Benevolence (OMI‐B) and Social Restrictiveness (OMI‐SR). Study also included OMI Mental health hygiene ideology subscale, however we excluded this as it relates to aetiology rather than stigma Secondary outcome measures: Knowledge: Seven items developed for study Measures selected for analysis if multiple measures per outcome: Prejudice: OMI‐B (following Grimshaw 2003 Knowledge: 'What is mental illness? item (following Grimshaw 2003) Methods of assessing outcome measures: Self‐complete questionnaire Validity and reliability of primary outcome measures: In present study Cronbach's alpha values were OMI‐A: alpha = 0.64, OMI‐B: alpha = 0.70, OMI‐SR alpha = 0.55 Methods of follow‐up for non‐respondents: N/A Timing of outcome assessment: Immediate
Notes	Contact with author: Sought, unsuccessful Power calculation: None reported  Unpublished: Dissertation
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Quote: ‘participants were randomly assigned to one of the three treatment groups’
Allocation concealment (selection bias)	Unclear risk	Quote: ‘participants were randomly assigned to one of the three treatment groups’
Incomplete outcome data (attrition bias) All outcomes	Low risk	Comment: No missing outcome data
Selective reporting (reporting bias)	Unclear risk	Comment: No protocol, outcomes in measures reported in results. For two knowledge items no data were reported and for one findings were only reported as statistical test findings rather than in full table as others were
Other bias	High risk	Comment: High risk for lack of evidence for reliability or validity of primary outcomes as in present study Cronbach's alpha values were OMI‐A: alpha = 0.64, OMI‐B: alpha = 0.70, OMI‐SR alpha = 0.55 N/A for difference in baseline measures or participant characteristics in cluster trials with few clusters N/A for evidence of counter‐discourse in follow‐up period N/A for outcomes between audience members and non‐audience not compared Low risk for items listed as potential other sources of bias in Higgins 2011 (section 8.15.1)
Blinding of participants and personnel (performance bias) All outcomes	Low risk	Comment: Cover story used, participants told they were rating new lecture material
Blinding of outcome assessment (detection bias) All outcomes	High risk	Comment: Outcome assessment was by self‐complete questionnaire

Penn 2003.

Methods	Aim of study: To examine the effects of a schizophrenia documentary on stigma‐related variables Study design: Cluster RCT Size and description of clusters: In classes of 20 to 30 students Recruitment: From university courses Inclusion/exclusion criteria: None stated Informed consent: Yes Ethical approval: Yes (a) Funding: None (a) Consumer involvement: None (a)
Participants	Description: University undergraduates (courses unspecified) Geographic location: Chapel‐Hill, North Carolina, USA Income level of country: High Setting: University Number: Total randomised to all groups 163, total randomised to relevant groups not stated (assume 82), lost to follow‐up (relevant groups) 1, included in analysis 81 (discrimination 81, prejudice 79) Age: mean (SD) 18.85 (0.87) Gender: 55.8% female Ethnicity: 81.5% Caucasian
Interventions	Aim of intervention: To personalise the disorder [schizophrenia] which would weaken negative stereotypes Content of intervention: Documentary film about schizophrenia. ‘I’m still here’ Contained ‘up‐close’ testimonials, depicts people with schizophrenia and their families. The clinical authors felt that it presented a realistic image of schizophrenia, by showing individuals with varying forms of illness severity and course. One individual who was depicted is employed and has a spouse and children, while some others are less independent and more symptomatic; one individual is acutely psychotic and shown homeless in Central Park, while another has negative symptoms and lives at home with her parents. The film also underscores the potential devastation of schizophrenia by depicting an individual who appears fully recovered but later commits suicide after the illness returns. Content of control A: Documentary film about polar bears. Control B: No intervention. Control A selected as relevant control for review as it was more closely matched to the intervention. Also one arm irrelevant to review: documentary film about overweight people and the stigma they face. Co‐interventions in all groups: None Delivery of intervention: Duration: intervention A: 67 minutes long (a), control: unspecified. Films were shown in a classroom of 20 to 30 people Details of providers: Not stated Type of mass media: Audiovisual (video) Number of mass media components: One Mass media component combined with non‐mass media component in intervention: No Contains personal narrative:  Yes Celebrities involved:  No Fictional portrayals: No Primary message: Multiple Mental health condition: Schizophrenia
Outcomes	Primary outcome measures: Discrimination: Index of Behavioural Intention (IBI) in which participants were asked to indicate their interest (response 'yes' and phone number) in attending a focus group with persons with schizophrenia to discuss issues related to stigma. Prejudice: Social Distance Scale (SDS), Dangerousness Scale (DS), Affect Scale (AS), two subscales of the Attributions Questionnaire: blame/responsibility (AQ‐B/R) and changeability (how likely it is that person’s condition – schizophrenia – would change) (AQ‐C) Secondary outcome measures: None Measure selected for analysis if multiple measures per outcome: SDS (following Grimshaw 2003) Methods of assessing outcome measures: Self‐complete questionnaires and if gave phone number for focus group Validity and reliability of outcome measures:Cronbach's alphas from study data: SDS alpha = 0.80, DS alpha = 0.75, AS alpha = 0.83. No psychometrics given for AQ‐B/R, AQ‐C or IBI Methods of follow‐up for non‐respondents: N/A Timing of outcome assessment: Immediate
Notes	Contact with author: Yes Power calculation: No (a)
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Quote: ‘Participants were tested in a classroom of 20‐30 persons/session to facilitate data collection, and each group of participants was randomly assigned to receive one of the experimental conditions’. Quote: 'I believe it was a table of random numbers' (a)
Allocation concealment (selection bias)	Low risk	Quote: ‘Participants were tested in a classroom of 20‐30 persons/session to facilitate data collection, and each group of participants was randomly assigned to receive one of the experimental conditions’. Quote: 'I believe the [research assistant] was given the random assignment list prior to testing participants' (a)
Incomplete outcome data (attrition bias) All outcomes	Low risk	Comment: Total N = 163, 158 observations represented in table 1, no reasons given for missing data, but appears balanced in numbers across intervention groups
Selective reporting (reporting bias)	Unclear risk	No protocol. All outcomes mentioned in measures section were reported in results section
Other bias	Unclear risk	Comment: Unclear risk for lack of evidence for reliability or validity of primary outcomes: Cronbach's alphas from study data: SDS alpha = 0.80, DS alpha = 0.75, AS alpha = 0.83. No psychometrics given for AQ‐B/R, AQ‐C or IBI Correlation coefficient not available from author (a) N/A for difference in baseline measures or participant characteristics in cluster trials with few clusters N/A for evidence of counter‐discourse in follow‐up period N/A for outcomes between audience members and non‐audience not compared Unclear risk for cluster trial not analysed to adjust for clustering, intra class Low risk for other items listed as potential other sources of bias in Higgins 2011 (section 8.15.1)
Blinding of participants and personnel (performance bias) All outcomes	High risk	Comment: Blinding of participants and personnel not possible
Blinding of outcome assessment (detection bias) All outcomes	High risk	Comment: Discrimination outcome: low risk as it was by adding a phone number on a slip about joining a focus group, therefore not evidently a stigma measure. Prejudice outcome: Outcome assessment was by self‐complete questionnaire, therefore high risk

Reinke 2004.

Methods	Aim of study: To contrast the effects of in vivo and videotaped contact and to investigate whether media presentations of people with mental illness that moderately disconfirm stereotype will yield the best effects on stigma compared to those that do not and highly disconfirm stereotypes. Study design: RCT Recruitment: Drawn from the at large student body Inclusion/exclusion criteria: None stated Informed consent: Unclear Ethical approval: Not stated Funding: Part‐funded by NIMH grant (MH‐62198) which funds the Chicago consortium for stigma research Consumer involvement: Yes, co‐author and in intervention
Participants	Description: Community college students Geographic location: Chicago, USA (assumed from affiliation of first 4 authors) Income level of country: High Setting: Community college Number: Total randomised to all groups 164, total randomised to relevant groups not stated (assume 131), included in analysis 131 (prejudice 131) Age: mean (SD) 24.7 (9.3) Gender:  66.5% female Ethnicity: 50.6% European American, 38.2% African American, 8.8% Latino, 0.6% Asian, 0.6% Native American Other social/demographic details: 46.5% had completed high school, 52.4% had received some college or an associate degree, 0.6% had earned a bachelors degree. For household income 25.3% earned less than $20,000, 30.6% earned $20,000 to $40,000, 16.5% earned $40,000 to $60,000, 5.9% earned $60,000 to $80,000 and 14.7% earned more than $80,000. 76.5% single, 14.1% married, 2.4% separated and 7.1% divorced
Interventions	Aim of intervention: To reduce stigmatising attitudes Content of intervention: Intervention A: videotaped contact with individual with schizoaffective disorder which moderately disconfirms stereotypes. Intervention B: videotaped contact with the same individual emphasising information that highly disconfirms stereotypes. Intervention C: videotaped contact with the same individual emphasising information with little or no information that disconfirms stereotypes. Also one arm not relevant to review: in vivo contact with the same individual that moderately disconfirms stereotypes Content of control  Videotape entitled ‘Hobbies and technology in the 90s’ which discussed no issues related to mental illness of physical disability Details of co‐interventions in all groups: All groups had five minutes of facilitated discussion after the presentations in which the participants could ask questions about the presentations Delivery: Each intervention lasted 10 minutes.  Presentations were given to groups of 4 to 8 participants by presenters who were not identified as having a mental illness. For the three video interventions and the video control conditions, group leaders rotated to control for presenter effects. In Intervention A, participants asked the person who provided the in vivo contact questions Providers: Intervention A and the control had been used in a previous study by the research group. Interventions B to D were created for the present study by the research team, with one team member talking about his experiences of mental illness in Interventions A to D Type of mass media: Audiovisual (video) Number of mass media components:  One Mass media component combined with non‐mass media component: Yes Contains personal narrative: Yes Celebrities involved in intervention: No Fictional portrayals: No Primary message: Interventions A and B recovery oriented and intervention C negative impact of mental illness Mental health condition: Schizoaffective disorder
Outcomes	Primary outcome measures: Discrimination: none. Prejudice: Social Distance Scale (SDS) Secondary outcome measures: None Methods of assessing outcome measures: Self‐complete questionnaire Validity and reliability of outcome measures: SDS: four references cited, internal consistency from earlier study of Cronbach's alpha = 0.76 Methods of follow‐up for non‐respondents: N/A Timing of outcome assessment: Immediate
Notes	Contact with author: Sought, unsuccessful  Power calculation: None stated
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Quote: ‘Participants were randomly allocated’
Allocation concealment (selection bias)	Unclear risk	Quote: ‘Participants were randomly allocated’
Incomplete outcome data (attrition bias) All outcomes	Low risk	Quote: Participants ‘completed all measures’
Selective reporting (reporting bias)	Unclear risk	Comment: No protocol mentioned.  Study reports that one outcome was assessed and this is reported.
Other bias	High risk	Comment: Low risk for lack of evidence for reliability or validity of primary outcomes N/A for difference in baseline measures or participant characteristics in cluster trials with few clusters N/A for evidence of counter‐discourse in follow‐up period N/A for outcomes between audience members and non‐audience not compared High risk for significant baseline imbalance in prejudice measure Low risk for other items listed as potential other sources of bias in Higgins 2011 (section 8.15.1)
Blinding of participants and personnel (performance bias) All outcomes	High risk	Comment: Blinding of participants and intervention providers not possible
Blinding of outcome assessment (detection bias) All outcomes	High risk	Comment: Outcome assessment was by self‐complete questionnaire

Russell 1988.

Methods	Aim of study: To determine the effects of a direct‐mail campaign on the attitudes of managers and presidents of industries in Alabama towards people with intellectual disability Study design: RCT Recruitment: From a total list of managers and presidents of manufacturing industries in Alabama employing five or more people Inclusion/exclusion criteria: Exclusion managers and presidents in industries employing four or fewer people Informed consent: Not stated Ethical approval: Not stated Funding: Not stated Consumer involvement: Intervention pamphlets were developed by the Association for Retarded Citizens
Participants	Description: Managers and presidents in manufacturing industry Geographic location: Alabama, USA Income level of country: High Setting: Workplace (direct mail campaign) Number: Eligible 4290, approached for participation 300, total randomised 150 (n per group not stated), lost to follow up 61, included in analysis 99 (prejudice 99) Age: Not stated Gender: Not stated Ethnicity: Not stated
Interventions	Aim of intervention: To improve attitudes of managers and presidents toward people with intellectual disability Content of intervention: Three pamphlets developed to provide specific information about ‘mentally retarded persons’ to employers (i) ‘The truth about mental retardation’ which provided a general overview of mental retardation (ii) ‘Working together with mentally retarded employees’  which provided positive information dealing with the specifics of employing a 'mentally retarded' person, (iii) ‘This isn't kindness, it’s good business’ described the on‐the‐job training project administered by the Association for Retarded Citizens and provided positive reasons for hiring mentally retarded persons Content of control: No intervention Co‐interventions in all groups: None Delivery: Sent out by mail at weekly intervals; length of pamphlets unspecified Details of providers: Developed by the Association for Retarded Citizens Type of mass media used in intervention: Print (direct mail pamphlets) Number of mass media components: Three components Mass media component combined with non‐mass media component: No Contains personal narrative: No Celebrities involved: No Fictional portrayals: No Primary message: Social inclusion/human rights Mental health condition: Intellectual disability
Outcomes	Primary outcome measures: Discrimination: none. Prejudice: Attitude Toward Disabled Persons Scale (ATDPS), modified with ‘disabled’ replaced with ‘mentally retarded’ Secondary outcome measures: None Methods of assessing outcome measures: Self‐complete questionnaire Validity and reliability of outcome measures Cited data from previous studies: test‐retest reliability ranged from 0.66 to 0.89 with a median of 0.73 with time intervals ranging from 2 weeks to 18 months, split‐half reliability ranged from 0.75 to 0.85, correlates with other relevant measures Methods of follow‐up for non‐respondents: None stated Timing of outcome assessment: One week after last mailing
Notes	Contact with author: Attempted, not sought (Internet search indicated that corresponding author was deceased and could find no email for other) Power calculation: Yes
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Quote: Random sampling techniques were used to select and assign managers and presidents into an experimental group and a control group’
Allocation concealment (selection bias)	Unclear risk	Quote: Random sampling techniques were used to select and assign managers and presidents into an experimental group and a control group’
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Comment: No details of responders or non responders provided.
Selective reporting (reporting bias)	Unclear risk	Comment: No protocol. One outcome in measures and this was reported in results
Other bias	Low risk	Comment: Low risk for lack of evidence for reliability or validity of primary outcomes N/A for difference in baseline measures or participant characteristics in cluster trials with few clusters Low risk for evidence of counter‐discourse in follow‐up period N/A for outcomes between audience members and non‐audience not compared Low risk for other items listed as potential other sources of bias in Higgins 2011 (section 8.15.1)
Blinding of participants and personnel (performance bias) All outcomes	High risk	Comment: Blinding of participants and personnel not possible
Blinding of outcome assessment (detection bias) All outcomes	High risk	Comment: Outcome assessment was by self‐complete questionnaire

Smith 2007.

Methods	Aim of study: To investigate whether differing amounts of exposure to educational advertisements will increase positive attitudes toward help‐seeking, reduce stigma endorsement and increase utilisation propensity Study design: RCT Recruitment of participants: From introductory psychology classes Inclusion/exclusion criteria: Exclusion (after randomisation): (i) those who failed to respond to indicate that they had received any of the adverts (n = 3); (ii) previous contact (personal or in immediate family) with a mental health service provider (n = 71) Informed consent:Yes Ethical approval: Yes Funding: None (a) Consumer involvement: Messages were partially based on information that National Association for Mental Health considers important
Participants	Description: Undergraduate university students in introduction to psychology courses at a medium sized university Geographic location: Midwestern USA Income level of country: High  Setting: Online (email) Number: Total randomised 202 (n per group not stated), excluded post randomisation 74, included in analysis 128 (prejudice 128) Age: range18 to 43; mean (SD) = 19.13 (2.58) Gender: 32 (31.1%), 71 (68.9%) female Ethnicity: 2(1.9%) Asian American; 97 (94.2%) Caucasian; 2(1.9%) Hispanic; 2 (1.9%) multiethnic. Other social/demographic details: 77 (74.8%) Freshman, 15 (14.6%) Sophomore; 8 (7.8%) Junior; 3 (2.9%) Senior
Interventions	Aim of intervention: To increase healthcare seeking and reduce stigma Content of intervention: Intervention A: One email educational advert with a logo for and details of a fictional behavioural health service and six email educational adverts for physical health conditions. Intervention B: three email mental health adverts and four for physical health. Intervention C: five email mental health adverts and two for physical health. Intervention D: seven email mental health adverts. Mental ill health messages included information about prevalence and burden of mental illness, effectiveness of treatments, groups most vulnerable to mental illness and importance of early detection and treatment. Content of control  Physical health email adverts Co‐interventions in all groups: None Delivery: Sent by email twice a week for four weeks, asked to read the message and to reply to notify researcher that the email had been opened and read. Each advert comprised a short paragraph and a fictitious logo and address for a mental health facility Providers: Developed by the researcher for the study Type of mass media used in intervention: Internet (emailed adverts, text and a logo) Number of mass media components: Interventions A, B, C, D: one, three, five and seven components respectively Mass media component combined with non‐mass media component: No Contains personal narrative: No Celebrities involved in intervention: No Fictional portrayals: No Primary message: Mixed Mental health condition: Mental illness
Outcomes	Primary outcome measures: Discrimination: none. Prejudice: Attribution Questionnaire (AQ) Secondary outcome measures: None Methods of assessing outcome measures Online self‐complete questionnaires Validity and reliability of outcome measures: reliability of the overall score (as used in study) not stated, factor analysis has yielded six distinct factors with Cronbach's alphas 0.70, 0.74, 0.89, 0.96, 0.88 and 0.89 Methods of follow‐up for non‐respondents: None stated Timing of outcome assessment: Immediate
Notes	Contact with author: Yes Power calculation: Yes Unpublished: Dissertation
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Quote: ‘Participants were randomly assigned to  a control condition or to one of the four experimental conditions’ Quote: 'A computer generated table of random numbers (1‐5) was used to generate the 5 groups used.  Subjects were assigned to randomly generated groups by going down the sign up lists presented in their introduction to psychology course' (a)
Allocation concealment (selection bias)	High risk	Quote: ‘Participants were randomly assigned to  a control condition or to one of the four experimental conditions’ Quote:'A computer generated table of random numbers (1‐5) was used to generate the 5 groups used.  Subjects were assigned to randomly generated groups by going down the sign up lists presented in their introduction to psychology course' (a)
Incomplete outcome data (attrition bias) All outcomes	High risk	Comment: Post‐randomisation exclusion of, and no data available for, those who had previous contact with mental health services or had someone in their immediate family had had such contact (35%, 71/202)
Selective reporting (reporting bias)	Unclear risk	No protocol. Data reported for all outcomes mentioned in measures section
Other bias	Unclear risk	Comment: Unclear risk for lack of evidence for reliability or validity of primary outcomes as overall reliability of overall score (used in study) was not stated N/A for difference in baseline measures or participant characteristics in cluster trials with few clusters N/A for evidence of counter‐discourse in follow‐up period N/A for outcomes between audience members and non‐audience not compared Low risk for items listed as potential other sources of bias in Higgins 2011 (section 8.15.1)
Blinding of participants and personnel (performance bias) All outcomes	High risk	Comment: Blinding of participants and personnel not possible
Blinding of outcome assessment (detection bias) All outcomes	High risk	Comment: Outcome assessment was by self‐complete questionnaire

Varughese 2010.

Methods	Aim of study: To look at the effect of viewing pictures of dysmorphic people with intellectual disabilities on stigmatised attitudes Study design: RCT (with cross‐over after six months deemed inappropriate by review authors, as washout cannot be assumed, requested separate data from first time point to enable it to be analysed as an RCT, no response) Recruitment: From a research panel of 200 members of the general public using direct mailshots and advertisements in local newspapers set up in a previous study Inclusion/exclusion criteria: None stated Informed consent: Not stated Ethical approval: Not stated Funding: None stated Consumer involvement: None stated
Participants	Description: General public Geographic location: UK (England and Scotland) Income level of country: High Setting: Home (postal intervention)) Number: Total randomised 200 (n per group for randomisation at first time point not stated), included in analysis 0 Age: mean (SE)  47.9 (1.5) Gender: 26% were male Ethnicity: 92% described their ethnic group as White British Other social/demographic details: 55% were in paid employment, 17% were retired
Interventions	Aim of intervention: To reduce stigma concerning people with appearance that indicates intellectual difficulties Content of intervention: Picture of a young man with the physical appearance of Down syndrome wearing a shirt and tie in an office with a cheerful expression (picture was of a model from a commercial photo‐image gallery)  Content of control: No intervention Co‐interventions in all groups: None, however as part of the outcome measure both groups read a vignette which stated ‘This is a fictitious report Oliver has Down syndrome. He is 32 years old and lives with his parents.  He cannot read or write, but he is happy and cheerful and keen to help people' Delivery: Material was sent in the post. Intervention was one image Providers: Selected by the researchers from a commercial photo‐image gallery Type of mass media: Print (photograph) Number of mass media components: One Mass media component combined with non‐mass media component: No Contains personal narrative: No Celebrities involved: No Fictional portrayals:Yes Primary message: Recovery oriented Mental health condition: Intellectual disability
Outcomes	Primary outcome measures: Discrimination: None. Prejudice: Attitudes to Mental Illness Questionnaire (AMIQ) Secondary outcome measures: None Methods of assessing outcome measures: Self‐complete questionnaire Validity and reliability of outcome measures: One component accounts for 80.2% of the variance, test‐retest reliability 0.702; Cronbach alpha was 0.93. Validated on samples with schizophrenia, substance abuse and alcoholism, it is unclear if measure is valid in intellectual disability Methods of follow‐up for non‐respondents: N/A Timing of outcome assessment: Immediate (as cross‐over data were not considered appropriate)
Notes	Contact with author: Sought, unsuccessful Power calculation: None reported
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Quote: ‘Participants were block randomised using the randomisation function of the Stats Direct Statistical Package’
Allocation concealment (selection bias)	Low risk	Comment: Randomisation was undertaken centrally and experimental material sent by post
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Quote: ‘response rate 93% at baseline and 87% at 6 month crossover’. No details of which groups
Selective reporting (reporting bias)	Unclear risk	Comment:  No protocol mentioned.  Study reports that one outcome was assessed and this is reported.
Other bias	Unclear risk	Comment: Unclear risk for lack of evidence for reliability or validity of primary outcomes as measure has been validated on samples with schizophrenia, substance abuse and alcoholism, but it is unclear if measure is valid in intellectual disability  N/A for difference in baseline measures or participant characteristics in cluster trials with few clusters N/A for evidence of counter‐discourse in follow‐up period N/A for outcomes between audience members and non‐audience not compared Low risk for items listed as potential other sources of bias in Higgins 2011 (section 8.15.1)
Blinding of participants and personnel (performance bias) All outcomes	High risk	Comment: Blinding of participants not possible.  No personnel involved in providing the intervention
Blinding of outcome assessment (detection bias) All outcomes	High risk	Comment: Outcome assessment was by self‐complete questionnaire

Woods 2002.

Methods	Aim of study: To determine if an educational video about Tourette’s Syndrome is effective in improving attitudes and behaviour toward persons with Tourette's Syndrome Study design: RCT Recruitment: from introductory psychology courses Inclusion/exclusion criteria: Post‐randomisation exclusion: participant, friend or relative has Tourette's Syndrome Informed consent: Yes Ethical approval: Not stated Funding: None stated Consumer involvement: Intervention film is sold and promoted by National Tourette’s Syndrome Association
Participants	Description: University students studying introductory psychology Geographic location: Wisconsin, USA (assumed from author affiliation) Income level of country: High Setting: University Number: Total randomised: 120 (intervention 60, control 60), excluded post randomisation 8 (6 as they, friend or relative had Tourette's Syndrome, 2 for unusable data), only half of sample were asked to take part in discrimination part of study 60, included in analysis 112 (discrimination 57, 112) Age: mean (SD) 20.5 (4.1) Gender: 26 (22%) men and 94  (78%) women Ethnicity: not stated
Interventions	Aim of intervention: To provide information about Tourette's Syndrome and reduce stigma Content of intervention: Educational video about Tourette's Syndrome, ‘Stop it, I can't’, provides information about the disorder and shows a variety of persons of differing ages with Tourette's Syndrome discussing their experiences with the syndrome Content of control:  No intervention Co‐interventions in all groups: None. As stimulus for outcome measure, all viewed video still of actor in film but not exhibiting any tics and a two‐ minute videotape of adult actor or actress engaging in motor tics (arm jerking) and vocal tics (grunting) while standing in line to buy tickets for an event (rated as realistic by mental health professionals) Delivery: Duration of intervention 13 minutes Providers: Seligman and James Stanfield Film Associates Type of mass media used in intervention: Audiovisual (video) Number of mass media components in intervention: One Mass media component combined with non‐mass media component in intervention: No Contains personal narrative: Yes  Celebrities involved: No Fictional portrayals: No Primary message: 'Not to blame' Mental health condition: Tourette’s syndrome
Outcomes	Primary outcome measures: Discrimination: Social Proximity Measure (SPM) developed for present study. Prejudice: Social Acceptance Scale (SAS) Secondary outcome measures: None Methods of assessing outcome measures: Measurement of distance between chairs and self‐complete questionnaires Validity and reliability of outcome measures: SPM, no psychometrics provided. SAS: Stated that previous research has shown it to be internally consistent, Cronbach's alpha = 0.88 (one reference) has good predictive validity in job hiring situations, good concurrent validity, and test‐retest reliability of r = 0.87 when evaluating people with tic disorders (one reference); Methods of follow‐up for non‐respondents: N/A Timing of outcome assessment: Immediate
Notes	Contact with author: Sought, unsuccessful Power calculation: None stated
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Quote: ‘Participants were then randomised to one of four groups’ Comment: in each of the two experimental and control groups participants saw a different gender of actor in the stimulus video, no gender effect was found and so the 2 experimental and the 2 control groups were collapsed
Allocation concealment (selection bias)	Unclear risk	Quote: ‘Participants were then randomised to one of four groups’
Incomplete outcome data (attrition bias) All outcomes	High risk	Comment: There was significant incomplete data for the discrimination measure as this was only assessed on the second half of the sample
Selective reporting (reporting bias)	Unclear risk	Comment:  No protocol mentioned.  All outcomes mentioned in measures section are reported in results section
Other bias	Unclear risk	Comment: Unclear risk for lack of evidence for reliability or validity of primary outcomes as no psychometrics provided for SPM N/A for difference in baseline measures or participant characteristics in cluster trials with few clusters N/A for evidence of counter‐discourse in follow‐up period N/A for outcomes between audience members and non‐audience not compared Low risk for items listed as potential other sources of bias in Higgins 2011 (section 8.15.1)
Blinding of participants and personnel (performance bias) All outcomes	High risk	Comment: Blinding of participants and intervention providers not possible
Blinding of outcome assessment (detection bias) All outcomes	High risk	Comment: Unclear risk for blinding in relation to discrimination outcome. SPM involved measuring between chairs, there was no statement that person measuring was blind to group allocation and the measurement could have been influenced by knowing group allocation High risk for blinding in relation to prejudice outcome as assessed by self‐complete questionnaire

Woods 2003.

Methods	Aim of study: To compare the effects of education versus no education on the social acceptability of persons who exhibit severe versus mild tics Study design: RCT Recruitment: from introductory psychology courses Inclusion/exclusion criteria: Exclusion: participant, friend or relative has Tourette's Syndrome Informed consent: Yes Ethical approval: Not stated Funding: None stated Consumer involvement: Intervention film is sold and promoted by National Tourette’s Syndrome Association
Participants	Description: University students studying introductory psychology Geographic location: Wisconsin, USA (assumed from author affiliation) Income level of country: High Setting: University Number: Total randomised 240 (n randomised to each group not stated), excluded post randomisation 17 (as they, friend or relative had Tourette's Syndrome), included in analysis 223, included for each outcome 223 (discrimination 223, prejudice 223)   Age: Not stated Gender: 75 (31%) men and 165 (69%) women Ethnicity: Not stated
Interventions	Aim of intervention: To provide information about Tourette's Syndrome and reduce stigma Content of intervention: Educational video about Tourette's Syndrome, ‘Stop it, I can’t’, provides information about the disorder and shows a variety of persons of differing ages with Tourette's Syndrome discussing their experiences with the syndrome Content of control:  No intervention Co‐interventions in all groups: None. As stimulus for outcome measure, all viewed video still of actor in film but not exhibiting any tics and a two‐ minute videotape of adult actor or actress engaging in motor tics (arm jerking) and vocal tics (grunting) while standing in line to buy tickets for an event (rated as realistic by mental health professionals) Delivery: Duration of intervention 13 minutes Providers: Seligman and James Stanfield Film Associates Type of mass media used in intervention: Audiovisual (video) Number of mass media components in intervention: One Mass media component combined with non‐mass media component in intervention: No Contains personal narrative: Yes  Celebrities involved: No Fictional portrayals: No Primary message: 'Not to blame' Mental health condition: Tourette’s syndrome
Outcomes	Primary outcome measures: Discrimination: Social Proximity Measure (SPM). Prejudice: Social Acceptance Scale (SAS); Tolerance Scale (Tourette's Syndrome) Secondary outcome measures: None Methods of assessing outcome measures: Measurement of distance between chairs and self‐complete questionnaires Validity and reliability of outcome measures: SPM, no psychometrics provided. SAS: Stated that previous research has shown it to be internally consistent, Cronbach's alpha = 0.88 (one reference) has good predictive validity in job hiring situations, good concurrent validity, and test‐retest reliability of r = 0.87 when evaluating people with tic disorders (one reference). Tourette's Syndrome cited as having excellent test‐retest reliability r = 0.96 and good discriminant validity (one reference) and concurrent validity (one reference) Methods of follow‐up for non‐respondents: N/A Timing of outcome assessment: Immediate
Notes	Contact with author: Sought, unsuccessful Power calculation: None stated
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Quote: ‘Participants were then randomised to one of eight groups’ Comment: Findings are collapsed for experimental and control groups participants across gender of actor and tic severity in the stimulus video for two of the outcomes, but not for final one as interaction found
Allocation concealment (selection bias)	Unclear risk	Quote: ‘Participants were then randomised to one of eight groups’
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Comment: Numbers lost to follow‐up not clear
Selective reporting (reporting bias)	Unclear risk	Comment:  No protocol mentioned.  All outcomes mentioned in measures section are reported in results section
Other bias	Unclear risk	Comment: Unclear risk for lack of evidence for reliability or validity of primary outcomes as no psychometrics provided for SPM N/A for difference in baseline measures or participant characteristics in cluster trials with few clusters N/A for evidence of counter‐discourse in follow‐up period N/A for outcomes between audience members and non‐audience not compared Low risk for items listed as potential other sources of bias in Higgins 2011 (section 8.15.1)
Blinding of participants and personnel (performance bias) All outcomes	High risk	Comment: Blinding of participants and intervention providers not possible.
Blinding of outcome assessment (detection bias) All outcomes	High risk	Comment: Unclear risk for blinding in relation to discrimination outcome. SPM involved measuring between chairs, there was no statement that person measuring was blind to group allocation and the measurement could have been influenced by knowing group allocation High risk for blinding in relation to prejudice outcome as assessed by self‐complete questionnaire

Woods 2005.

Methods	Aim of study: To clarify the effectiveness of video‐based education by determining whether education about any mental health condition has an effect on attitudes toward persons with Tourette's Syndrome or wether the education needs to be specific to Tourette's Syndrome Study design: RCT Recruitment: From introductory psychology courses Inclusion/exclusion criteria: Exclusion: friend or relative has Tourette's Syndrome Informed consent: Yes Ethical approval: Not stated Funding: None stated Consumer involvement: Intervention film is sold and promoted by National Tourette’s Syndrome Association
Participants	Description: University students studying introductory psychology Geographic location: Wisconsin, USA (assumed from author affiliation) Income level of country: High Setting: University Number: Total randomised 180 in total (n randomised to each group not stated), randomised to relevant groups 120 (assumed), excluded post‐randomisation (from all groups as they had a friend or relative with Tourette's Syndrome) 10, included in analysis 114 (discrimination 114, prejudice 113) Age: mean (SD) 22.33 (5.89) Gender: 35 (19%) men and 135 (81%) women Ethnicity: not stated
Interventions	Aim of intervention To provide information about Tourette's Syndrome and reduce stigma Content of intervention: Educational video about Tourette's Syndrome, ‘Stop it, I can’t’, provides information about the disorder and shows a variety of persons of differing ages with Tourette's Syndrome discussing their experiences with the syndrome Content of control:  No intervention Co‐interventions in all groups: None. As stimulus for outcome measure, all viewed video still of actor in film but not exhibiting any tics and a two‐ minute videotape of adult actor or actress engaging in motor tics (arm jerking) and vocal tics (grunting) while standing in line to buy tickets for an event (rated as realistic by mental health professionals) Delivery: Duration of intervention 13 minutes Providers: Seligman and James Stanfield Film Associates Type of mass media used in intervention: Audiovisual (video) Number of mass media components in intervention: One Mass media component combined with non‐mass media component in intervention: No Contains personal narrative: Yes  Celebrities involved: No Fictional portrayals: No Primary message: 'Not to blame' Mental health condition: Tourette’s syndrome
Outcomes	Primary outcome measures: Discrimination: Social Proximity Measure (SPM). Prejudice: Social Acceptance Scale (SAS); Tolerance Scale (Tourette's Syndrome) Secondary outcome measures: None Methods of assessing outcome measures: Measurement of distance between chairs and self‐complete questionnaires Validity and reliability of outcome measures: SPM, no psychometrics provided. SAS: Stated that previous research has shown it to be internally consistent, Cronbach's alpha = 0.88 (one reference) has good predictive validity in job hiring situations, good concurrent validity, and test‐retest reliability of r = 0.87 when evaluating people with tic disorders (one reference). Tourette's Syndrome cited as having excellent test‐retest reliability r = 0.96 and good discriminant validity (one reference) and concurrent validity (one reference) Methods of follow‐up for non‐respondents: N/A Timing of outcome assessment: Immediate
Notes	Contact with author: Sought, unsuccessful Power calculation: None stated
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Quote: ‘Participants were then randomised to one of six groups’ Comment: Findings are collapsed for experimental and control groups participants across gender of actor as no difference found. Two groups (depression condition) were not relevant to this review
Allocation concealment (selection bias)	Unclear risk	Quote: ‘Participants were then randomised to one of six groups’
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Comment: numbers lost to follow‐up are not clear
Selective reporting (reporting bias)	Unclear risk	Comment: No protocol mentioned.  All outcomes mentioned in measures section are reported in results section
Other bias	Unclear risk	Comment: Unclear risk for lack of evidence for reliability or validity of primary outcomes as no psychometrics provided for SPM N/A for difference in baseline measures or participant characteristics in cluster trials with few clusters N/A for evidence of counter‐discourse in follow‐up period N/A for outcomes between audience members and non‐audience not compared Low risk for items listed as potential other sources of bias in Higgins 2011 (section 8.15.1)
Blinding of participants and personnel (performance bias) All outcomes	High risk	Comment: Blinding of participants and intervention providers not possible
Blinding of outcome assessment (detection bias) All outcomes	High risk	Comment: Unclear risk for blinding in relation to discrimination outcome. SPM involved measuring between chairs, there was no statement that person measuring was blind to group allocation and the measurement could have been influenced by knowing group allocation High risk for blinding in relation to prejudice outcome as assessed by self‐complete questionnaire

Yoshida 2002.

Methods	Aim of study: To assess the effects of a public educational campaign on improving knowledge of and attitudes towards people with mental illness Study design: Cluster RCT Size and description of clusters: 10 cluster arms in intervention group, and 10 cluster arms in control group, each cluster being the geographical area around a sheltered workshop. Data are reported for 19 clusters with an average cluster size of 78 Recruitment: 20 small‐scale sheltered workshops for people with mental illness in two prefectures were randomly assigned to intervention (10 cluster arms) and control (10 cluster arms). In order to recruit participants in each cluster arm, random samplings from the voter registration list and the basic resident register were conducted before and after the intervention respectively. The study did not have same people before and after the intervention and only the post‐intervention sample is considered in this review Inclusion/exclusion criteria: Inclusion: People who lived near 20 sheltered workshops in two prefectures, and who were registered on the voter registration list and basic resident register. Informed consent: Yes  Ethical approval: Yes (a)   Funding: The Nippon Foundation (No. 21990) Consumer involvement: None stated
Participants	Description: General adult population on voter registration lists Geographic location: East Japan Income level of country: High Setting: Communities containing small‐scale sheltered workshops in two prefectures Number: Intervention group area: Eligible (given questionnaire) 418, responded 331; Control group area: eligible 483, responded 408; included in analysis: 732 (discrimination 721 prejudice 732, knowledge 692, audience reactions 321) Age: mean (SD) intervention group: 48.9 (14.0); control group: 50.0 (14.8) Gender: In intervention group: 61.8% female; control group: 52.3% female Ethnicity: Not stated
Interventions	Aim of intervention:To change Images of and attitudes towards people with mental health problems, and to improve knowledge of mental health services Content of intervention: Three small booklets, containing a comic which shows a story of recovery for a father with depression and a list of mental health services. The first booklet gives an introduction about people with mental illness and medical/social services. The second one gives details about small‐scale sheltered workshops. The third one shows the care of people with mental health problems and information about volunteer activities. Also each small‐scale sheltered workshop in intervention group had a community event to provide social contact with people with mental illness and hear their experiences, provide information about the and mental health services (2.4% of people participated in the community events) Content of control: No intervention Co‐interventions in all groups: None Delivery of intervention: Three small booklets were handed out to people who lived in the areas of intervention groups by staff from the small‐scale sheltered workshops, local government staff and members in neighbourhood associations. Providers: Staff from small‐scale sheltered workshops, local government staff and members in neighbourhood associations Type of mass media: Print (booklets) Number of mass media components in intervention: Three Mass media component combined with non‐mass media component in intervention: Yes Contains personal narrative: Yes (third‐person) Celebrities involved: No Fictional portrayals: Yes (a) Primary message: Recovery‐oriented Mental health condition: No specific mental health problem (a)
Outcomes	Primary outcome measures: Discrimination: self‐report of number of visits to small‐scale sheltered workshops (none, one or two times, or several times) (dichotomised as none/any by review authors) (VW); self‐report of any contact with people with mental illness (AC). Prejudice: Negative Attitudes Scale (NAS), selected items; Social Distance Scale (SDS) (selected items) Secondary outcome measures: Knowledge: Knowledge of Mental Illness scale, developed for study; Audience reactions: Three items developed by study authors Measure(s) selected for analysis if multiple measures per outcome: Discrimination VW (as defined by authors as primary discrimination measure) (a); prejudice: NAS (following Brennan 2009) Methods of assessing outcome measures: Self‐complete questionnaires Validity and reliability of outcome measures: VW and AC: single self‐report items developed by authors, validity and reliability of measures were unclear; NAS: scale was developed from the Opinion about Mental Illness by Japanese researchers. However its statistical validity and reliability for a Japanese sample are unclear, and the study used selected items a part of this scale, rather than the whole scale; SDS: scale developed from the Social Distance Scale developed by a Japanese researcher. This is a statistically validated measure, but reliability (test‐retest reliability) is unclear. Also, this study used a part of this scale, rather than the whole scale Methods of follow‐up for non‐respondents: Researchers visited the home of people who did not response to the questionnaires in several times (a) Timing of outcome assessment: 9 months
Notes	Contact with author: Yes, review author (SY) met with the study author on two occasions to obtain further information and data Power calculation? None Unpublished: Dissertation Data: Raw data provided by author and re‐analysed to account for clustering by review team statistician Translation: No (data were extracted by native Japanese speaker from review team)
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Quote: '10 small‐scale sheltered workshops were recruited in two prefectures independently, then half of them were randomly allocated to the intervention group which performed educational campaign, and rest of them were also assigned to control group.' Quote 'The 20 small scale sheltered workshops were randomly allocated into the intervention and control groups, using a random number table' (author via SY).
Allocation concealment (selection bias)	Low risk	Comment: Participants were randomly selected, using the voter registration list and the basic resident register of the areas which had the 10 small‐scale sheltered workshops. Japanese review author (SY) discussed with study author and reported that the author believes that there were no problems for allocation concealment.
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Comment: There was a differential response rate at the post‐intervention follow‐up point of 73.6% (intervention) vs 81.6% (control)
Selective reporting (reporting bias)	Low risk	Comment: In the dissertation the methods section list a discrimination outcome but this is not reported in the results section, however these data have subsequently been provided by the author.
Other bias	High risk	Comment: High risk for baseline imbalance as the sex, age, and educational qualification proportions differed between intervention and control groups. Unclear risk for lack of evidence for reliability or validity of primary outcomes as the discrimination measure was devised by the authors and no evidence on reliability or validity is presented. The two prejudice measures were adapted scales using only part of the original validated scales and for one (NAS) Cronbach's alpha was 0.67 High risk for evidence of counter‐discourse during follow‐up period as two serious crimes (bus hijacking and kidnap followed by long‐term imprisonment) were committed by people reported to have mental illness during the study period. They were nationally reported although it is possible that they may have had a differential effect of those in the two study arm Low risk for outcomes between audience members and non‐audience not compared Low risk for items listed as potential other sources of bias in Higgins 2011 (section 8.15.1)
Blinding of participants and personnel (performance bias) All outcomes	High risk	Comment: Blinding of participants and intervention providers not possible
Blinding of outcome assessment (detection bias) All outcomes	High risk	Comment: Discrimination outcome: Outcome assessment was by self‐complete questionnaire Prejudice outcome: Outcome assessment was by self‐complete questionnaire

(a) indicates information provided by study author

Characteristics of excluded studies [ordered by study ID]

Study	Reason for exclusion
Althaus 2002	Not RCT or ITS study
Altindag 2006	Mass media component not >50%
Andrewes 1995	No discrimination or prejudice outcomes
Arrillaga 1993	Not RCT or ITS study
Bailey 2001	Intervention not mass media
Balaji 2011	Mass media component not >50%
Barber 1990	Not RCT or ITS study
Battaglia 1990	Not RCT or ITS study
Bayar 2009	Not RCT or ITS study
Bennett 2001	Mass media component not >50%
Bennett 2002	No discrimination or prejudice outcomes
Bhugra 1997	Not RCT or ITS study
Billings 2008	No discrimination or prejudice outcomes
Binney 1992	Intervention not mass media
Bock 2003	Not RCT or ITS study
Boysen 2008	No inactive control
Buckley 2005	No inactive control
Campbell 2004	No inactive control
Campbell 2011	Intervention not mass media
Chan 2009	No inactive control
Chang 2008	No inactive control
Chung 2005	Intervention not mass media
Commons Treloar 2008	Not RCT or ITS study
Corrigan 2001	Intervention not mass media
Corrigan 2002a	Intervention not mass media
Corrigan 2004	Intervention not mass media
Corrigan 2007	No inactive control
Cross 2010	No discrimination or prejudice outcomes
Dietrich 2006	No inactive control
Dipaula 2011	Mass media component not >50%
Epstein 1995	Not RCT or ITS study
Esters 1998	Not RCT or ITS study
Ewers 2002	Intervention not mass media
Faigin 2008	Not RCT or ITS study
Fauteux 2008	No inactive control
Friedrich 1996	No inactive control
Galletly 2011	Not RCT or ITS study
Godeau 2010	Mass media component not >50%
Goldney 2008	Not RCT or ITS study
Griffiths 2004	No inactive control
Hayes 2004	Intervention not mass media
Hegerl 2003	Not RCT or ITS study
Holtz 2006	No discrimination or prejudice outcomes
Jorm 2003	No inactive control
Jorm 2004	Intervention not mass media
Jorm 2010b	Mass media component not >50%
Kalyanaraman 2010	Intervention not mass media
Kennedy 1995	Not RCT or ITS study
Kim 2010	No inactive control
Kim 2011	No inactive control
Kitchener 2004	Intervention not mass media
Kitchener 2008	Not RCT or ITS study
Ladoucer 2005	Not RCT or ITS study
Lam 2005	No inactive control
Larkin 2000	Not RCT or ITS study
Lincoln 2008	Not RCT or ITS study
Livni 1994	Mass media component not >50%
Locke 2010	About media reporting of violent acts
Luty 2007	No inactive control
Luty 2008	No inactive control
Lüllmann 2011	No discrimination or prejudice outcomes
Mandrusiak 2006	Not RCT or ITS study
Martielli 2006	No discrimination or prejudice outcomes
Masuda 2007	Intervention not mass media
Mehta 2009	ITS study not meeting additional criteria
Merritt 2007	No discrimination or prejudice outcomes
O'Kearney 2006	Not RCT or ITS study
O'Kearney 2009	Not RCT or ITS study
Owen 2007	No inactive control
Penn 2002	No inactive control
Pinto‐Foltz 2011	Intervention not mass media
Pitre 2007	Intervention not mass media
Rahman 1998	Mass media component not >50%
Ritterfeld 2006	No inactive control
Romer 2008	No inactive control
Sadow 2008	Not RCT or ITS study
Saecker 2010	No inactive control
Schulze 2003	Not RCT or ITS study
Seo 2010	Not RCT or ITS study
Shabsigh 2009	Intervention not about mental health
Sharp 2006	Intervention not mass media
Shera 1996	Not RCT or ITS study
Sheridan 1994	Not RCT or ITS study
Silberman 1993	Not RCT or ITS study
Silton 2010	No inactive control
Sogaard 1995	Not RCT or ITS study
Stuart 2006a	Not RCT or ITS study
Sun 2011	Intervention not mass media
Swaim 2001	No inactive control
Teng 2009	Intervention not mass media
Thornton 1996	Not RCT or ITS study
Tolomiczenko 2001	No inactive control
Tsutsumi 2005	Intervention not mass media
Ucok 2006	Not RCT or ITS study
Varughese 2011	No inactive control
Vaughan 2004	Not RCT or ITS study
Wahl 1989	About media reporting of violent acts
Walachowska 2009	No inactive control
Walker 2002	No inactive control
Walker 2005	Not RCT or ITS study
Wall 2004	Not RCT or ITS study
Winkler 2006	Not RCT or ITS study
Wong 2000	Not RCT or ITS study
Wong 2007	Mass media component not >50%
Wood 2006	No inactive control
Worakul 2007	Not RCT or ITS study
Wright 2006	Not RCT or ITS study
Yamamoto 1997	No discrimination or prejudice outcomes

Differences between protocol and review

The protocol for this review is Clement 2011.

Types of outcome measure

We had originally planned to include discriminatory behavioural intentions under discrimination (stigmatising behaviour), but elected to treat these as under prejudice (stigmatising attitudes) as an intention is more akin to an attitude than a behaviour.

Data extraction and management

A post‐hoc decision was made not to use the 'see the person' message type, as this message type only arose when interventions contained personal narratives and if we had categorised these as having a 'see the person' primary message we would have missed messages contained in what the narrators said (or other aspects of the intervention). We also decided post‐hoc to include commonly‐used categories of primary message that were not in Clement 2010. We had not pre‐specified the method of deciding which message was primary and decided this would be undertaken independently by two authors who would resolve disparities by discussion, and with arbitration if necessary.

We had not specified methods for classifying levels of risk for outcome measures in our protocol, therefore, through discussion (SC EB, SEL and FL), we established the following pragmatic criteria. We rated as high risk: measures developed by the study authors with no psychometric data reported; measures for which the authors reported a Cronbach's alpha of < 0.7; and un‐referenced measures. We rated as 'unclear': referenced measures with no psychometric data reported; referenced measures with no statement that the measure was reliable or valid; and validated measures being used for the first time in a different type of population. We rated as low risk: measures which study authors reported had a Cronbach's alpha of 0.7 or greater, or referenced the measure as being reliable or valid.

Measures of treatment effect

Standardised mean differences rather than mean differences were calculated, as different measures were used for the same outcome and this precludes the use of mean differences. We did not pre‐specify actions if data were skewed data. When this was the case the data were transformed into the logarithmic scale using methods described by Higgins and colleagues (Higgins 2008).

Unit of analysis issues

It was clarified that where intervention arms fell into different subgroups each intervention arm was compared to the control group and the possibility of meta‐analysis was only considered within each subgroup, thereby avoiding potential unit‐of‐analysis errors. We had not made an a priori plan for dealing with studies with two control groups. When this arose we selected the one that was most similar to the intervention, that is an intervention containing irrelevant material rather than a no intervention control.

Data synthesis

Where there was an even number of outcomes, we made a post‐hoc decision, after consultation with the Cochrane Consumers and Communication Review Group, to follow Brennan 2009 and to select the outcome with the n/2 ranked effect size (using data from the final follow‐up point when there were two or more follow‐up points). A post‐hoc decision was also needed about which outcome to select when multiple outcomes were used in studies with median data. In these cases, on the advice of UK Cochrane Centre training staff, we used an adapted version of the methods proposed by Grimshaw 2003 and Brennan 2009 whereby, after checking that the interquartile ranges were similar, we examined medians at the latest time point and selected the one ranked (n+1)/2 when there was an odd number of outcomes and the one ranked n/2 when there was an even number. Because standardised mean differences rather than mean differences were used, baseline differences are not reported. The review author group discussion to decide about the appropriateness of meta‐analysis was originally planned as a face‐to‐face meeting but altered to an email discussion for practical reasons, and given that for the vast majority of comparisons meta‐analysis was precluded on statistical grounds. We planned to produce a 'Summary of findings' table using GRADEprofiler (GRADEpro) software. In the event the 'Summary of findings' table was produced using the template in RevMan, but still following the GRADE approach (Guyatt 2008).

Subgroup analysis

The main analysis became, in effect, a subgroup analysis by timing of outcome, as we had not anticipated the issue of differential follow‐up time points. This approach was undertaken because it was not appropriate to combine outcomes assessed immediately post‐intervention with those assessed at six months or longer, and because issues of multiplicity would have arisen for studies with more than one follow‐up time point, had we not separated the data by follow‐up time point. In the remaining subgroup analyses we selected the earliest follow‐up time point as the one to present data for. We found that there were two unanticipated types of intervention: simulated audio‐recordings, and interventions containing third‐person narratives. As we considered each of these to be sufficiently distinct from the groups already listed in our subgroup analysis, these groups were added into the analyses post‐hoc. As no mobile phone, broadcast media or cinema interventions were found, we did not refer to these and they did not appear in the 'type of media' subgroup analysis. There was just one intervention ‐ a CD‐ROM ‐ that fell in the 'other' category for media type and a decision was made to group this in the Internet category as Internet‐delivery would not have materially changed participants' experience of the intervention.

Sensitivity analysis

As minimal meta‐analysis was warranted we examined the effects of removing studies at risk of bias and less precise studies primarily through examining changes in median effect sizes. We had intended to test for small study effects of binary outcomes by performing the arcsine‐Thompson test, but this was precluded because we found only two studies with binary primary outcomes, and these had very different timings of outcome (immediate and 9 months). Where reporting bias was discovered we planned to investigate the impact in a sensitivity analysis, but this did not prove possible (see Effects of interventions), as the only studies for which meta‐analysis was possible had identical bias. As we found that three of the multi‐arm studies included arms that the study authors considered unlikely to reduce stigma (Reinke 2004; Brown 2010; Corrigan (submitted)), we undertook a post‐hoc sensitivity analysis to examine the effects of removing these studies.

Quality of the evidence

We had pre‐specified in the protocol the main outcomes for assessment of the quality of the evidence, but had not specified which would be categorised as critically important and which as important. We made a post‐hoc decision to categorise discrimination as a critically important outcome and the remainder as important.

Contributions of authors

Sarah Clement wrote the protocol (Clement 2011), with Paul Williams writing some sections and Sara Evans‐Lacko co‐writing some sections. All authors contributed ideas to the protocol and critically revised it. Sarah Clement produced an initial draft MEDLINE search strategy which was developed further and finalised by John Kis‐Rigo (Cochrane Consumers and Communication Review Group).

Sarah Clement supervised the adaptation of the MEDLINE search strategy for the other English language databases and Sosei Yamaguchi adapted it for the Japanese database Ichushi. Francesca Lassman ran some of the database searches and undertook the non‐database searches. Francesca Lassman was the primary screener for the English language databases and Sosei Yamaguchi for the Japanese database. Sarah Clement developed the manuals for inclusion decisions and data extraction and the Data Extraction Template, and these were piloted with Sara Evans‐Lacko, Elizabeth Barley, Paul Williams and Francesca Lassman. Sarah Clement made inclusion decisions and extracted data for all English articles and Sosei Yamaguchi did this for the Japanese articles. Elizabeth Barley and Sara Evans‐Lacko shared the role of second author for inclusion decisions and data extraction, with Francesca Lassman also contributing. Nicolas Rüsch made inclusion decisions on German language articles. Paul Williams and Sarah Clement undertook the analyses, with Paul providing statistical advice and guidance and performing the complex analyses. Sarah Clement drafted the review with major assistance from Francesca Lassman. Nicolas Rüsch, Mike Slade and Graham Thornicroft provided information, advice and support at intervals throughout the review process and assisted in interpreting the review findings and drawing implications from them; and in addition Graham Thornicroft provided supervision to Sarah Clement. All authors critically revised the review.

Sources of support

Internal sources

• No sources of support supplied

External sources

• This publication is independent research funded by the National Institute for Health Research under its Programme Grants for Applied Research scheme (Improving Mental Health Outcomes by Reducing Stigma and Discrimination: RP‐PG‐0606‐1053)., UK.

  The Programme grant supports the salary of SC, FL and contributes to the time of PW, GT and MS.  The views expressed in this publication are those of the authors and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health.

• NIHR Specialist Mental Health Biomedical Research Centre at the Institute of Psychiatry, Kings College London and the South London and Maudsley NHS Foundation Trust, UK.

  This contributes to the time of GT.

• Big Lottery and Comic Relief grant for Evaluation of the Time to Change Programme, UK.

  This supports the salary of SEL.

Declarations of interest

Review authors have been involved in some studies on the effectiveness of mass media interventions in reducing mental health‐related stigma. Graham Thornicroft leads the independent evaluation team for England's national ‘Time to Change: Let’s end mental health discrimination now’ programme (Henderson 2009) and has received research funding for this. Sara Evans‐Lacko is employed as a researcher on this evaluation team.  Graham Thornicroft has been a member of the independent evaluation team for Scotland’s ‘See Me’ campaign (Dunion 2005) and received research funding for this.  Both Time to Change and See Me are multifaceted initiatives with mass media components. Graham Thornicroft is chief investigator and Sarah Clement is study lead for a randomised controlled trial on the effectiveness of an anti‐stigma DVD (Clement 2012). Graham Thornicroft received research funding for this study from the England's National Institute for Health Research and Sarah Clement is employed on this study. However, none of these studies proved eligible for the review.

Elizabeth Barley, Francesca Lassman, Nicolas Rüsch, Mike Slade, Paul Williams, Sosei Yamaguchi have no known potential conflicts of interest.

New