Abstract
The negative health effects of racism have been well documented, but how to intervene to redress these effects has been little studied. This study reports on RISE (Racism Still Exists), a high-risk, high-reward public health intervention that used outdoor advertising to disseminate a “countermarketing” campaign in New York City (NYC). Over 6 months, the campaign advertised stark facts about the persistence of racism in the USA. A probability sample of N = 144 participants from two predominantly Black NYC neighborhoods completed measures of health status, health behaviors, and social attitudes. Three months postintervention, statistically significant declines in psychological distress were seen among study participants who were exposed to the campaign compared to those who were not. There were no changes in other hypothesized outcomes. The campaign also generated significant public discourse, particularly in social media. The results suggest that racism countermarketing campaigns may have promise as a community-based intervention to address health inequalities.
Keywords: African American/Black, Neighborhoods, Racism, Outdoor advertising, Countermarketing, Public health intervention
How are the negative effects of racism on African American health to be redressed? A substantial literature shows that racism takes a significant toll on health.1–5 More than discrete interpersonal acts, racism inheres in pervasive subordinating messages that degrade, delegitimize, and dehumanize blackness; and in institutional policies, law, social organization, and the allocation of resources and opportunities. In other words, racism is part and parcel of the business of daily life,6 deeply pervading the social world in the USA, and doing so at the cost of African American health and well-being.
But in the USA, racism is frequently unseen or denied, and most people are ignorant of social inequalities and the means by which they are perpetuated. For example, two thirds of Americans believe that school desegregation has worked, when American schools have long been resegregating, with Black-White dissimilarity rates among primary school children has high as 85 % in Rust Belt cities such as Detroit.7 Because American culture strongly promotes the idea that individuals are responsible for their life station, many believe that inequalities are justified by individual expenditures of effort. Indeed, system justification theory holds that people tend to be invested in justifying the system of which they are a part—and must do so if the system is to survive.8 Therefore, the populace perceives White privilege as the natural order of things, and overlooked are the social arrangements that confer unfair advantages to Whites.
Consequently, individuals who are the targets of racism may internalize it, and come to believe negative messages about their own abilities and intrinsic worth. Internalizing racism results in accepting limits to self-determination and self-expression,9 and may stem in part from failing to recognize racism as such. That is, people who believe unfair treatment is deserved may not report it as discrimination. Indeed, in some research, a seemingly paradoxical finding showed that working class Black women and men who denied experiencing discrimination had higher blood pressure than those who reported discrimination.10 Others reported similar relationships related to health outcomes among a sample of gay men.11
It would appear that acknowledging and expressing the existence of racism could counter these negative outcomes. If the stress of day-to-day experiences with racism is analogous to continually stubbing one’s toe on an enormous steamer trunk, what is a reasonable course of action if we cannot move the trunk quickly enough to prevent more injuries? A viable strategy must remedy not only the physical pain of a stubbed toe, but also the social pain wrought by cultural narratives that stridently argue that in fact no such trunk exists. And if the trunk has been carefully crafted so as to blend in unobtrusively with the rest of the room décor, most people will fail to perceive how large it looms. One solution would be to put on broad display the trunk’s contours. Rather than fading into the background where it may continue to be overlooked or denied, public statements about the trunk’s unwieldy presence would equip those at risk of stubbing their toes. For those who are already hurt, unequivocal proclamations of the trunk’s role in their pain would create a much-needed affirmation and bolster coping mechanisms. As well, individuals who are inclined to deny that the trunk comprises a real obstacle may be motivated to reconsider that position.
Bringing the analogy back to racism in the African American experience, a potential intervention to reduce negative health effects of racism would be to pointedly declare the nature of racism and inequality in the USA. Doing so would acknowledge structural inequalities deeply felt by Black folks, and could increase individual and community awareness, coping and resilience. The optimal scale for such an intervention would reach large communities of people (e.g., at the neighborhood level) who are subjected to racism, rather than attempting to intervene on an individual basis. Although it may be argued that health benefits could be derived from reducing the endorsement of racist beliefs among Whites, research has shown that there is scant evidence on the utility of targeting Whites for prejudice reduction.12 Moreover, focusing on Whites does nothing to address the pain racism inflicts on Black lives.
In this paper, I report on a novel community-based intervention that used outdoor advertising to stimulate public discourse about racism. Outdoor advertising has broad reach in Black neighborhoods, and is often used to promote health-negating messages and products such as alcohol consumption.13–15 More broadly, scarce and subordinating depictions of Black people in the media are a common point of contention, making the use of outdoor media particularly powerful as a site for the interrogation of racism. Industry groups contend that placing outdoor ads where people work, shop, travel, and play provides “bold messages that make them stop, notice and respond”.16 It is precisely this sort of impact that is needed to counteract the stress of racism.
Health researchers also argue that billboards are cost effective, have repetitive value, and visual prominence.17 Though few have used billboards, health campaigns have used mass-reach media to encourage discrete health behaviors such as physical activity.18,19 In targeting the health effects of racism, there are no health behaviors to encourage. Instead, an appropriate implementation would be to disseminate ideas; messages that ask people to see the world differently may be taken up in multiple ways to benefit well-being. One such example is a youth-targeted tobacco intervention employing countermarketing. “Truth” (see thetruth.com) is a campaign (television, street demonstration, and internet) designed to reduce smoking not with exhortations to avoid cigarettes, but with the delivery of stark facts about tobacco and industry practices. Smoking depends on widespread attractive social portrayals of tobacco, and in the Truth campaign, replacing these with facts about industry policies and health consequences prompted a 22 % decline in youth smoking and initiation.20 Counter ads such as these are thought to be effective because they define the problem environmentally, partially reassigning responsibility for health outcomes to institutional action that shapes the social and physical environment.21
As part of The Black LIFE Study, a project investigating the health effects of racism, I assessed whether a countermarketing campaign designed to deliver stark facts about racism would mitigate against deleterious health outcomes. Entitled RISE (Racism Still Exists), the campaign was designed as a high-risk, high-reward community-level intervention, and used standard outdoor advertising media to disseminate a 6-month campaign about the persistence of racism in the USA. The study is, to my knowledge, the first empirical investigation of a racism countermarketing campaign.
Method
Setting
The Black Life Study (BLS) took place New York City (NYC), the most populous city in the USA. The BLS took place in two of NYC’s largest predominantly Black neighborhoods, Brooklyn’s Bedford-Stuyvesant (intervention) and Manhattan’s Central Harlem (control) shown circled in Fig. 1. Bedford-Stuyvesant is one of New York City’s largest Black neighborhoods (152,985 residents), though the percentage of Black residents has declined over the years. Manhattan’s Central Harlem is recognized nationwide as an historic Black capital with a significant history of arts, culture, and political activism. Like Bedford-Stuyvesant, gentrification has increased the number of affluent and White households. The two neighborhoods are also similar with regard to land use characteristics and architecture, making them well matched to serve as comparison groups. Assigning Bedford-Stuyvesant to intervention was not random because logistical constraints (e.g., cost of advertising in each area) were important. Because the two areas are distant spatially, spillover effects were minimized.
FIG. 1.
The Black Life Study (BLS) was conducted in two of NYC’s largest predominantly Black neighborhoods (2000 Census data), Brooklyn’s Bedford-Stuyvesant (intervention) and Manhattan’s Central Harlem (control) (circled).
Sample
The project received Institutional Review Board approval from Rutgers, the State University of New Jersey. N = 144 participants were recruited at baseline from a probability sample of African American residents in the two neighborhoods using Address-Based Sampling, which employs the most recent Sequence Delivery File from the US Postal Service. A simple random sample of addresses was selected from within each area, with careful attention to the exact number of addresses available for selection, therefore enumerating the exact probabilities of selection for each sample address and enabling the strongest sample design for making estimates from within, and across each neighborhood. Selected households were visited between December 2011 and May 2012 and screened to identify eligible adults; if more than one household member was eligible, one person was randomly selected to complete the interview. Eligible adults were at least 18 years old, English speaking, self-identifying as Black/African American, and had lived in the USA since at least age 5. Those who did not spend their formative years here were excluded because experiences with racism in the USA are likely to be significantly different for individuals who grew up elsewhere, e.g., see 22. Because other portions of the project involved blood draws to examine biomarkers of stress pathways, exclusion criteria were surgery or blood transfusions within the past 6 months. Most non-eligible households were those with no Black residents or residents who could not speak English well enough to complete the survey. The sample was 52 % female, 48 % male, with a mean age of 44.6.
Overall response rates were in the 30–35 % range, reflecting difficulty in screening—only slightly more than half of households were successfully reached for screening. This is to be expected in a city like New York, where it is a challenge to be present when people are home, and because most individuals live in apartment buildings, rather than single-family homes, making face-to-face invitation a challenge. Additionally, two-phase interviews, as employed here (screening interview to determine eligibility followed by administration of the interview to qualified members) face additional challenges by eliciting participation twice, often from two different household members.23 Rates for successfully interviewing people was about 60 % once a household was known to have an eligible person. Participants completed a baseline interview lasting approximately 1 h using a computer-assisted in-person interviewing program. The baseline interview assessed demographic characteristics, medical history, physical and mental health status and health behaviors, a number of constructs related to neighborhood conditions, racial identity and experiences with racism, and attitudes about advertising.
Three months after the conclusion of baseline data collection, the intervention commenced (described fully below), comprising an outdoor ad campaign of six advertisements in bus shelters, beginning in August 2012, and running for 6 months, one ad per month. The last ad ran in January 2013, and follow-up surveys were then initiated in February 2013. The 1-year follow-up interview rate was 71 % for Harlem and 75 % for Bedford-Stuyvesant, resulting in a final sample size of N = 103. Comparing baseline characteristics among individuals lost to follow-up to those who completed both interviews showed few differences. Individuals who did not complete the follow-up were younger (mean age = 41.88, SD = 17.26) than those who did (46.20, SD = 16.87), and were more likely to be men (58.5 %) than women (41.5 %). No differences were seen in baseline assessments of financial strain, employment status, racism or Africentrism, perceived neighborhood characteristics, attitudes about advertising, or health outcomes.
A manipulation check used recall and recognition during the follow-up interview to assess participant exposure to the campaign. Participants were presented with nine letter-size reproductions of the advertisements, of which six were the original ads and three were foils. Participants were first shown incomplete versions of each ad (e.g., the ads included images but no tagline), were asked if they had seen it on the street, and if so, were asked to report the content in the ad. They were then shown the complete version, and again asked if they had seen the ad on the street or online.
Measures
Within the sample, individual-level outcomes were: beliefs about the extent of racism faced by African Americans (Group Impact subscale of the Racism and Life Experiences Scales,24; orientation towards African American culture, collective self-determination and political goals (Africentrism Scale,25; empowered strategies to cope with racism (e.g., talking to someone about it) based on items from the Everyday Discrimination Scale and the Experiences of Discrimination Scale26,27; network discussions of experiences with racism (one-item measure, “How often do your friends or family talk about racism that happened to them?”); perceptions of reciprocated exchange among neighbors28 measures of mental and physical health (Behavioral Risk Factor Surveillance System); psychological distress29; and one-item questions created to assess number of upper respiratory infections and average number of alcoholic beverages consumed on weekends and on weekdays.
As described in the next section, I hypothesized that beneficial changes in those outcomes would be enacted through individual and community-level pathways. Although an exhaustive study of community impact was beyond the scope of study resources, both I and research staff employed a number of methods to study this broader impact. First, once RISE commenced, we conducted observations at bus shelters, assessing whether passersby or individuals waiting for the bus appeared to take notice of the ad (e.g., stopping to read it, taking photos, talking with others about it), noting the condition of the ad (e.g., whether the panel was defaced or broken). Second, we searched social media platforms for photographs of RISE. On Instagram, we searched for hashtags that involved actual ad taglines or other content (e.g., #servants, #nypd, #racismstillexists), words related to geography (e.g., #bedstuy, #brooklyn), or other related keywords (e.g., #inequality, #disparities). Because people often used hashtags that could not be anticipated (e.g., #truth, #message, #realtalk), once we identified users who posted images, we began to check their photo streams and paged through previous postings for any other photos of RISE. Third, we searched for commentary on other social media including Twitter, Facebook, and blogs. Finally, I tracked website visits and activity with Google Analytics.
Intervention design
Advertising space in NYC bus shelters was purchased from CEMUSA, a media company that has an exclusive contract with the city for transit street furniture. Figure 2 is a map showing the dispersion of all bus shelter advertising spaces within Bedford-Stuyvesant (blue dots), and those initially available to disseminate the intervention (red dots). The specific locations varied slightly in some months, based on availability, but overall, broad coverage across the neighborhood was maintained with 30 ad spaces per month. Figure 3 shows a typical bus shelter. “Impressions” quantify exposure with estimates of the numbers of people likely to have seen an ad based on census data, traffic volume, pedestrian counts, bus service, ad panel location, and more. Based on audits of 25 of the 30 spaces, for individuals aged 18 or older, CEMUSA reported weekly impressions as 36,465 for 1 week, 145,860 for 1 month, and 875,161 for the entire duration of the campaign. At street level, ads were striking and difficult to miss, clearly visible from one avenue (approximately two city blocks) away.
FIG. 2.
A map showing the dispersion of all bus shelter advertising spaces within Bedford-Stuyvesant (blue dots), and those initially available to disseminate the intervention (red dots).
FIG. 3.
A typical bus shelter.
RISE was designed to showcase six different topics, one per month. These were: 1. Representations of African Americans in film; 2. Racial disparities in public school suspension rates; 3. Disproportionate density of fast food in African American neighborhoods; 4. Targeted marketing of cigarettes to African Americans 5. Black-White housing-based wealth disparities; and 6. Racial disparities in NYPD Stop & Frisk policies. I developed the campaign’s topical content and copy, including “taglines”, and collaborated with a professional firm to create the visual design for each ad, including the campaign logo. The campaign used bold graphics and typography, and consistency in overall “look” to capture attention and reinforce recognition and retention.
The six ads are shown in Fig. 4; Fig. 5 is an example of how ads actually appeared at street level. Concurrent with the outdoor ad campaign, I created a website (professionally customized by the design firm) hosted on Tumblr (http://racismstillexists.tumblr.com) to present in-depth information about the ad content. Given that ads in the bus shelter had to be pithy and were constrained by size, the web presentation sought to provide greater context and supporting evidence for the assertions in the ads and alleviate some of the human subjects risk associated with presenting potentially disquieting information in such a broad format.
FIG. 4.
The six ads comprising RISE.
FIG. 5.
An example of how ads appeared at street level.
Hypothesized Causal Mechanisms
I hypothesized a number of individual and community-level pathways through which RISE could affect health, as shown in Fig. 6. Individual-level effects would occur among participants as a result of viewing the advertisements. Although not all participants would see all of the ads, I anticipated that given the geographic spread, number, and duration of the campaign, most individuals would see most of the ads—not only while waiting for the bus, but simply by walking around the neighborhood. Most NYC residents walk frequently and take public transportation, and many do not own cars. As well, individual health could improve vis-à-vis community-level processes. Though effects are likely to be smaller, the ad campaign could stimulate outcomes in the broader neighborhood that would promote the health of individual participants. In the interest of space, I discuss selected pathways below.
FIG. 6.
Potential individual and community-level pathways through which RISE could affect health.
Individual-level pathways
RISE could affect intrapsychic and interpersonal processes, individual behavior, and resource ownership. Intrapsychic cognitive and emotional processes change in response to the ads by stimulating several capacities through three sub-mechanisms. First, coping with an acute racist event requires the mobilization of resources to manage immediate consequences such as anger, sadness, physiological correlates, and blocked opportunities; but coping also requires awareness that such events are likely to be ongoing stressors.30 RISE could act as a coping “booster shot,” prodding viewers to consider the ways racism may affect their lives, and thus potentially fostering awareness and facilitating the ability to cope.
Second, by exposing the unfair policies and practices that have produced negative outcomes for African Americans, the campaign may reduce self-blame and internalized racism. By bringing the Black experience to the foreground in shared public space, the campaign activates racial identification, and may blunt the pain of racist injuries. Similarly, RISE speaks to an Africentric worldview, which can counter negative images of African Americans and foster group togetherness.31 For example, individuals who hold a nationalist ideology and perceive outgroups as viewing African Americans unfavorably are less likely than others to have high blood pressure and show a negative relationship between racism and blood pressure.30 Conversely, youth who receive fewer messages about the realities of racism have lower resilience in the face of adversity.32 Finally, RISE may reduce psychological distress. The campaign validates individual experiences with racism by bearing witness to subordination that is not often named, acknowledged or recognized in the dominant society. Distress may be reduced when individuals embody “the essence of the blues: to stare painful truths in the face and persevere without cynicism or pessimism”.33
Interpersonal processes are a second means by which the campaign could benefit health. The arresting visual and topical content of the ads is likely to stimulate discussion about the campaign, and racism more broadly, having a cathartic effect. For example, African American men participating in a therapeutic group centered on experiences with racism reported that the group validated stressful experiences, and gave strength to recharge and confront a racist society.34
A third health benefit could be realized through health-promoting behaviors, engagement with art, and advocacy and action. Health behaviors may improve in response to the two ads that confront fast food and tobacco industry practices, as demonstrated in the Truth campaign. To the extent that the campaign is viewed as public art, the ads may stimulate individual viewers to create their own artistic works. “A person who engages in art—who creates, witnesses, is made angry by, or is enraptured by art—has the possibility of being changed.” While these changes may be subtle, art can increase happiness, expand thinking, analytic understanding and intellectual openness, and broaden social spheres35 (p. 8). Finally, the ads may empower action and advocacy, either for oneself or for others. Research has shown that more empowered responses to racism are beneficial for health. For example, among US-born African Americans, accepting unfair treatment as a fact of life is associated with greater psychological distress.22
The final individual-level pathway is the facilitation of resource ownership. Resources are entities including money, power, prestige, knowledge, social support, and social networks, which help individuals avoid disease and their negative consequences.36 RISE confers specific knowledge about the existence and perpetuation of racial inequality to viewers, raises consciousness, emphasizes social analysis and critique, and unmasks the normativity of racial inequality. More broadly, advertising has been labeled harmful because it promotes harmful products, promotes a consumption ethic, and limits the flow of information.21 If so, RISE could facilitate knowledge by reducing imperatives to consume, and by increasing the flow of information in general.
Community-level pathways
If a significant number of community residents see the campaign, this may set in motion other forces that affect the health of individual study participants through the social environment. First, perception of the ads as art can act as a source of enrichment in the community. Engaging with art contributes to quality of life, which reproduces the creative capacity of a community, instills community pride, increases aesthetic values, and builds support for equality and human rights.35 Furthermore, we know that deep-seated collective efficacy and reciprocated exchange—social cohesion, shared expectations for control, and capacity for mobilization—are beneficial for communities in many ways.37 If art increases community cohesion and resilience,35 RISE may benefit community residents. Second, RISE could also affect the social environment through community organizations. The ad content may stimulate programmatic work around race and history, motivate discourse on racism and social inequality, and intensify activism and agitation for policy change and social justice. These activities would thereby benefit the well-being of neighborhood residents regardless of whether they saw the ads themselves.
Analytic Plan
Quantitative analysis of the intervention impact on study participants was anchored in what is often called the “Rubin Causal Model” a notational system that models the logic underlying causal effects, regardless of whether inferences come from experimental or observational data.38 An important issue for the study is noncompliance, where participants do not receive the conditions intended for their group assignment. Treatment compliance was assessed with the manipulation check described earlier. Fifty-eight percent of Bedford-Stuyvesant participants (intended viewers) saw no ads at street level, while 42 % saw at least one ad, along with 11 % of Harlem participants (unintended viewers); 89 % of Harlem participants did not see any ads. Only one person incorrectly reported having seen one of three foils. A negligible number of participants in either neighborhood saw any ads online.
Regressing actual on assigned treatment estimates ITTD, the proportion of participants who are treated in the event that they are assigned to treatment, minus the fraction that would have been treated even if assigned to control.39 In this study, ITTD = .326, (SE = .089), indicating that assignment to treatment caused approximately 33 % of the target participants to be treated. Given that, analysis proceeded as intent-to-treat (ITT), which assesses the average effect of treatment assignment on outcomes, regardless of the proportion of individuals assigned to treatment who actually receive the treatment. It is useful in analyses in which the focus is to determine whether average outcomes are changed, notwithstanding the proportion of intended targets that were reached.39 Thus, we obtain an estimate of the average impact the campaign had on participants who live in the intervention neighborhood compared to those who live in the non-intervention neighborhood. ITT can be expressed as:
![]() |
where Yi is the outcome; z is the assigned treatment group, and 1 = treated and 0 = control; and d = the actual treatment. Another possible analysis would be to assess effects on Compliers (complier average causal effect, or CACE)—in other words, the intervention impact on those who actually receive treatment. In this study, Compliers are individuals in Bedford-Stuyvesant who saw the ads at street level, and Harlem residents who did not. However, when the goal is to test intervention effects that are mandatory for everyone, rather than interventions made to those who seek it out, ITT is preferable.39 This applies here, as the neighborhood-wide dissemination makes it a “mandatory” treatment. And, because I hypothesized that the campaign would affect the health of individual participants through community-level processes—regardless of whether they saw the ads—ITT is again the more appropriate analysis. Finally, the small samples sizes in each CACE category would make this approach less useful.
Because I hypothesized that the campaign would affect individual health both as a function of seeing the ads, and through community-wide processes, I present evidence for both direct (participants) and indirect (local and extralocal) impact. Direct impact was assessed quantitatively with regression analysis to estimate the difference in difference in outcomes between the two neighborhoods from baseline to follow-up, and with qualitative reports of recall of the campaign content among treated participants. Indirect impact was assessed with observations of community reactions at street level and on a variety of social media.
Results
Pre-intervention covariate balance
Assessments of neighborhood differences showed that though not statistically significant, Harlem participants had higher levels of psychological distress (mean = 3.85, SD = .66) and more upper respiratory infections (common colds) (mean = 2.47, SD = 6.35) than Bedford-Stuyvesant (mean = 3.58, SD = .66, and mean = 1.23, SD = 3.98, respectively). Bedford-Stuyvesant participants consumed more drinks on average on weekends (mean = 3.11, SD = 5.52) than Harlem participants (mean = 2.88, SD = 11.86).
Intervention impact: study participants
To estimate the ITT, I regressed outcomes, Yi (difference score between baseline and follow-up) on the treatment assignment and a set of covariates. Analyses employed poststratification weights to adjust for probability of selection, nonresponse, and underrepresentation of any gender or age group differences in the final interview data compared to the known representation within the two neighborhoods. SAS 9.4 and Stata 13.1 were used to conduct statistical analysis accounting for the complex survey design. Separate regressions were run for each outcome, controlling for age, gender, counts of positive and negative life events (e.g., getting married, losing a job) between baseline and follow-up, and severity of impact on participants and their friends and family from Hurricane Sandy, which struck the East Coast halfway through the ad campaign. Statistical significance was assessed as p < .05.
Change over time did not differ significantly between Harlem and Bedford-Stuyvesant for most targeted outcomes—reports of, or responses to racism, Africentrism, reciprocated exchange among neighbors, number of colds or days in poor health (physical and mental), or number of alcoholic beverages consumed. Initially, two outcomes were statistically different at the .05 level: network discussions of racism (the extent to which friends or family talked about their experiences with racism with the participant) and the average number of weekday drinks. However, after omission of n = 4 extreme cases in weekday drinking, these results were reduced to nonsignificance.
One outcome emerged as statistically significant: psychological distress declined from baseline to follow-up to a greater degree in Bedford-Stuyvesant than Harlem. The K-6 counts the number of symptoms of nonspecific psychological distress, and ranges from 0 to 24. On average, Bedford-Stuyvesant declined by −3.41 points, while Harlem declined by −1.51; the difference-in-differences was statistically significant p = .02. Table 1 lists the final ITT regression coefficients for the listed outcome variables, where neighborhood is the explanatory variable and the outcome is the difference for each variable. In Table 2, the full model for psychological distress is shown.
TABLE 1.
Regression coefficients for ITT analyses
| Outcome variable | B | Standard error |
|---|---|---|
| Group racism | 0.096 | 0.146 |
| Africentrism | 0.012 | 0.067 |
| Empowered responses to racism | 0.127 | 0.312 |
| Network discussions of racism | −0.388 | 0.204 |
| Reciprocated exchange | −0.061 | 0.185 |
| Number of colds | 0.967 | 1.16 |
| Number of poor mental health days | 3.51 | 2.14 |
| Number of poor physical health days | 0.598 | 2.27 |
| Number of average weekday drinks | 0.613 | 0.424 |
| Number of average weekend drinks | 0.6524 | 1.99 |
Coefficients represent the difference for each outcome in separate regressions with the primary explanatory variable being neighborhood, and with controls for age, gender, count of positive and negative life events between baseline and follow-up, and a rating of impact from Hurricane Sandy
TABLE 2.
Neighborhood differences-in-differences for psychological distress
| Explanatory variable | B | 95 % C.I. | |
|---|---|---|---|
| Bedford-Stuyvesant | −2.15 | −3.94 | −0.357 |
| Age | 0.029 | −0.018 | 0.076 |
| Gender | −0.052 | −1.77 | 1.67 |
| Positive life events | 0.431 | −1.49 | 2.35 |
| Negative life events | 0.347 | −2.02 | 2.72 |
| Impact from Hurricane Sandy | 0.105 | −1.34 | 1.55 |
Assessment of recognition and recall revealed that of the n = 27 participants who saw at least one ad, some were able to recognize them, but unable to recall the content (e.g., “I can't remember, but I saw it. I just don't remember. And I would read it almost everyday because it’s right there at the bus stop”). Others both recognized and were able to articulate at least some portions of the ads: (e.g., “It talked about stop and frisk. This is the one I had my son read and there was another one that I had him stop and read too”). Some respondents recalled only that the ads were related to race or racism, but could not remember the specific messages, (e.g., “About race, I forgot, it was about a racial thing”). Finally, a smaller number incorrectly recalled the content (e.g., “About Shirley Chisolm”).
Intervention impact: local and extralocal impact
Impression ratings for viewership would suggest that apart from the study participants, many in the community would have seen RISE. Indeed, as shown in Fig. 7, many individuals did take notice of the ad, sometimes studying them intently. Many who viewed the ads took pictures, shared the images or website links, and wrote about them on social media sites such as Twitter, Instagram, Facebook, and blogs, where commenters engaged in extensive debate. Importantly, this evidence is supportive of the pathways hypothesized to improved health outcomes (Fig. 6). An example of the campaign inducing health behavior change was seen in one individual’s posting—and the ensuing commentary—of ad #3 (fast food) on Instagram. Commenters debated the health consequences of patronizing fast food restaurants and the implications of fast food density for social justice and community self-determination. Engagement with RISE as art took shape in a documentary photographer posting photos on Instagram in which New Yorkers were captured in the bus shelters to illustrate ad themes (http://ruddyroye.tumblr.com/). Additionally, a summer arts program teaching youth about racism and public health using art referred to RISE as an example of how art could combine race and public health messaging (Gloria Diaz, personal communication, 7/17/13). There was some evidence that community organizations centered on race and activism were paying attention to the campaign. For example, Brooklyn organizations working on African American history and policy issues posted campaign images to their Facebook pages.40,41 Elsewhere in NYC, RISE appeared on the Facebook page of a group of social work students working “to have a stronger anti-racist, anti-oppressive lens in our curriculum”.42
FIG. 7.
Street observations of reactions to RISE at bus shelters.
Instagram was one of the primary settings in which public discourse around the campaign took place. We identified 150 postings through April 2013, each with as many as 20 hashtags to describe them, and often garnering numerous “likes.” The RISE tumblr website quickly gained viewers, and immediately reached beyond the local area. By 6:00 pm on the first day that the ads appeared in the bus shelters, Google Analytics data revealed 839 unique visitors to the site, from 32 countries and all US states save Utah, Wyoming and West Virginia. On January 11, 2013 a prominent news story covered the campaign,43 and this report was picked up by several national news sources (e.g., Yahoo, Huffington Post), including those targeting African American audiences (e.g., Ebony, Grio, The Root). Thus, website traffic is reported in Table 3 through January 10, representing website visits from 1,810 cities in 95 countries including the USA. Google Analytics data on search keywords revealed that a large proportion of visitors arrived at the site actively seeking information about RISE, while others appeared to seek evidence or confirmation of the persistence of racism, as well as topics that were addressed by the campaign.
TABLE 3.
Google Analytics data on RISE website visits
| Total visits | 7,653 | |
| Unique visits | 6,131 | |
| Page views | 13,965 | |
| Percent direct visists | 20 % | |
| Percent referred visits | 38 % | |
| Referral source | 44 % | |
| Tumblr | 40 % | |
| 9 % | ||
| 6 % | ||
| Percent online search visits | 42 % | |
| Percent mobile devices | 1 % | |
| Percent desktop computers | 95 % | |
| Percent tablets | 4 % | |
| NYC visitors, number by Borough | ||
| Manhattan | 2,351 | |
| Brooklyn | 97 | |
| Queens | 133 | |
| Bronx | 15 | |
| Staten Island | 0 | |
Discussion
This study investigated whether a 6-month neighborhood-level intervention using outdoor advertising to disseminate stark facts about racism could improve the health of a community sample of African Americans in New York City. Results suggested that immediately after the campaign concluded, attitudes related to race and racism and most health outcomes did not differ significantly between neighborhoods, but there was a greater decline in psychological distress among treatment participants. Declines in distress were hypothesized as an individual-level pathway; seeing the ads would validate social experiences and therefore reduce distress. Because most participants in the intervention neighborhood did not see the ads, declines in distress may indicate that the campaign fostered community processes that resulted in greater well-being for residents. The campaign clearly struck a chord in the broader community, as observed in pedestrians stopping to view, read and photograph advertisements; a high volume of sharing on social media, including posts that revealed individuals were following the campaign sequentially; reports in local media; Internet searches for the campaign; visits to the website; and anecdotal reports about community impact. Taken together, the study findings highlight the potential of racism countermarketing as a public health intervention. However, a number of issues must be considered.
Some limitations apply to the study. First, the modest sample size may have reduced the likelihood of detecting statistically significant associations. Second, the study design (lack of random assignment) makes it is impossible to account for all unmeasured variables. Therefore, despite contextual similarities between the two neighborhoods, baseline covariate balance on explanatory and outcome variables, and controlling for key covariates, there remains the possibility that the observed change in distress was due to neighborhood factors other than the campaign. Third, response rates suggest that the findings should be interpreted with caution with regard to representativeness, although response rates should not be used as the only measure of representativeness.44 This study employed probability sampling to obtain a representative sample, but lower response rates were countervailing forces; researchers must consider the juxtaposition of representative sampling methods with actual participation, loss to follow-up and other factors.45–47 In assessing representativeness, one useful comparison is that between study estimates and those from other sources, such as the 2012 NYC Community Health Survey (CHS).48 The closest approximation of geographies in the CHS to those in the present study are United Hospital Fund (UHF) neighborhoods, which are aggregations of 42 zip codes. The Bedford-Stuyvesant/Crown Heights UHF neighborhood is comprised of two separate communities, only one of which was studied in RISE, and the UHF Central Harlem neighborhood includes Morningside Heights (location of Columbia University), which was not studied here. Using those UHF boundaries, the prevalence of fair/poor self-rated health in Bedford-Stuyvesant/Crown Heights was 16.4 % (95 % CI = 11.7–22.6), while it was 24 % in the present study. In the UHF’s Central Harlem, the percentage was 19.5 % (95 % CI = 13.1–28.1), while it was 34 % in the present study. The present estimates are close to the confidence intervals in the CHS, despite the mismatch in geographies, suggesting that study estimates may approximate those in the target neighborhoods.
Nonresponse to surveys in US households has increased substantially over time and does not appear to be abating, with refusals comprising the largest component of nonresponse, much higher than noncontact and other reasons.23 In the present study, the opposite was true; many households could not be reached, but once successful, refusals were relatively low. This may suggest that participants were individuals with a particular interest in discussing racism or participating in a project related to African American health.
A fourth limitation is that study timelines precluded follow-up further out from the campaign’s end, and it may be that health impacts may become evident over a longer time period. Because the two neighborhoods were so closely matched on change over time (or lack thereof) on most health indicators, these may be outcomes that do not shift much over the course of a year. Finally, I did not test a placebo intervention to assess whether artfully designed ads on an unusual topic other than racism would have the same effects on psychological distress or public discourse. A pre-intervention ad census of 75 ad panels in Bedford-Stuyvesant revealed a restricted set of products and services in outdoor advertising. Mobile phones and plans comprised the vast majority (43 %) of ads, with the remaining 57 % promoting a rather banal mix of educational institutions, non-profit organizations, NYC initiatives (e.g., tourism, disaster preparedness), television/movies, and “Poor Richard” ads. The last are so named based on Vergara’s49 interpretation of “public service billboards,” (e.g., messages about drug abuse), which “teach ghetto residents how to live. The admonitions in these billboards are a contemporary equivalent of Ben Franklin’s Poor Richard’s Almanac.” In Bedford-Stuyvesant, these took the form of advertisements to “Keep America Beautiful” (i.e., refrain from litter and other disorder), and messages about texting and driving (e.g., “stoptextsstopwrecks.org”). It is possible, but unlikely, that any ads with eye-catching designs and interesting topics would have similar health effects, simply by virtue of being markedly different from the standard fare. Discourse about racism is not often confronted at the scale embodied in RISE, in public space, and with pointed messages in a visual medium. This is particularly so in a time when an alleged postracial and postfeminist, colorblind republic makes discussions of racial equity risky.50 Thus, although some hypothesized outcomes did not materialize in this study, an equally important outcome may have been to prime residents for subsequent interventions designed to tackle the health consequences of racism. The online searches that led people to the website suggest that whether or not they were aware of RISE, some people are seeking information about the persistence of racism and how it affects their lives. This should be a call to action for social scientists to apply the substantial research on inequality and racism to accessible public interventions.
Beyond the present study, certain limitations will be key considerations for all research employing similar methodologies. One is the obvious inability to randomize individuals to live in a given neighborhood, or to randomize individuals within the same neighborhood to different conditions (e.g., ads that magically display the target content when treatment, rather than control participants walk by). Random assignment produces groups of observations that are expected to be identical prior to application of treatment, and therefore produces an unbiased estimator of average treatment effects.38 Barring random assignment, close assessment of contextual differences between neighborhoods is critical. Still, these differences are not reducible to purely quantitative summaries of population or municipal characteristics. For example, Harlem and Bedford-Stuyvesant are different places, inhabited by different people, as is true for most neighborhoods. Vergara51 cites urban planner Alexander Garvin, who stated in 1967 that “Harlem pulled Arthur Schomburg from Puerto Rico, Josephine Baker from St. Louis, a. Phillip Randolph from Florida, Thelonius Monk from North Carolina, Duke Ellington from Washington, D.C., and Roy Wilkins from Minnesota” (p. 1). The forces that pulled these individuals to Harlem are illustrative of Sampson’s37 analysis of selection effects—“neighborhoods choose people rather than the common idea that people choose neighborhoods” (p. 327, emphasis in original). Differences in neighborhood vibes, legacy, and trajectory are not easily quantifiable, but exert considerable pressure on how interventions play out.
By the same token, assuming independence of social interactions among neighborhoods is deeply problematic; just as classrooms are nested within schools, so too are neighborhoods embedded in larger communities and metropolitan structures.36 In this study, the two neighborhoods are distant spatially, but are still nested within the broader Black community in NYC. In fact, although Harlem may have a distinct history, in the late 1930s, the extension of the A train fostered the movement of large numbers of Black residents from Harlem to Bedford-Stuyvesant,52 giving residents in the two neighborhoods shared history. Social networks and information are likely to traverse geographic space. While the 11 % of Harlem participants who were treated is not egregious, neither is it trivial. Researchers therefore need to think carefully about how to model potential spillover effects, and whether distance, time, measures of networks or other formulations are most appropriate.39
Even as an intervention may spillover to unintended areas, it is also clear that many in the target population will not experience the intervention. Attention is a scarce resource, and given the many stimuli people encounter outdoors, advertising can recede into the background. Only 42 % of participants in Bedford-Stuyvesant saw at least one ad, but even these rates are higher than other mass communication health interventions. In one study, 27 % of participants saw a billboard (targeting motorist, not pedestrian traffic) message at least once,17 while others53 report exposure rates of 13 % for billboards of an undefined size. It is difficult to assess what factors shaped Bedford-Stuyvesant participant exposure to the campaign. One explanation could be that participants often drove, and therefore did not pass the ads at close range on foot. However, only 32 % of participants in the total sample owned a car at baseline, and among Bedford-Stuyvesant participants, those who saw ads were actually more likely to own a car (58 %) than those who did not (31 %). More likely are activity spaces that did not intersect with the ad locations. Some people may routinely follow the same path to commute or run errands, using streets that did contain ads. Or, they may maintain a relatively circumscribed activity space around their homes.
Conditional on exposure to campaign advertisements, an additional consideration is decay. For product advertising, attitudes towards the product and purchase intention decay rapidly; in politics, advertising has a half-life of 2–3 days to 1 week, with effects decaying more rapidly for the least politically aware.54 Although high volume TV ads ($3 million per week) can exert a strong effect on voter preferences, shifting the target candidate’s standing by six percentage points, these effects are smaller and statistically equivocal effects one week later, and vanquished thereafter.54 Decay was apparent among RISE participants, as indicated by recall of ad content. One year out, participants recognized ads that they had seen, but were not necessarily able to recall the exact messages each displayed. It is possible that individuals were more likely to remember campaign content if it confirmed previously held beliefs and attitudes.19
If ad half-lives are short, outcome assessments in future studies would need to take place on much tighter timelines, more proximal to the end of the campaign, or perhaps at interim time points while the campaign is still running. However, apart from the significant increase in required resources to implement such a study design, it would be difficult to complete such assessments without sensitizing participants to the campaign purpose and introducing internal validity threats such as expectancy effects.
Directions for future research
A number of endeavors would be fruitful lines for future research. First, replication would be important to assess whether the findings reported here extend to other settings, and which mechanisms are responsible for any changes in well-being. As well, more in-depth study of other outcomes, such as coping and resilience is warranted. This study examined only whether participants talked about racist events or kept it to themselves, but a greater repertoire of coping strategies should be investigated.
Non-randomized designs have inherent challenges, and there is no easy way to solve them if interventions are deployed by neighborhood. On the one hand, such studies do well to maximize similarities between neighborhoods, in order to minimize residual confounding. However, an alternative approach might be to compare quite different neighborhoods, and investigate whether the intervention can keep a neighborhood with worsening trajectory on the rails and reduce existing or widening gaps. Field research allows assessment of how well effects can emerge through the real-world cacophony of political, economic, and social changes and distractions,12 and that cacophony might as well be embraced.
The substantial response on social media suggests that this may be a viable platform to enact interventions, and research studies could investigate the utility of such outlets to affect identity, beliefs, and health. However, this introduces new challenges. Although not all neighborhood residents will see outdoor ads, it is at least theoretically possible for anyone who lives in the neighborhood to be exposed. On social media, individuals who do not belong to or visit social media sites have no possibility of exposure, and participation in social media is of course not randomly distributed. Moreover, it is difficult to construct intervention and control groups on social media, as anyone who wishes to view the material could do so.
Qualitative research could drill deeper into individual reactions to the intervention, particularly because quantitative studies cannot entirely assess the impact of limited but meaningful engagement, or account for the fact that a small number of people might be deeply affected while larger numbers are unaffected. Moreover, subgroups of people may be differentially affected, and consequences can be removed in space in time.35 One participant described the campaign as, “I was like, ‘This is interesting.’ It’s like if people see and think about it… it could be very helpful and very aware. And it can help people be aware in the community.” Qualitative research could examine the extent to which people are aware, and how they use that awareness to negotiate the physical and social world. Organizational impact is suited to qualitative research as well, to assess mobilization to take up the issues raised by the campaign.
Future research could also investigate downstream effects. For example, researchers should look at social-interactional, organizational, and cultural mechanisms of city life, rather than solely individual attributes.37 Thus, rather than examining only individual health, useful outcomes could include civic capacity, political work by elected officials, neighborhood change, and more.
Conclusion
RISE was an outdoor ad countermarketing campaign designed to deliver stark facts about racism as a means to redress the negative health effects of racism. Statistically significant decrements in psychological distress were evident in the neighborhood in which the ads appeared, public response in social media was animated, and a broad community impact was observed, generating further evidence of the hypothesized mechanisms that would drive change. In describing their research on the impact of television advertising on voter preferences, Gerber et al.54 contend, “It should come as no surprise that a study such as this one generates as many research questions as it answers. This experiment is among the first in any discipline to estimate the effects of a large-scale media campaign using random assignment” (p. 149). The same might be said for RISE, conceived as a high-risk, high-reward intervention. Although the results are promising, more research is needed on this kind of novel public health intervention.
Interventions that target the stress of racism borne out of routine and painful subordination in most aspects of society (such as watching White passengers avoid occupying open seats next to Black passengers)55 are scarce, but sorely needed. Racial subordination is not easily addressed through policy, and individual psychotherapy to address the daily wear and tear of racism is not the appropriate scale. Bold, community-disseminated designs are required to stimulate public discourse and a willingness to confront the reality of inequality. The failure to implement field experimental designs is partly a failure of creativity, and theory development in laboratory research tends to create a theoretical echo chamber in which ideas are not tested in real-world settings.12 For a pressing social concern such as racism, with its pervasive effects on the day-to-day life and health of Black people, the need is high for creative, risky, community-based solutions.
Acknowledgments
This research was funded by the NIH Director’s New Innovator Award Program, Award Number DP2 OD006513 from the Office Of The Director, National Institutes Of Health and the National Institute Of General Medical Sciences (NIGMS). The content is solely the responsibility of the author and does not necessarily represent the official views of the Office Of The Director, National Institutes Of Health or the National Institutes of Health. I would like to thank the Center for Survey Research at the University of Massachusetts, Dr. Melody Goodman, Studio Kudos, and Community Board 3 in Brooklyn for their collaboration, consulting, and assistance; and the individuals who made this research possible—the respondents who participated in the project, and members of the Black LIFE Study research team.
References
- 1.Cuffee YL, Hargraves JL, Allison J. Exploring the association between reported discrimination and hypertension among African Americans: a systematic review. Ethn Dis. 2012;22:422–431. [PubMed] [Google Scholar]
- 2.Mays VM, Cochran SD, Barnes NW. Race, race-based discrimination, and health outcomes among African Americans. Annu Rev Psychol. 2007;58:201–225. doi: 10.1146/annurev.psych.57.102904.190212. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Paradies Y. A systematic review of empirical research on self-reported racism and health. Int J Epidemiol. 2006;35:888–901. doi: 10.1093/ije/dyl056. [DOI] [PubMed] [Google Scholar]
- 4.Pascoe EA, Smart RL. Perceived discrimination and health: a meta-analytic review. Psychol Bull. 2009;135(4):531–554. doi: 10.1037/a0016059. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Williams DR, Mohammed SA. Discrimination and racial disparities in health: evidence and needed research. J Behav Med. 2008;32(1):20–47. doi: 10.1007/s10865-008-9185-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Delgado R, Stefancic J. Critical race theory: an introduction. New York: New York University Press; 2001. [Google Scholar]
- 7.Hochschild JL. Ambivalence about equality in the United States or, did Tocqueville get it wrong and why does that matter? Soc Justice Res. 2006;19(1):43–62. doi: 10.1007/s11211-006-9999-2. [DOI] [Google Scholar]
- 8.Jost JT, Banaji MR, Nosek BA. A decade of system justification theory: accumulated evidence of conscious and unconscious bolstering of the status quo. Political Psychol. 2004;25(6):881–919. doi: 10.1111/j.1467-9221.2004.00402.x. [DOI] [Google Scholar]
- 9.Jones CP. Levels of racism: a theoretic framework and a gardener's tale. Am J Public Health. 2002;90:1212–1215. doi: 10.2105/ajph.90.8.1212. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Krieger N, Sidney S. Racial discrimination and blood pressure: the CARDIA Study of young Black and White adults. Am J Public Health. 1996;86(10):1370–1378. doi: 10.2105/AJPH.86.10.1370. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Huebner DM, Davis MC. Perceived antigay discrimination and physical health outcomes. Health Psychol. 2007;26(5):627–634. doi: 10.1037/0278-6133.26.5.627. [DOI] [PubMed] [Google Scholar]
- 12.Levy Paluck E, Green DP. Prejudice reduction: what works? A review and assessment of research and practice. Annu Rev Psychol. 2009;60:339–367. doi: 10.1146/annurev.psych.60.110707.163607. [DOI] [PubMed] [Google Scholar]
- 13.Hackbarth D, Silvestri B, Cosper W. Tobacco and alcohol billboards in 50 Chicago neighborhoods: market segmentation to sell dangerous products to the poor. J Public Health Policy. 1995;16:213–230. doi: 10.2307/3342593. [DOI] [PubMed] [Google Scholar]
- 14.Kwate NOA, Jernigan M, Lee TH. Prevalence, proximity, and predictors of alcohol ads in Central Harlem. Alcohol Alcohol. 2007;42(6):635–640. doi: 10.1093/alcalc/agm053. [DOI] [PubMed] [Google Scholar]
- 15.Moore H, Jones-Webb R, Toomey T, Lenk K. Alcohol advertising on billboards, transit shelters, and bus benches in inner-city neighborhoods. Contemp Drug Problems. 2009;35(2/3):509–532. [Google Scholar]
- 16.Outdoor Advertising Association of America I. Take your message further (video). 2013; Available at: http://www.oaaa.org/OutofHomeAdvertising/OutofHomeAdvertising.aspx
- 17.DuRant RH, Wolfson M, LaFrance B, Balkrishnan R, Altman D. An evaluation of a mass media campaign to encourage parents of adolescents to talk to their children about sex. J Adolesc Health. 2006;38:298.e1–298.e9. doi: 10.1016/j.jadohealth.2004.11.133. [DOI] [PubMed] [Google Scholar]
- 18.King EL, Grunseit AC, O'Hara BJ, Bauman AE. Evaluating the effectiveness of an Australian obesity mass-media campaign: how did the 'Measure-Up' campaign measure up in New South Wales? Health Educ Behav. 2013;28(6):1029–1039. doi: 10.1093/her/cyt084. [DOI] [PubMed] [Google Scholar]
- 19.Leavy JE, Bull FC, Rosenberg M, Bauman A. Physical activity mass media campaigns and their evaluation: a systematic review of the literature 2003–2010. Health Educ Res. 2011;26(6):1060–1085. doi: 10.1093/her/cyr069. [DOI] [PubMed] [Google Scholar]
- 20.Farrelly MC, Davis KC, Haviland ML, Messeri P, Healton CG. Evidence of a dose–response relationship between "truth" antismoking ads and youth smoking prevalence. Am J Public Health. 2005;95(3):425–431. doi: 10.2105/AJPH.2004.049692. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Dorfman L, Wallack L. Advertising health: the case for counter-ads. Public Health Rep. 1993;108(6):716–726. [PMC free article] [PubMed] [Google Scholar]
- 22.Krieger N, Kosheleva A, Waterman PD, Chen JT, Koenen K. Racial discrimination, psychological distress, and self-rated health among US-born and foreign-born Black Americans. Am J Public Health. 2011;101(9):1704–1713. doi: 10.2105/AJPH.2011.300168. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Brick JM, Williams D. Explaining rising nonresponse rates in cross-sectional surveys. Ann Am Acad Political Soc Sci. 2013;645:36–59. doi: 10.1177/0002716212456834. [DOI] [Google Scholar]
- 24.Harrell SP. Development and initial validation of scales to measure racism-related stress. In: 6th Biennial Conference on Community Research and Action, Society for Community Research and Action (Division 27-American Psychological Association). Columbia, South Carolina; 1997.
- 25.Grills C, Longshore D. Africentrism: psychometric analyses of a self-report measure. J Black Psychol. 1996;22:86–106. doi: 10.1177/00957984960221007. [DOI] [Google Scholar]
- 26.Krieger N, Smith K, Naishadham D, Hartman C, Barbeau EM. Experiences of discrimination: validity and reliability of a self-report measure for population health research on racism and health. Soc Sci Med. 2005;61:1576–1596. doi: 10.1016/j.socscimed.2005.03.006. [DOI] [PubMed] [Google Scholar]
- 27.Williams DR, Jackson JS, Anderson NB. Racial differences in physical and mental health: socio-economic status, stress, and discrimination. J Health Psychol. 1997;2:335–351. doi: 10.1177/135910539700200305. [DOI] [PubMed] [Google Scholar]
- 28.Earls FJ, Brooks-Gunn J, Raudenbush SW, Sampson RJ. Project on human development in Chicago Neighborhoods: Community Survey, 1994–1995, data collection instrument. Ann Arbor, MI: ICPSR; 1994. [Google Scholar]
- 29.Kessler RC, Barker PR, Colpe LJ, Epstein JF, Gfroerer JC, Hiripi E, et al. Screening for serious mental illness in the general population. Arch Gen Psychiatry. 2003;60:184–189. doi: 10.1001/archpsyc.60.2.184. [DOI] [PubMed] [Google Scholar]
- 30.Brondolo E, Brady Ver Halen N, Pencille M, Beatty D, Contrada RJ. Coping with racism: a selective review of the literature and a theoretical and methodological critique. J Behav Med. 2009;32(1):64–88. doi: 10.1007/s10865-008-9193-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Neblett EW, Carter SE. The protective role of racial identity and Africentric worldview in the association between racial discrimination and blood pressure. Psychosom Med. 2012;74:509–516. doi: 10.1097/PSY.0b013e3182583a50. [DOI] [PubMed] [Google Scholar]
- 32.Brown DL, Tylka TL. Racial discrimination and resilience in African American young adults: examining racial socialization as a moderator. J Black Psychol. 2011;37(3):259–285. doi: 10.1177/0095798410390689. [DOI] [Google Scholar]
- 33.West C. Democracy matters. New York, NY: Penguin; 2004.
- 34.Elligan D, Utsey S. Utility of an African-centered support group for African American men confronting societal racism and oppression. Cult Divers Ethn Minor Psychol. 1999;5(2):156–165. doi: 10.1037/1099-9809.5.2.156. [DOI] [PubMed] [Google Scholar]
- 35.National Endowment for the Arts. How Art Works: The National Endowment for the Arts' Five-Year Research Agenda, with a System Map and Measurement Model. Washington, DC: Author; 2012.
- 36.Link BG, Phelan J. Social conditions as fundamental causes of disease. J Health Soc Behav. 1995;36:80–94. doi: 10.2307/2626958. [DOI] [PubMed] [Google Scholar]
- 37.Sampson RJ. Great American city: Chicago and the enduring neighborhood effect. Chicago, IL: University of Chicago Press; 2012.
- 38.Milstein Sondheimer R, Green DP. Using experiments to estimate the effects of education on voter turnout. Am J Polit Sci. 2010;54(1):174–189. doi: 10.1111/j.1540-5907.2009.00425.x. [DOI] [Google Scholar]
- 39.Gerber AS, Green DP. Field experiments: design, analysis, and interpretation. New York, NY: Norton; 2012.
- 40.Weeksville Heritage Center. 2013; Available at: https://www.facebook.com/weeksville
- 41.Brooklyn Movement Center. 2013; Available at: http://brooklynmovementcenter.org
- 42.Interschool Council on Undoing Racism. 2013; Available at: https://www.facebook.com/ICUR.NYC
- 43.King J. Series of Brooklyn billboards put racial inequity on display. Colorlines: News for Action 2013 January 11, 2013. Available at: http://colorlines.com/archives/2013/01/bed_stuy_billboards.html. Accessed 5 Apr 2014.
- 44.Peytchev A. Consequences of survey nonresponse. Ann Am Acad Political Soc Sci. 2013;645:88–111. doi: 10.1177/0002716212461748. [DOI] [Google Scholar]
- 45.Nohr EA, Olsen J. Commentary: epidemiologists have debated representativeness for more than 40 years-has the time come to move on? Int J Epidemiol. 2013;42:1016–1017. doi: 10.1093/ije/dyt102. [DOI] [PubMed] [Google Scholar]
- 46.Richiardi L, Pizzi C, Pearce N. Commentary: representativeness is usually not necessary and often should be avoided. Int J Epidemiol. 2013;42:1018–1022. doi: 10.1093/ije/dyt103. [DOI] [PubMed] [Google Scholar]
- 47.Rothman KJ, Gallacher JEJ, Hatch EE. Why representativeness should be avoided. Int J Epidemiol. 2013;42:1012–1014. doi: 10.1093/ije/dys223. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.New York City, Department of Health and Mental Hygiene. EpiQuery: NYC Interactive Health Data, 2012. Available at: https://a816-healthpsi.nyc.gov/epiquery/. Accessed 5 Apr 2014.
- 49.Vergara CJ, Gillette H. "Invincible Cities." 2006; Available at: http://invinciblecities.camden.rutgers.edu/intro.html. Pg. 3. Accessed 5 Apr 2014.
- 50.Hill CP. Just another American story? The first Black First Family. Qual Sociol. 2012;35:123–141. doi: 10.1007/s11133-012-9225-5. [DOI] [Google Scholar]
- 51.Vergara CJ. Harlem: the unmaking of a ghetto. Chicago, IL: University of Chicago Press; 2013.
- 52.Wilder CS. A covenant with color: race and social power in Brooklyn. New York, NY: Columbia University Press; 2000.
- 53.Wray RJ, Jupka K, Ludwig-Bell C. A community-wide media campaign to promote walking in a Missouri town. Prev Chron Dis. 2005;2(4):1–17. [PMC free article] [PubMed] [Google Scholar]
- 54.Gerber AS, Gimpel JG, Green DP, Shaw DR. How large and long-lasting are the persuasive effects of televised campaign ads? Results from a randomized field experiment. Am Political Sci Rev. 2011;105(1):135–150. doi: 10.1017/S000305541000047X. [DOI] [Google Scholar]
- 55.Wideman JE. The seat not taken. New York Times. 2010;6:2010. [Google Scholar]








