The Neighborhood Voice: Evaluating a mobile research vehicle for recruiting African Americans to participate in cancer control studies

Abstract

The Neighborhood Voice is a vehicle customized for conducting health research in community settings. It brings research studies into neighborhoods affected most by health disparities and reaches groups often underrepresented in research samples. This paper reports on the experience and satisfaction of 599 African American women who participated in research on board the Neighborhood Voice. Using bivariate, psychometric, and logistic regression analyses, we examined responses to a brief post-research survey. Most women (71%) reported that they had never previously participated in research, and two-thirds (68%) rated their Neighborhood Voice experience as excellent. Satisfaction scores were highest among first-time research participants (p<0.05). Women’s ratings of the Neighborhood Voice on Comfort (OR=4.9; 95% CI=3.0, 7.9) and Convenience (OR=1.8; 95% CI=1.2, 2.9) significantly predicted having an excellent experience. Mobile research facilities may increase participation among disadvantaged and minority populations. Our brief survey instrument is a model for evaluating such outreach.

Introduction

Rates of participation in research among African American populations are low, and inclusion of community-based samples of African Americans in public health studies has been uncommon (Graham, 1992; Lee, 1993; Yancey, Ortega, & Kumanyika, 2006). A host of perceived and real challenges to recruiting African Americans can result in study samples that are not representative of all socioeconomic strata of African American adults, but rather are comprised of those that researchers can conveniently contact, are not deterred by transportation or other barriers, are college educated, or have previously participated in research activities (Andresen, Diehr, & Luke, 2004; Qualls, 2002). To fully understand and help eliminate health disparities, strategies are needed to reach and recruit into research studies a more representative cross-section of disadvantaged and minority populations, especially those who are least likely to be exposed to or respond to conventional targeted recruitment approaches, i.e., using lay health workers, “cultural insiders” or churches and other community-based organizations (Yancey et al., 2006).

Although mobile units have long been used by hospitals, home health care providers, churches and social service agencies to engage socioeconomically disadvantaged populations, their use as a tool for conducting health research is uncommon. Our National Cancer Institute-designated Center of Excellence in Cancer Communication Research (CECCR) introduced the Neighborhood Voice to increase opportunities for participation in intervention research among African Americans in lower-income neighborhoods who might not otherwise be represented in such work. This multi-purpose mobile research unit was designed to host a range of research activities throughout communities. The Neighborhood Voice sought to increase research participation by addressing significant barriers to participation among African Americans such as lack of time and transportation (Russell, Maraj, Wilson, Shedd-Steele, & Champion, 2008; Yancey et al., 2006).

Given the novelty of this approach, we report here an evaluation of research participants’ experience on board the Neighborhood Voice. Specifically, we: (1) describe development and preliminary psychometric testing of an exit survey to assess reactions to participating in research on the Neighborhood Voice; (2) identify factors most strongly related to overall satisfaction with research participation on the Neighborhood Voice; and (3) assess awareness of the Neighborhood Voice and history of research participation among women who completed a study on board the vehicle.

Method

This research was approved by the Institutional Review Board at Saint Louis University.

The vehicle

The Neighborhood Voice is a shuttle-type vehicle customized for research purposes. It can host an eight-person focus group interview with perimeter seating around a conference-style table, and also converts into two separate partitioned interview areas. Because our Center focuses on cancer disparities research among African Americans, the vehicle exterior was customized to appeal to this population. While the Neighborhood Voice is used in several ways, it is generally driven into specific high-priority neighborhoods where residents are invited on board, administered an initial assessment, delivered an intervention, and administered a follow-up assessment under controlled conditions.

Data collection

Our evaluation was conducted among participants in three randomized studies in St. Louis, MO, all involving African American women exclusively. Recruitment focused on predominantly black neighborhoods with incidence rates of late-stage breast cancer diagnosis that were twice the overall State rate. The studies evaluated effects of videotaped breast cancer survivor stories and other cancer communication interventions on women’s use of screening mammography. Participants were recruited to the three studies via neighborhood canvassing approaches and small media. In order to publicize the van, we designed flyers that were appropriate for the intended audience (e.g., using photos of individuals similar to the study population) and included pictures of the van and the times during which the van would be visiting their area. We placed flyers in large apartment complexes and local grocery stores in an effort to maximize repeat exposure to the flyers. Because each study sampled a different geographic area within specific neighborhood boundaries, there was no overlap between samples across studies. All study participants received a $20 grocery store gift card incentive. Methods specific to each study are described in detail elsewhere (Kreuter et al., 2008; Kreuter et al., 2010; McQueen & Kreuter, 2010).

To encourage honest and critical responses, the Neighborhood Voice evaluation was anonymous, and distinct from participation in the main studies. Therefore, individual-level data gathered in the main studies were not linked to a participant’s responses to the Neighborhood Voice evaluation. Samples for the three main studies were rather homogeneous: all participants were African American women ages 40 and older with low levels of education and income. In addition, intervention effects in the main studies did not vary by age or income (Kreuter et al., 2010; McQueen & Kreuter, 2010).

Immediately after completing study activities, participants were asked if they would be willing to answer questions about their experience on the van. Of 839 participants in the three main studies, 58 (6.9%) completed part or all of the research protocol someplace other than the Neighborhood Voice and were therefore ineligible to complete the Neighborhood Voice evaluation. Of 781 who completed the protocol on the Neighborhood Voice, 3% (n=24) were not offered the opportunity to complete the evaluation due to time constraints. Of the remaining participants (n=757), 79% (n=599) agreed to complete the evaluation. The evaluation was self-administered and took 5–7 minutes to complete.

Measures

All items in the evaluation instrument fit on one sheet of paper (front and back), which was professionally formatted for reading ease. The instrument was not connected in any way to the research protocol of the main studies, and survey items focused on the reactions to the research setting rather than the research protocol.

Satisfaction domains

We used an iterative idea-generating process involving the CECCR research team and results from an extensive literature review on user satisfaction to identify possible domains of participant satisfaction. Research on user satisfaction has its foundation in business domains (Bailey & Pearson, 1983), and informs the ways manufacturers produce goods and services and develop brand loyalty. In the health services arena, these concepts have been used to assess patient satisfaction with health care devices, experiences, and interactions (Adler & Edsall, 2004; Kim & Chang, 2007). For example, although satisfaction measures are specific to a product’s intended purpose, central ideas such as perceived usefulness and ease of use may be applicable to a wide range of products (Venkatesh, Morris, Davis, & Davis, 2003). In addition, the recruitment literature (e.g. the PEN-3 model (Stallings et al., 2000); Abernethy et al., 2005) emphasizes the need for culturally-appropriate recruitment strategies. Guided by this literature, we hypothesized that seven domains were theoretically related to user satisfaction and applicable to the specific setting of the Neighborhood Voice: convenience, security, comfort, cultural appropriateness, distractions, time, and institutional affiliation.

We operationalized each of these domains with multiple descriptive items, and a panel of health communication and survey development experts reviewed the resulting assessment tool. The item selection and refinement process was also iterative, and items were revised multiple times until the group reached consensus. We also hypothesized that the gift card incentive might be a motivating factor for participation and influence satisfaction of the research experience in this population, and therefore added an item related to this issue. The final exit survey included 13 items, each with Likert-type response options (strongly agree/agree/neither agree or disagree/disagree/strongly disagree). Two items asked about women’s motivation for participation: whether they had participated because of the sponsoring institution (Saint Louis University), and whether they had participated because of the $20 gift card incentive. Five items asked about the vehicle environment: whether getting in and out of the van was easy, whether it was too noisy on the van, whether the chairs and benches in the van were comfortable, whether the temperature inside the van was just right, and whether there was enough space in the van for participants to talk about their opinions. Two items asked about participants’ well being: whether they felt in danger being on the van, and whether they felt safe getting to the van. Other items asked whether coming to the van was convenient, whether the van came to the participant’s neighborhood at a convenient time, whether participants liked seeing the van in their neighborhoods, and whether the van was easy to find.

Other questions measured their research experience. All participants were asked to rate their overall experience with the van (excellent/very good/good/fair/poor). Participants were asked whether they had ever been in a research project about health (yes/no). One item asked participants to indicate where, if at all, they had previously seen or read about the van (saw the van on TV/received a flyer at their home/saw a flyer in their neighborhood/saw the van in their neighborhood/saw the van somewhere else in the city/read about the van in the newspaper/other). Participants could check none, one, or more than one response as applicable. One item asked participants how long they had waited outside the van before their interview began for the main study (less than 5 minutes/5–9 minutes/10–15 minutes/more than 15 minutes).

Statistical analyses

The primary goal was to assess participants’ satisfaction with their research experience on the Neighborhood Voice. Individuals with missing data on items related to overall satisfaction and previous research participation were excluded from analyses. For this reason 34 survey respondents (5.7%) were excluded. All analyses include a final sample of 565.

We examined the internal consistency reliability of survey constructs related to women’s reactions to participating in research on board the Neighborhood Voice. Reliability was assessed using Cronbach’s alpha. We conducted an exploratory factor analysis using principal axis factoring to identify satisfaction subscales. We allowed interscale correlation and thus used an oblique (direct oblimin) rotation. Bartlett’s Test of Sphericity assessed whether the item correlation matrix was significantly different from the identity matrix, and the Kaiser-Meyer-Olkin Measure of Sampling Adequacy assessed the factorability of the data. Factors with Eigenvalues of 1.0 or greater were retained, and only factor pattern loadings greater than 0.40 were interpreted. The resulting satisfaction subscales were scored by taking the unweighted mean of responses for all items comprising the subscale. Subscale scores were considered missing when data for 50% or more of subscale item responses were missing.

Descriptive statistics were used to examine prior participation in health related research, prior awareness of the Neighborhood Voice, and wait time. To examine participant satisfaction and which factors were predictive of overall satisfaction, we created a dichotomous satisfaction variable (excellent/less than excellent) and conducted a binomial logistic regression analysis to determine the contribution of satisfaction subscale scores, previous research participation and wait time (less than 5 minutes/5 minutes or more) to overall satisfaction. Adjusted odds ratios and 95% confidence intervals are reported. Additionally, we examined whether Neighborhood Voice experience and satisfaction varied by a woman’s prior research experience.

Participants’ prior awareness of the Neighborhood Voice, wait time, and overall satisfaction are categorical variables reported as proportions. Satisfaction subscale scores are continuous variables reported as means. Chi-square analyses were used to compare proportions, and independent samples t-tests were used to compare means. All analyses were conducted using SPSS version 15.

Results

Participants

Participants had a mean age of 58.3 years, 69.0% had completed 12 years of education or less, and 78.2% had an annual household income of $20,000 or less. Nearly three-quarters (71.0%) of survey respondents had never before participated in a research project about health.

Satisfaction domains

Internal consistency reliability of the 13 items representing satisfaction domains was fair (Cronbach’s α=0.68), and the items were suitable for factor analysis (Bartlett’s test p<0.001; KMO=0.86). Two items loaded poorly (loadings<.4) and were therefore excluded: “I participated today because it is part of Saint Louis University” and “The main reason I agreed to participate today was for the money.” The remaining 11 items loaded on three factors that explained 66.0% of the total item variance. Factor 1 was labeled Convenience, Factor 2 was labeled Distraction, and Factor 3 was labeled Comfort. Specific items and factor pattern loadings are listed in Table 1.

Table 1.

Factor loadings of items for a measure of satisfaction with mobile van research participation.

Item	Factor 1: Convenience	Factor 2: Distraction	Factor 3: Comfort
Coming to the van was very convenient	0.80
The van was very easy to find	0.77
The van came to my neighborhood at a time that was very convenient for me	0.73
I felt very safe getting to the van	0.65
Getting in and out of the van was very easy for me	0.52
I felt like I was in danger being on the van+		0.80
It was too noisy on the van+		0.74
The chairs and benches in the van were very comfortable			0.89
There was enough space in the van to talk about my/our opinions			0.78
The temperature inside the van was just right			0.48
I liked seeing the van in my neighborhood			0.42

⁺reverse scored item

The Convenience subscale contained five items and explained 43.3% of the total item variance. Internal consistency reliability for items comprising this subscale was high (Cronbach’s α=0.84). The sample’s mean Convenience score was 4.6±0.6 on a scale from 1 (least convenient) to 5 (most convenient). The Comfort subscale contained four items and explained 9.4% of the total item variance. Internal consistency reliability for items comprising the subscale was acceptable (Cronbach’s α=0.73). The mean Comfort score was 4.5±0.6 on a scale from 1 (least comfortable) to 5 (most comfortable). The Distraction subscale contained two items and explained 13.3% of the total item variance. Internal consistency reliability for items comprising this subscale was acceptable (Cronbach’s α=0.72). The mean Distraction score was 1.6±1.0 on a scale from 1 (least distracted) to 5 (most distracted).

On average, women who had not previously participated in health research had higher Distraction subscale scores than women with prior research participation experience (4.5 vs. 4.2; p<0.05). Mean scores for Comfort and Convenience did not vary by women’s prior research experience.

Overall satisfaction and experience

Most respondents rated their overall experience on the Neighborhood Voice as excellent (68.0%) or very good (27.8%). Binomial logistic regression analysis found the subscale scores for Comfort and Convenience to be predictors of an “excellent” satisfaction rating (Table 2). For every one-point increase in comfort score, overall satisfaction was 4.9 times as likely to be rated as excellent. With every one-point increase in convenience score, overall satisfaction was 1.8 times as likely to be rated as excellent. Distraction score, previous research participation, and wait time were not significant predictors of an excellent satisfaction rating. A larger proportion of women who had not previously participated in health research rated their experience on the Neighborhood Voice as excellent compared to women with prior research participation experience (70% vs. 62%; p<0.05).

Table 2.

Predictors of an excellent satisfaction rating with participation in research on a mobile van.

	aOR*	95% CI+
Satisfaction subscale scores
Convenience	1.8	1.2 – 2.9
Distraction	0.8	0.7 – 1.0
Comfort	4.9	3.0 – 7.9
Previous research participation	0.9	0.5 – 1.4
Wait time ≥5 minutes	0.7	0.4 – 1.2

^*Adjusted odds ratio per one point increase in subscale score

⁺95 % confidence interval

Most respondents (74.4%) reported waiting less than 5 minutes to participate in the one of the main studies, although 10.2% reported waiting more than 15 minutes. Fewer reported waiting 5–9 minutes (7.5%) or 10–15 minutes (7.9%).

Prior awareness of the Neighborhood Voice

Prior awareness of the Neighborhood Voice was high, with 92.6% of women reporting that they had seen or heard about the van before. Among individuals with prior awareness (n=523), 33.7% had seen the van in their neighborhood, 19.1% had received a flyer at home that showed a picture of the van, 10.1% had seen a flyer in their neighborhood, 7.6% had seen the van somewhere else in the city, 5.0% had seen the van on TV, and 0.6% had read about the van in the newspaper. Nearly one-third (29.8%) had heard about the van through other sources such as word-of-mouth or seeing a flyer about the van outside of their neighborhood.

Discussion

This study examined women’s experience participating in health research in a novel research environment. Our mobile van, the Neighborhood Voice, was generally reported to be comfortable and convenient, and these characteristics were predictors of overall satisfaction. If participants are comfortable in a research setting and if participating in research is convenient, it is reasonable to expect that satisfaction is higher and thus engagement in the research process may be greater. These findings are consistent with other studies showing that research participation among African Americans is related to the accessibility of the research activities, scheduling flexibility, comfort of the research environment, and the degree of interference with family and employment obligations (Corbie-Smith, Thomas, Williams, & Moody-Ayers, 1999; Russell et al., 2008; Sengupta, Strauss, DeVellis, Quinn, DeVellis, & Ware, 2000; Yancey et al., 2006).

Many health and social service efforts now use mobile units to reach populations most affected by disparities, although seldom has this approach been evaluated in terms of its potential improvement over traditional modes of delivery and outreach. The brief survey instrument that was used to assess participants’ experience on the Neighborhood Voice can serve as a model to evaluate these efforts. The study identified three constructs – Convenience, Distraction, and Comfort – among the satisfaction items administered to study participants. Psychometric properties of these measures are promising, although results might be improved with additional items that enhance the strength and internal consistency of subscales. The instrument may also be enhanced by eliminating the use of reverse scored items or adding items that elicit neutral and negative opinions (e.g., suggestions for improving the van experience) and items that address very specific aspects of research participation (e.g., how often a particular distraction occurred). Measuring specific experiences, in addition to and other measures of feelings and motivation for participation, should provide more varied responses (see a discussion of ratings versus reports by Harris-Kojetin, Fowler, Brown, Schnaier, & Sweeny, 1999). Additionally, measures capturing motivation for participating in research would complement these satisfaction scales. Still, our assessment tool is an important first step in developing measures to evaluate outreach activities employing mobile units.

Limitations

The study has several limitations. First, survey responses had limited variability. Most women rated their overall satisfaction with the van experience as excellent or very good, which may reflect a positive interaction with the research team, but could also be influenced by social desirability bias. However, global and other satisfaction items typically have a skewed positive distribution (e.g., Moret, Nguyen, Pillet, Falissard, Lombrail, & Gasquet, 2007). Second, we did not link the exit survey data with data from the main studies that included individual participant descriptors, and thus cannot describe individual-level demographic or other participant characteristics that may have been related to satisfaction with and experience on the Neighborhood Voice. Finally, it is possible that women who chose not to complete the exit survey (21%; n=158) differed from those who completed the survey, or had less positive experiences on the van. However, the sample was relatively homogeneous, and anecdotally, the reasons women gave for refusal were commonly related to lack of time rather than disinterest or dissatisfaction.

Conclusion

This evaluation provides some initial evidence to support continued use of mobile units in research studies, and contributes a measurement tool that others can build upon and apply in assessing research and outreach strategies in their own work.