Members of Minority and Underserved Communities Set Priorities for Health Research

Abstract

Policy Points.

• Engaging and involving underrepresented communities when setting research priorities could make the scientific research agenda more equitable, more just, and more responsive to their needs and values.

• Groups and individuals from minority and underserved communities strongly prioritized child health and mental health research, often choosing to invest at the highest possible level.

• Groups consisting of predominantly Native American or Arab American participants invested in culture and beliefs research at the highest level, while many groups did not select it at all. The priority given to culture and beliefs research by these groups illustrates the importance of paying special attention to unique preferences, and not just commonly held views, when getting public input on spending priorities for research.

Context

A major contributor to health disparities is the relative lack of resources—including resources for science—allocated to address the health problems of those with disproportionately greater needs. Engaging and involving underrepresented communities in setting research priorities could make the scientific research agenda more equitable, more just, and more responsive to their needs and values. We engaged minority and underserved communities in informed deliberations and report here their priorities for health research.

Methods

Academic‐community partnerships adapted the simulation exercise CHAT for setting health research priorities. We had participants from minority and medically underserved communities (47 groups, n = 519) throughout Michigan deliberate about health research priorities, and we used surveys and CHAT software to collect the demographic characteristics and priorities selected by individuals and groups.

Findings

The participants ranged in age (18 to 88), included more women than men, and were overrepresented by minority groups. Nearly all the deliberating groups selected child health and mental health research (93.6% and 95.7%), and most invested at the highest level. Aging, access, promote health, healthy environment, and what causes disease were also prioritized by groups. Research on mental health and child health were high priorities for individuals both before and after group deliberations. Access was the only category more likely to be selected by individuals after group deliberation (77.0 vs 84.0%, OR = 1.63, p = .005), while improve research, health policy, and culture and beliefs were less likely to be selected after group deliberations (all, p < .001). However, the level of investment in many categories changed after the group deliberations. Participants identifying as Black/African American were less likely to prioritize mental health research, and those of Other race were more likely to prioritize culture and beliefs research.

Conclusions

Minority and medically underserved communities overwhelmingly prioritized mental health and child health research in informed deliberations about spending priorities.

Amajor contributor to health disparities is the relative lack of resources—including resources for science—allocated to address the health problems of those with disproportionately greater needs.1, 2 Health research priorities do receive attention and influence from scientists, clinicians, advocacy groups, the private sectors of health care and health research, and other influential groups. But allocating scarce resources for health research requires attention to justice and science.3, 4 Engaging and involving underrepresented communities when setting research priorities could make the scientific research agenda more equitable, more just, and more responsive to their needs and values.5, 6, 7 Listening to communities also can enhance trust in researchers, research institutions, and funders.

Funders of research are increasingly looking for input from patients, the public, and/or stakeholders.8, 9, 10, 11, 12 Yet how best to engage minority and underserved communities when setting priorities for research remains a challenge. Traditional methods of engagement, like polling or focus groups, may not be very useful for a topic like health research, which is outside of normal life experience.13 Education and consultation do not usually emphasize reflection and may lead communities to question whether their input will affect decision making. Deliberative strategies have, therefore, been justified by appeals to develop a more informed public,14 create decisional legitimacy,15 and/or claim that deliberators and their constituents have consented to informed decisions.10 With the intention of forming a policy recommendation, deliberative procedures gather nonprofessional members of the public to learn about a topic and reason through various positions.13 Deliberative procedures are most appropriate when (1) nonexperts’ informed opinions provide important information that experts do not have; (2) informed opinions are difficult to obtain; (3) individual opinions will benefit from group discussion and insight; and/or (4) group judgments are relevant.13 All these conditions apply to setting health research priorities. Combined with trusting partnerships—in which communities contribute expertise, influence priorities, and decide how best to pursue research goals—informed deliberations provide an avenue for minority and underserved community members to share their opinions about what research is most important to pursue.

We adapted an existing simulation exercise, CHAT (CHoosing All Together), to facilitate deliberations about health research priorities constrained by limited resources. The aim of CHAT is to promote, in an inclusive and engaging manner, informed, reasoned dialogue among ordinary persons about complex and value‐laden allocation decisions.16 Research has demonstrated that participation in CHAT influences individuals’ understanding of and opinions about setting health‐related priorities and has found evidence of public‐spiritedness.17, 18 In this article, we describe the research priorities chosen by minority and underserved communities and how individual priorities changed after group deliberation.

Methods

Gaming and simulation have been applied for many years to policy decisions, including resource allocation, in public and private organizations.19 Those engaged in serious games are often those in positions of decision‐making power or other persons they wish to include. While the potential of “gamification” for civic engagement has received some recognition,20 policy gaming rarely emphasizes the voices of minority and low‐income communities. To adapt CHAT to the unique needs and objectives of research priority setting for minority and underserved communities, we used a participatory process that engaged community partners in all phases of the project. A steering committee, composed of a majority of community leaders and several leaders of research institutions,21 accordingly adapted CHAT, informed by the following resources:

1. Documents describing research priorities from governmental organizations, foundations, and research institutions.

2. Key informant interviews with

   • a.
     organizations that conduct and/or support research—how they categorize types of research and set priorities, how they present options and assess relative costs and what public input they would find valuable;
   • b.
     physicians practicing in underserved areas—what health research they need to inform patient care;
   • c.
     community leaders with experience collaborating in research—whether and how they would frame and present those types of research identified by organizations and clinicians, and what other options they would include in a priority‐setting exercise.

3. Regional groups organized in 5 areas of the state—identify issues for health research, determine the options they would include in a priority‐setting exercise, and describe scenarios or events that demonstrate health research issues facing their community.

Our research team, the steering committee, and regional advisory groups collaboratively authored CHAT content to be credible and comprehensible to a lay audience. After several iterations and revisions, we designed spending options to reflect the current priorities of industry and government as well as other options, so as to yield decisions useful to decision makers but not constrained by the status quo. The final content (which included definitions and explanations of a number of scientific terms) had a Flesch‐Kincaid readability score of 55 and was written at an 8th‐grade reading level (see the Online Appendix, Table 1). All the content was translated into Spanish.

Because we were asking laypersons with different levels of basic knowledge to deliberate about health research, we began the sessions with a brief video about health research goals, methods, costs, funders, and uses, and introduced the deliberators to their task. The CHAT game is played on tablet devices, presenting participants with an interactive game board resembling a pie chart (see Figure 1). Each wedge of the circle represented a category of health research spending, and each wedge had different levels of spending (including the option of no spending at all). Each of 16 categories of health research offered up to 3 cumulative spending levels that could be selected, with the higher levels (toward the center of the wheel) investing in more research at a higher cost. The costs assigned to different levels of spending within the categories reflected the assumption that there would be fixed costs associated with funding research within a category, so the first (lowest) level of spending in every category would need the largest marginal increase. In other words, moving from no spending in a category to any spending (level 1 in CHAT) requires more markers than increasing funding from a lower level to a higher one (eg, from level 1 to level 2). These categories and levels are described in the Online Appendix, Table 1.

Figure 1.

Screen Shot of the CHAT Wheel

Each wedge represents a category of research. Descriptions of each category and what the different levels of investment (level 1 = outer ring, level 2 = outer + middle rings, level 3 = outer + middle + inner rings) provide appear when clicking on a wedge. For each wedge, participants can choose not to allocate any of their 50 available markers, or they can use the number of markers needed for level 1, 2, or 3. [Color figure can be viewed at wileyonlinelibrary.com]

The participants chose the level of funding for each category by allocating the markers required for that level. However, participants were given a limited number of markers (50 markers with 92 open spaces) so if they selected a high level of funding in one category, they would have to choose less or no funding in another. The CHAT game was played in 4 rounds. In the first round, the participants set priorities as individuals; in the second round, they set priorities in small groups of 2 to 4; in the third round, they set priorities with the entire group (up to 15); and in the fourth round, they again set them as individuals. After rounds 1 and 2, the group heard and discussed scenarios (“events”) that illustrated the consequences of their choices. In all the rounds, trained facilitators asked the deliberators to make fair decisions on behalf of their fellow community members and to explain the reasons for their priorities.

Sampling and Recruitment

We recruited participants from minority and medically underserved communities22 primarily through community‐based organizations (eg, Michigan Center for Urban African American Aging Research, Arab Community Center for Economic & Social Services), using flyers and a variety of local advertising (newspapers, craigslist, radio) in English and Spanish; some we recruited through personal contacts; and others we recruited through the University of Michigan website UMHealthResearch.org. We excluded those volunteers under 18 years of age and those reporting during their initial screening that they were currently working in health care or health research, although some of them later reported on surveys that they were currently employed in these areas or had been employed in them, in various capacities. We aimed to recruit equal numbers of men and women, with a disproportionate representation of minority and low‐income residents. We convened 47 focus groups of 4 to 15 participants across the state of Michigan (total n = 519, see the Online Appendix, Figure 1) from February to November 2015. Most of the groups (42/47, 89.4%) contained at least 8 participants. The groups met in locations familiar to and convenient for participants (eg, community centers), to encourage an open and frank dialogue. Two group meetings (4.3% of all the meetings) were conducted in Spanish.

Data Collection

Pre‐ and post‐deliberation surveys measured the participants’ demographic and health characteristics. We calculated their poverty level using the upper portion of the participants’ income range identified and the number of persons living in their household, which thus represents a conservative (under)estimate (see the Online Appendix, Table 2). CHAT software recorded the categories and levels that the participants selected initially and after group deliberation, and the categories and levels that the group prioritized. Missing data for survey responses ranged from 0% to 7%. The 26 participants (5%) who did not select priorities after group deliberations were excluded from analyses that required those responses. The group deliberations were recorded and transcribed verbatim, and the discussion was coded by research category to examine the reasons for selecting specific priorities.

Analysis

We describe the proportion of individuals choosing each of the 16 research priorities, at which level before and after deliberation, and the number of groups selecting each priority at each level. We tested the effect of deliberation on changes in the level each participant selected using a hierarchical regression model with level changes from pre‐deliberation to post‐deliberation as a response variable and groups as random intercepts to adjust for within‐group clustering. The unadjusted effect of deliberation on individual priority selection (at any level) was summarized as odds ratios (ORs) and tested for its significance using a hierarchical logistic regression model, accounting for within‐participant repeated assessments nested within groups. For each post‐deliberation priority selection, we used a hierarchical logistic regression model with groups as random intercepts to estimate the independent associations between each post‐deliberation priority selection, and we summarized both various individual‐level (eg, demographics) and group‐level (eg, urban vs rural) characteristics and all associations as adjusted odds ratios (aORs). All analyses were done using Stata 14.1.

Results

The deliberators ranged from 18 to 88 years old, with 20% over 65 (see Table 1). About two‐thirds were women, and about one‐third resided in a rural area. About 45% identified as White, 30% Black/African American, 8% Hispanic, 6% Native American, and 4% Arab American, Arab, or Chaldean. Most of the participants (63%) had incomes of less than $35,000, and at least 157 (32.6%) were under the federal poverty level.

Table 1.

Participants’ Characteristics (N = 519)

Participants’ Characteristics	N (%) Except as Noted
Female	351 (67.6)
Age in years (n = 509), mean (sd, range)	48.3 (17.6, 18‐88)
Self‐identified race (n = 505)
White	252 (49.9)
Black or African American	158 (31.3)
Other, including multiracial	95 (18.8)
Native American	32 (6.1)
Arab American	23 (4.4)
Hispanic (n = 481)	35 (7.3)
Education (n = 510)
High school/GED or less	140 (27.5)
Some college	192 (37.7)
Bachelor's degree or more	178 (34.9)
Region (n = 519)
Southeast	230 (44.3)
Southwest	102 (19.7)
North	109 (21.0)
Upper	58 (11.2)
Thumb	20 (3.9)
Urbanity (n = 494)
Urban	298 (60.3)
Suburban	25 (5.1)
Rural	171 (34.6)
Income (n = 490)
Less than $15,000	165 (33.7)
$15,000 to $34,999	144 (29.4)
$35,000 or more	181 (36.9)
No. of people in household (n = 503), mean (SD, range)	2.7 (1.5, 1‐9)
At or below 100% federal poverty level (n = 481)	157 (32.6)
At or below 200% federal poverty level (n = 482)	257 (53.3)
Living alone (n = 502)	118 (23.5)
Perceived health status (n = 511)
Fair or poor	87 (17.0)
Good	179 (35.0)
Very good or excellent	245 (48.0)
Used to work in health care (n = 510)	115 (22.5)
Currently work in health carea	95 (18.6)

Cell values are N (%) unless otherwise described, and % does not add to 100 when some responses are missing.

^aThis includes any sort of work in a health care setting. Of the 91 who specified the work they do, 8 were certified or degree‐holding professionals (eg, nurse, social worker).

Research Priorities Selected by Individuals Before Deliberation

Table 2 shows the percentage of participants selecting each possible funding level for each research category before (round 1) and after (round 4) deliberation.“None” refers to not selecting that category for any funding. Before deliberation, most of the individuals selected, at least at the minimum level, mental health (90.6%) and child health (89.3%) research, and about a third invested at the highest possible level in each of those categories. The next most commonly selected categories prior to group deliberations were what causes disease (85.6%), promote health (82.7%), health care quality (82.7%), aging (81.1%), and healthy environment (81.1%). About three‐quarters of the individuals selected the access, families/caregivers, and multiple conditions categories before deliberations, at least at the minimum level. Before the deliberations, individuals were least likely to choose culture and beliefs, health disparities, what works better, health policy, communication, and improve research.

Table 2.

Individual Priority‐Level Selections in Research Categories Before (N = 513) and After (N = 493) Deliberation

	Before Deliberation				After Deliberation
	Selected Level (%)a				Selected Level (%)a
	None	1	2	3	None	1	2	3	Change (%)b	Mean Changec
What Causes Disease?	14.6	32.7	20.7	32.0	15.2	28.6	27.6	28.6	40.6	−.01
Promote Health	17.3	39.0	24.0	19.7	21.5	33.3	24.7	20.5	39.3	−.03
Communication	35.3	38.0	16.6	10.1	34.9	37.5	20.1	7.5	39.6	.00
What Works Better?	38.4	33.1	14.6	13.8	41.4	29.4	18.3	11.0	41.0	−.07
Health Disparities	40.2	34.5	16.8	8.6	44.0	26.6	18.7	10.8	40.6	.04
Families/Caregivers	28.5	39.4	18.9	13.3	24.7	31.8	27.6	15.8	37.9	.17d
Access	23.0	36.6	22.8	17.5	16.0	32.3	31.2	20.5	34.2	.22d
Improve Research	32.6	34.5	20.7	12.3	46.5	26.4	16.4	10.8	36.5	−.21e
Health Care Quality	17.3	36.5	26.7	19.5	19.9	39.4	25.4	15.4	34.0	−.15e
Aging	19.1	35.1	24.4	21.4	16.8	28.6	29.2	25.4	42.4	.14e
Child Health	10.7	28.8	28.7	31.6	10.3	19.7	32.9	37.1	41.4	.15e
Health Policy	35.5	39.4	17.9	7.2	48.5	29.2	15.4	6.9	41.8	−.17e
Healthy Environment	18.9	34.1	24.6	22.4	18.1	30.2	27.8	23.9	38.9	.09
Culture and Beliefs	44.4	30.6	14.6	10.3	54.0	29.0	9.9	7.1	50.0	−.20f
Mental Health	9.4	30.6	25.5	34.5	6.9	19.9	27.6	45.6	42.0	.28f
Multiple Conditions	25.3	39.4	17.7	17.5	25.6	32.3	23.7	17.4	37.7	.03

Cell values are percentages unless otherwise specified in column headings.

^aPercentages are calculated from those who prioritized health research in each round.

^bPercentage of participants who changed their selected level after deliberation out of those with data in both rounds.

^cCalculated as the mean of the priority levels after deliberation (round 4) minus before deliberation (round 1); positive values correspond to higher investment‐level selection after deliberation. Multilevel regression models adjusted for within‐CHAT group clustering.

^dp < .01

^ep < 0.05

^fp < .001

Research Priorities Selected by Groups

During the full group deliberations in round 3, nearly all the groups selected child health and mental health research (93.6% and 95.7%, respectively), and most chose the highest possible level of investment (level 3) for those two categories, which required using almost an eighth of their resources on each category (see Figure 2). The next most likely categories to be selected by groups were healthy environment, what causes disease, aging, access, and promote health, and at least 25% of the groups selecting those categories also chose to invest at the highest possible level. The categories what works better, health policy, culture and beliefs, and improve research were not selected at any level by more than 50% of the groups. However, with the exception of health policy research, at least 10% of groups chose to invest at the highest possible level in each of those categories.

Figure 2.

Group Selections (Round 3)

Level 3 is the highest possible level of spending. For definitions of research categories, see the Online Appendix, Table 1. [Color figure can be viewed at wileyonlinelibrary.com]

Research Priorities Selected by Individuals After Deliberation

Table 2 shows both the percentage of participants who changed their level of selection after deliberation and the mean difference in the level of selection. The level of investment in many categories changed after the group deliberations. The deliberators increased their investment in mental health research (within‐participant mean increase in level = 0.28, p < .001), access research (.22, p = .002), families/caregivers research (.17, p = .002), child health research (.15, p = .03), and aging research (.14, p = .049). They decreased their investment in improve research (within‐participant mean decrease = −.21, p = .02), culture and beliefs (−.20, p < .001), health policy research (−.17, p = .01), and health care quality research (−.15, p = .02). However, in terms of whether they selected at all, versus not, the only category significantly more likely to be selected by individuals after group deliberations (77.0% vs 84.0%, OR = 1.63, p = .005) was access, whereas improve research, health policy, and culture and beliefs were significantly less likely to be selected after group deliberations (all, p < .001; ORs are available upon request). Mental health and child health research were high priorities for individuals both before and after group deliberations.

Predictors of Priority Selection

After the deliberations, older age was modestly associated with a greater likelihood of selecting aging research (aOR = 1.03, p < .01) and a lower likelihood of selecting child health research (aOR = 0.98, p < .05) (see Table 3). Those identifying as Black/African American or another nonwhite race were more likely to prioritize communication research. Blacks were less likely than whites to choose mental health and what works better, and those of other races (compared with whites) were more likely to select health care quality and culture and beliefs. Hispanics were less likely to prioritize aging research. Rural residents were more likely than urban residents to prioritize child health research and less likely to choose the culture and beliefs and improve research categories. Those living under the federal poverty level were more likely to select health policy and improve research. More education was associated with a greater priority for healthy environment and a lower priority for improve research, multiple conditions, and health care quality. Better health status was associated with a greater tendency to select what causes disease, promote health, and mental health. Gender was not predictive of any priority selection.

Table 3.

Participant Characteristics Predicting Priority Selection at CHAT Post‐deliberation Round, N = 493a

		Race (Ref: White)
Priority	Age	Black	Other	Hispanic	Work in Health Care	Education	FPL	Rural vs Urban	Suburban vs Urban	# of Persons in Household	3‐Level Health Statusb	Poor Healthb
What Causes Disease?	.99	.59	.88	–	–	–	–	–	–	–	1.45 c	–
Promote Health	1.00	.54	.91	–	–	–	–	–	–	–	1.59 c	–
Communication	1.00	1.95 c	1.93	–	–	–	–	–	–	–	–	–
What Works Better?	.99	.52 c	.64	–	.65	–	–	–	–	–	–	–
Health Disparities	.99	1.84	1.81	–	–	–	–	–	–	–	–	–
Families/Caregivers	1.01	1.66	1.07	–	–	–	–	–	–	–	–	–
Access	1.00	1.95	.98	–	–	1.46	.53	–	–	–	–	–
Improve Research	1.02	.64	1.88	–	.61	.66 c	2.13 d	.27 d	2.44	–	–	–
Health Care Quality	1.01	.76	2.60 c	–	–	.66 c	–	–	–	–	–	–
Aging	1.03 d	.92	1.35	.27 c	–	–	.53	–	–	–	–	–
Child Health	.98 c	.92	1.59	.42	–	–	–	3.57 c	.38	–	–	–
Health Policy	1.00	1.55	1.68	–	–	–	2.04 d	.49	–	–	–	–
Healthy Environment	1.00	.65	.57	–	–	1.82 d	–	–	–	–	–	–
Culture and Beliefs	.99	1.46	4.06 d	–	–	–	–	.38 c	4.06	–	–	–
Mental Health	1.00	.15 c	.52	–	–	–	–	.29	.06 d	–	2.03 c	–
Multiple Conditions	.99	.92	1.66	–	–	.71 c	–	–	–	.83 c	–	5.87

Cell values are adjusted odds ratios. Adjusted odds ratios are obtained by fitting separate hierarchical logistic regression models for each priority and selecting the priority (at any level) as the dichotomous response, adjusted for clustering within CHAT group and covariates. Final models included significant predictors (p < 0.05) and marginally significant predictors with parameter estimates of large magnitudes. Models always included age, race, and gender, but gender is not included in the table, as it was not predictive of any priority selection. Potential predictors considered include 3‐level education (0 = ≤ high school, 1 = some college, 2 = college or more), 3‐level income, either a 3‐level self‐perceived health status (0 = fair/poor, 1 = good, 2 = excellent or very good) or a dichotomized (poor health versus not) measure. Hispanic ethnicity, living alone, number of people in the household, prior or current work in health care (yes/no), or FPL (below 100% federal poverty lines for 2015 guidelines). Blank cells reflect variables not predictive of the priority selection and thus not included in the model.

^aThere were 493 participants in the post‐deliberation round, but the sample size used for each priority model differed slightly depending on frequency of missing data in the predictors.

^bTwo alternate definitions of current general health status: The models used either the dichotomized (poor health versus not) or 3‐level (coded as 0 = poor/fair, 1 = good, 2 = very good/excellent) health status as ordinal measure, but not both.

^cp < 0.05

^dp < .01

Reasons Given for Priorities

Because deliberation should include “reason‐giving,”23 ^(p3) we next cite quotations illustrating the common or predominant reasons given for or against specific health research categories.

Investing in the future was a prominent reason given for the high priority given to child health research:

Because they're the future.

There are so many things that if you catch them early before they have done a lot of damage, you can pay me now or you can pay me a whole lot later. I vote for as much research as we can do in that area.

The priority given to mental health research appealed to the importance of good mental well‐being and also the relative lack of knowledge about it, that is, the need for research:

I think it's a basic foundation for people's well‐being.

Even though it's really ubiquitous, we still don't talk about it or really know anything about it.

Arguments for prioritizing healthy environment research often referred to its direct, and sometimes unknown, effect on health:

Our asthma rates here are 15.6 compared to [another] county, which is 9%. So we are really almost double that way in counting, which is one of the worst environmental counties in the United States of America. Our asthma rates are very, very bad.

I think environment is huge because we cannot say enough to our big corporations: “stop polluting our groundwater and our air.”

I think not having a healthy environment is leading to many diseases out there right now. From what we—the land we live on, the water, the air.

The reasons given for research on what causes disease often referred to the need to understand causes in order to treat or prevent illness:

What about those conditions that we can research and doctors still don't know why you have it or where you got it from or if there is a cure for it? For instance, I have RA, rheumatoid arthritis. But I got diagnosed at 20 and they can't figure out why I have RA, why it's progressing like it is, what's causing it to progress as fast as it is, and how to treat it.

Look at ‘“what causes disease”’ because if you don't understand what causes it, you can't fix it.

I think there are a lot of unknowns in that area. A lot of people that are getting sick for reasons that they really don't know or haven't researched enough.

I think that's the key to everything. If I knew that something I was doing would cause a disease, I wouldn't do it.

The deliberations on aging research, which was also given high priority, although not as high as that for child health or mental health research, included references to aging as universal and deserving respect, and to the special needs of aging well:

We all age.

Knowing and researching about aging and how to age gracefully rather than have the health deteriorate.

Elders are important.

Access was the only category more likely to be selected by individuals after the group deliberation. The deliberations attested to the importance of access to care, although the need for (or importance of) research about access was not always articulated:

I think something like “access” here is something we want to tackle … because something like getting an appointment or how we can get more doctors to the area if that became an issue or more nurses to the area if there was a nursing shortage or something like that.

I'm going to start with level 3 because what I found this last year, 2014, when the Affordable Care Act went into place, it was affordable as in if I got a tax break and I had to pay this amount and I was making this much money, it was almost bearable, but what happened was—like you—we had this humongous deductible that meant it might be affordable, but it's not valuable.… We had no way to use what we were being forced to pay for.

I think everything should be accessible to anyone that needs it. No matter your gender, your race, your income. Whatever it is, there should be a way for people to get their medication, transportation, the whole 9 yards.

Having access to health care is the first step and there's kind of a ripple effect from if you don't have access, the problems get worse.

Deliberations on promote health research often referred to its impact on future quality of life, future health care needs, and costs:

The idea it's cheaper to prevent it than to cure it.

If we can promote—like you said, “nip it in the bud”—we are not all going to be tied to that. We are not all going to be tied to 5, 6, 7, 8 medications a day.

Because if you can prevent from getting ill and diseases and know what healthy choices are, it's going to make a better life down the road.

Our society doesn't promote health, it promotes stress.

Deliberators who argued for prioritizing research on families and caregivers suggested that family members are increasingly caring for their loved ones, emphasized the importance of home care for keeping people out of nursing homes, and expressed concern with the health of caregivers:

Because without family caregivers, a lot of people would be placed in nursing homes and assisted living and stuff like that, and they need to know that somebody does care.

[People] can burn out so easily caring for someone, and then there is your next patient that is in line, that now you have two instead of just one. So I think we can't neglect it totally. I think there needs to be some type of support system there for that.

I think that as we see health care costs rising, that we are seeing that people are spending less time in the hospital and there is more and more need for families to be responsible for caring for each other.

There have been cases where the caregiver is the one who dies first before the person they are taking care of because of the stress.

Those who argued against investing in families and caregivers thought that information about the consequences of providing care for the caregiver and how to best provide care to family members was already available:

I don't know how much research you need to know that stress levels rise the more care you give—you've got 4 siblings and only 2 are helping.

Everyone's aware of the stress that puts on the caregiver. I mean, it's a known fact it's there, this is research to find something we already know? We know it's stressful. This is not a new issue.

You know, if I had someone in my family that was sick, I bet I could go on the internet and get 9 days’ worth of good constructive reading on how to help.

The deliberators’ arguments for prioritizing multiple conditions research referred to the numbers of people afflicted with more than one condition. The deliberators also suggested that having multiple conditions often leads to complications from taking too many prescriptions and leads to complex relationships between a patient and multiple health care professionals:

Most people do have multiple conditions. Not just one thing is going on with them.

Because a lot of people have multiple situations and they had to take pills for every last one of them. I'm talking about 15 or 20 pills.

Seldom you get just one problem. It's a mixture of problems, and so it is important to recognize how they interact in order to know what the medication or the treatment would be.

It's difficult dealing with different doctors, trying to get them to come to a consensus on treatments and such.

Those who did not wish to prioritize research on multiple conditions thought that the availability of quality health care and research in other CHAT categories made investing in this category unnecessary:

If the doctor is up on his medical studies and treatments, then if you have multiple conditions and you have a good doctor, then they're going to treat you properly.

If you have multiple conditions, hypertension and diabetes, you're going to find out about those individually.

I think that's going to be covered by a lot of these other research areas.

I think that is already existing and what every doctor will do is try to manage multiple conditions for patients.

Deliberations about communication research emphasized the importance of doctor‐patient communication, the way that technology can improve communication in health care, and the importance of informing the public about health:

[Doctors] need to keep the communication open, and their responses need to be where you are comfortable enough to keep communicating with them honestly, without holding anything back. So they need to learn not to be judgmental.

Believe it or not, there are a lot of old wives’ tales that a lot of people believe, and they avoid vaccinations; they avoid all things that were put in place to protect.

Sometimes there's no communication between the patient and the doctor because the doctor might be speaking and telling you something and you can't understand what he's saying.

You know, I think this is an area where we can really make some progress, and with technology advancing the way it is, there's all sorts of opportunities for us putting together databases that are useful that it just couldn't be done 10 years ago.

It's very important, especially in the community that we live in. Some people don't actually speak English, like most of the people that came from Lebanon. Like my grandparents don't know how to speak English that well, so I always have to take them to their doctor's appointments.

Those who were against prioritizing research on communication thought enough research was already being done in this category, that it was not a local concern, or that effective communication was the responsibility of individual patients and providers:

I think that responsibility should fall on people. The patient needs to be an advocate for themselves, and the doctor needs to be responsible to make sure that he performs that service to the patient.

I would say there's a big emphasis on this right now, and I think they have done a lot of research. For example, now when I go to the doctor, you can go into those portals and you can see everything that the doctor has written about you and everything! I mean, that's communication!

We save that for the bigger cities. Our doctors talk to each other. They go to tea together.

Those deliberators interested in investing in research on health care quality thought that research in this area was lacking and could benefit patients and communities by aligning the priorities of providers and health systems with those of patients and improving the delivery of care by adequately trained health care providers:

Quality of care is huge when it comes to client rights, when it comes to healthy outcomes, when it comes to support for the family and the community. All those things are involved with health care quality.

It does seem like over the years there has been a change, and I don't want to say “quality of health care,” but it seems to be a change in the focus of health care professionals and it's about the bottom line, and that bottom line is no longer you being the healthy patient; it's about them getting the insurance money, getting paid for the service they provide.

Because there's doctors out there that, I'm sorry, are hacks.

It's self‐explanatory. If you don't have quality in your health care, what's the point of having health care?

I honestly don't think enough research has been done in order to actually improve the quality, and we're still hearing stories of people who have had bad experiences all across the board.

Those deliberators who were not interested in research on health care quality thought that quality was already enforced through federal regulation and that programs to implement change, not redundant research, were needed in this area:

It's being covered by the feds really, really well.

Nielsen does research studies on that, and there are a number of other companies that currently do it, so my feeling is that it's already being dealt with in a number of ways.

This isn't a program that they're trying to do, this is research and finding data and—this isn't implementing a solution. It's, oh, do we think there's a problem, identifying the problem. We already know there's a problem.

Although it was not a highly prioritized category, many deliberators did argue in favor of health disparities research. They emphasized the importance of research in this area for investigating disparities that they were aware of based on race, ethnicity, gender, sexual orientation, age, and income. They also argued for greater representation in research:

Specific to our area, we do have all these different socioeconomic groups, races, cultures, and each culture has their own diseases that are specific to them.

I am a part of the queer community, and the doctor is a very scary place.

Black women die at higher rates from breast cancer than Caucasian women because they are not getting the best care.

I think history has shown us that there has been a grave difference in the care that people get according to their economic status and especially because of their race. I even believe that a lot of the research and the prescriptions are not based upon us.

It became fairly obvious to me that in your lower‐income areas, you have much greater health problems, both physical and mental. And I think we need to do more research into why, because the health facilities are there, they're available.

That's discrimination at the end of the day.

Those deliberators who did not wish to prioritize this category often expressed the need for implementing programs to act on existing knowledge of health disparities, not further research:

My only issue with spending money in this direction is that it's something that has been widely researched and already established and known. What is being done about it, I have no idea. And I don't know how further research will impact one way or the other.

This literally just says let's keep looking at the problem. We all know it's a problem, especially if you come from it.

All these are good ideas for implementation, but how is research going to help get this out?

If this was about a program and we were funding a program, fine, but for us to have another guy go out and do a research study about telling me that I'm different than you just seems like a waste of time and a waste of money.

Comparative effectiveness research was not highly prioritized. While a few deliberators argued for the importance of finding out what works better, others argued either that effectiveness varied by individual or that other systems (other research categories, the FDA) were sufficient:

I think in the long run that would actually save money, you know, if we knew what works better for an individual, like problems and illnesses and stuff.

I guess for me, research on that would be like, OK, what could we get to actually work better? That would be where I would see research being a high priority.

I think this is more individualistic, like what works for me probably isn't going to work for you and probably won't work for other people.

You can find out what works best through the research in these other categories.

I know in terms of getting drugs approved by the FDA, they have to show better efficacy than what is on the market. So if you have like 6 analgesics that want to come to market, they have to show superiority over what's already on the market.

Arguments made to prioritize health policy research typically argued that without it, changes would not or could not happen. Arguments against health policy research tended to convey skepticism that the research would have any impact on policy:

Nothing matters unless we do something about it.… Sure, we can say smog is bad for us and all that, but unless we actually have policy … nothing is going to happen.

The reason I'm thinking “health policy” is because the laws, the things that govern how health care is actually put out there, need to be looked at and tweaked constantly. It should be a living thing and not just something you did 50 years ago.

Policy is very important … because we have to go … to our senators because they have cuts on research and cuts on programs so we need to talk, to raise our voices, so that we keep the funds.

The policymakers don't pay a lot of attention to research anyway, in my opinion. So why bother?

“Health policy” … I hate to say this but I don't think there's very much we can do about it, and research on how we can impact it is just not going to produce many results.

Because the government already tells us what to do.

Culture and beliefs research was selected at the highest level by 5 groups, but not selected at all by 30 groups. The deliberations conveyed a range of importance attached to the need for understanding the impact of culture and beliefs on health:

If you're from a culture where you're not allowed to get this done to you or you can't take this medication, then what are you to do?

That's the most important thing, to become aware that those … certain habits that have to do with our culture are keeping us unhealthy.

In order to get people to come into the door, you have to understand their culture.

Because everything that I do is wrapped around my culture. I think more people need to be educated on how to communicate with not just Indians but other minorities too.

I mean, I don't see why you would really have to invest money in that. I mean, people know their culture; people know their beliefs.

Like it sounds like this category is basically like how are we going to be sensitive to everybody? Well, as long as everybody is getting care—like the care is what's important, not the sensitivity so much—as long as somebody can come in and get the care that they need.

We don't live in a very diverse population, so the need for that is not as great as it might be in another area.

The deliberations about allocating funds to improve research ranged widely. Some argued for it as a priority to improve accountability and the quality of research. Others reasoned that the existing systems were sufficient or that investing in improving research would not necessarily constrain unethical research:

I think my argument for funding it more heavily was that at a higher, bigger‐picture level, you know, for instance, like a governmental level, you've got all of these billions of dollars going into these different areas, and you want to know that it's being spent a little wisely.

To me, it's research about making research better.… [T]hat, to me, could impact positively every area of research.

I read some studies that I would call “meta” studies where people have looked at a large group of publications in a certain area and found very interesting things, that certain things were being studied, certain things were not. Certain things that are being done depending on who funds the study, you get very different results and stuff like that.

It's probably not going to be funded unless it's good research, so I'm not sure. It just seems redundant is all I am saying.

If you are doing research, you are doing research on how to improve research at the same time. Your methodology is always going to be to improve it, hopefully—unless you are indifferent.

Even though it is important to improve research and make it more ethical, how does that make these pharmaceutical companies believe that, OK, if they can't do it here in the states, they are going to take their experiments outside the country and do it where they can do it, so there's nothing that's going to stop them.

I think there's enough things overseeing research now as it is.… You have your IRBs, and you have all of them watching over what's happening in research and you have to answer back to them.

I generally tend to think that most research that's being done is ethical.

Discussion

In this large mixed‐methods study of deliberative procedures applied to setting health‐related priorities, groups and individuals strongly prioritized child health and mental health research, often choosing to invest at the highest possible level. Greater investment also might have been prioritized had it been available (ie, if they could have invested more resources than needed for level 3). The public often prioritizes the health needs of children,24 citing the importance of a healthy start, investment in the future, children's vulnerability, and the acceptance that with age comes illness. The priority given to mental health research, however, could signal a change in awareness of and attitudes toward mental health, which has suffered from stigma and misconceptions.25, 26, 27 Individuals increased their allocation to mental health research after the group deliberations. Previous work has shown deliberation can influence health priorities,28 including the priority given to mental health.29 The deliberators articulated reasons for investing in mental health research, such as the importance of mental well‐being and a lack of knowledge about mental health.

Health disparities research, surprisingly, was not highly prioritized by groups from minority and underserved communities. Based on the deliberators’ cited reasons, this could be because the category was interpreted as research describing or finding disparities and not research addressing or ameliorating them. Since access was the only research category more likely to be selected by individuals after deliberation, compared with before deliberation, perhaps minority and underserved community members consider research on how to improve access to care as a way to address health disparities. As one deliberator said, “I think everything should be accessible to anyone that needs it. No matter your gender, your race, your income.” Comparative effectiveness research, which many funders have highly prioritized, did not receive priority from our deliberators. Health policy research and improving research were also low priorities, although interestingly, those living under the federal poverty level were more likely to select them.

Although most of the relationships between the participants’ characteristics and their priorities were modest (eg, age and aging research), they did support the validity of this process for measuring health research priorities.29 Mental health and culture and beliefs research were notable exceptions to these modest relationships. It is possible that the lower priority given to mental health research by African Americans comes from the disparity they have experienced with mental health services,30 a disparity attributed to various factors, such as the small number of mental health professionals of color, concerns about stigma, and a history of misdiagnosis, maltreatment, and distrust. The range of priority given to culture and beliefs may reflect, in part, the substantial number of Native American and Arab American participants, who constituted a majority in several focus groups. The discussion also revealed the range of opinions, with statements representing starkly contrasting views (eg, “In order to get people to come into the door, you have to understand their culture” compared with “as long as everybody is getting care—like the care is what's important, not the sensitivity so much—as long as somebody can come in and get the care that they need”).

Those identifying as Black/African American were more likely to prioritize communication research, which doubtless has some overlap with research on culture and beliefs. Priority given to culture and beliefs and communication research by some illustrates the importance of paying special attention to unique preferences, and not just commonly held views, when getting public input on spending priorities.

Proportions and relationships should be interpreted cautiously, however, given that the sampling did not try to be statistically representative. Random sampling during recruitment does not predictably lead to proportional representation in face‐to‐face deliberations. Instead, we aimed to oversample those groups typically underrepresented in both research and policy decision making, and we had an excellent representation of minority and medically underserved populations. Similarly, although we adjusted for possible clustering by group in analyzing individuals’ post‐deliberation priorities, the groups’ priorities may have been influenced by the size or composition of the particular group.

Participants ranged in educational attainment and age, and about half were at or below 200% of the federal poverty level. Women were overrepresented, as is often true in research engaging minority and underserved populations.31, 32 The overrepresentation of women could have influenced the process of decision making, since a group's gender composition influences, for instance, the degree of cooperation or competitiveness. A group's gender composition also could have influenced outcomes, as the female democratic and collaborative approach has been shown to make it easier to find and achieve consensus close to median preferences.33 But evidence shows that the typical gender gap in deliberative participation weakens when women are in the majority or there is a decision rule requiring unanimity, so the overrepresentation of women in our project may have provided more balance in voice and authority.34 Because we did not find any differences in individual priority selections by gender, the overrepresentation of women probably did not dramatically affect the groups’ priorities. Finally, we collected the data in 2015, and because the Flint (Michigan) water crisis came to national attention in mid‐2015, it may well have influenced the priority for healthy environment research.

How do the priorities of deliberating groups from minority and underserved communities compare with how health research dollars are allocated? While the National Institutes of Health's (NIH's) budgets allocated less money to the National Institute of Child Health and Human Development and to the National Institute of Mental Health than to other institutes, there are no easy ways to compare spending on the categories in this project with NIH's budgets or other sources of spending, including foundations, other governmental agencies, and for‐profit and not‐for‐profit private entities.35 Spending on, for instance, research on allergies and infectious diseases likely includes conditions that affect children, and an agency devoted to cancer may also fund research on mental health. Still, the overwhelming priority given by our participants—as individuals and as groups—to research on child health and mental health provides evidence that could justify taking a hard look at how spending in those areas compares with spending in others and could motivate placing a greater priority on those domains. The emphasis, for many funders of health disparities research, including the National Institute of Minority Health and Health Disparities, on interventions rather than documentation of differences would likely appeal to our deliberators. Although they did not highly prioritize health policy research, they did prioritize access research and, in deliberations, often mentioned the possible impact (for good or ill) of policies on access to care. The lower priority given by laypersons to comparative effectiveness research, which has been highly prioritized by clinical and policy experts, raises questions about how to address decisions when experts and community stakeholders disagree, a very important unresolved matter for future research. There are many determinants of decisions about research funding, and we argue that the perspectives of those communities with a stake in the research findings should be given more attention in funding decisions.36

Gaming and simulation in policy rarely emphasize the voices of minority and low‐income communities. Our project has illustrated the potential of combining participative methodology with gaming and simulation to allow persons with different perspectives to communicate (“multilogue”) about a complex topic.37, 38 At least 4 of the “5 Cs” articulated by Geurts, Duke, and Vermeulen are illustrated in this project.38 Complexity certainly exists in setting spending priorities for health research, both technical‐physical complexity and social‐political19 complexity. Creativity contributed to the participatory process of tailoring the CHAT exercise to the task of setting health research spending priorities. Open discussion including diverse points of view illustrates the impact of the game on communication. Perhaps most important, this process provided the “level playing field” and space for negotiation to enable a fair process for building a consensus. Future work should emphasize the importance of a commitment to action, that is, an examination of the impact of public engagement in priority setting on actual allocation decisions. Ultimately, innovative partnerships among policymakers, researchers, and minority and disadvantaged communities could lead to research that better meets their needs. Future research should examine not just whether but how communities are engaged in research and what impact, if any, particular methods of community engagement have on community health and well‐being.

Future use of CHAT could adapt it to various priority‐setting contexts, adding or subtracting categories to better reflect the needs of a population or the mission of a funder, or creating content that engages the public in deliberations about priorities within a particular type of research (eg, mental health research).

Supporting information

Online Appendix, Table 1. Research Categories and Levels in CHAT

Online Appendix, Figure 1. Focus Group Locations and Medically Underserved Populations Map

Online Appendix, Table 2. Survey Questions Related to Participant Demographics