Abstract
Objective
To learn how minority and underserved communities would set priorities for patient‐centered outcomes research (PCOR).
Data Sources
Sixteen groups (n = 183) from minority and underserved communities in two states deliberated about PCOR priorities using the simulation exercise CHoosing All Together (CHAT). Most participants were minority, one‐third reported income <$10,000, and one‐fourth reported fair/poor health.
Design
Academic–community partnerships adapted CHAT for PCOR priority setting using existing research agendas and interviews with community leaders, clinicians, and key informants.
Data Collection
Tablet‐based CHAT collected demographic information, individual priorities before and after group deliberation, and groups' priorities.
Principal Findings
Individuals and groups prioritized research on Quality of Life, Patient‐Doctor, Access, Special Needs, and (by total resources spent) Compare Approaches. Those with less than a high school education were less likely to prioritize New Approaches, Patient‐Doctor, Quality of Life, and Families/Caregivers. Blacks were less likely to prioritize research on Causes of Disease, New Approaches, and Compare Approaches than whites. Compare Approaches, Special Needs, Access, and Families/Caregivers were significantly more likely to be selected by individuals after compared to before deliberation.
Conclusions
Members of underserved communities, in informed deliberations, prioritized research on Quality of Life, Patient‐Doctor, Special Needs, Access, and Compare Approaches.
Keywords: Patient‐centered outcomes research, resource allocation, minority groups, decision making, research priorities
Patient‐centered outcomes research (PCOR) recognizes that policy makers, scientists, and clinicians must incorporate patients' and potential patients' views and values for the priorities of research as well as their views of health outcomes and processes (Patient Protection and Affordable Care Act 2010). Should we prioritize research on common diseases over rare ones? Quality of life over prolonging life? Preventing “bad” outcomes, curing minor ailments, improving or restoring basic human functioning, or relieving suffering? How should we trade off research that tests promising interventions and research that aims to improve delivery of proven interventions? These tradeoffs require attention to justice and science. Justice is enhanced by the participation in decision making, and the leadership, of those most affected by the decisions (Goold 1996; Fleck 2001; Vayena 2014). Engaging patients and the public in priority setting can illuminate and inform decisions and make the PCOR agenda more just, more accountable, and more responsive to patients' needs and values.
Yet how to engage communities in priority setting has been a challenge. Traditional methods of engagement, such as polling or focus groups, are useful primarily for accessing individuals' “top of the head” considerations on issues where the public has preexisting informed opinions (Solomon and Abelson 2012). Community engagement about PCOR demands a different approach for several reasons. First, the public is unlikely to hold preexisting informed opinions about PCOR. It is also a complicated policy area where developing an opinion requires substantial learning, including learning about the views and experiences of others. Finally, setting priorities for research supported by public resources to create the public good of knowledge means group judgments, rather than simply individual opinions, have relevance (Rowe et al. 2010).
Deliberative methods of community engagement offer one solution to these challenges. Deliberative strategies have been justified by appeals to develop a more informed public (Fishkin 1997), create decisional legitimacy (Cohen 1997), and/or claim that participants in deliberations and their constituents have consented to informed decisions (Fleck 1992). In general, deliberative procedures call for gathering nonprofessional (lay) members of the public to learn and deliberate about a topic with the intention of forming a policy recommendation or casting an informed vote. Deliberation goes beyond mere dialog or focus groups by adding reasoning through various positions and a task for the group (Solomon and Abelson 2012). Deliberative procedures may be appropriate when: (1) the informed opinions of nonexperts provide essential information experts do not have; (2) informed opinions are difficult to obtain; (3) individual opinions will benefit from group discussion and insight; and/or (4) group judgments are relevant. All of these conditions apply to the task of setting research priorities.
To educate and engage diverse members of the public in priority setting for PCOR, we adapted an existing deliberation exercise, CHAT (originally Choosing Health Plans Altogether, now CHoosing All Together), to facilitate deliberative priority setting constrained by limited resources. Designed based on theories of deliberative democracy, CHAT aims to promote reasoned dialog about complex and value‐laden allocation decisions among ordinary persons in an inclusive, informative, and engaging manner (Burkhalter, Gastil, and Kelshaw 2002). Research has demonstrated that participation in CHAT influences individuals' understanding and opinions about health‐related priority setting, and it has found evidence of public‐spiritedness (Goold et al. 2005; Danis, Ginsburg, and Goold 2010).
Here, we report the PCOR priorities chosen by minority and underserved communities in two states, how priorities changed after group deliberation, and report characteristics of deliberators associated with their priorities.
Methods
CHoosing All Together asks participants to prioritize spending across different categories, presenting participants with information about the consequences of greater or lesser levels of spending in each category. We developed this content specific to PCOR by:
Reviewing documents from scientific, professional, and public entities that describe PCOR, including public comments about priorities where those were available.
Interviewing key informants at entities that conduct and/or support PCOR about how they categorize types of research and set priorities, how to present options and relative costs, and what public input they would find valuable.
Interviewing physicians, predominantly in underserved areas, about what PCOR needs they see in their practices. For PCOR, physicians and patients comprise a key audience.
Interviewing community leaders with experience in research, especially from minority and underserved communities. Interviews began with open‐ended questions about types of research and then sought comment on categories identified in (2) and (3).
Content was designed to be credible, sufficient, and comprehensible to a lay audience. Final content (which includes definitions and explanations of a number of scientific terms) was at approximately a seventh‐grade reading level. All content was translated into Spanish.
Options were designed to avoid bias and to reflect both current PCOR priorities and other options, so as to yield decisions useful to decision makers but not constrained by the status quo. To accomplish this, our team included community partners from more than 10 diverse medically underserved communities and researchers familiar with the funding priorities of PCORI, NIH, and others. The academic and community partners used input from steps 1 through 4 (above), particularly responses to open‐ended questions asked of clinicians in underserved areas and community leaders familiar with research, to collaboratively develop content. This content was iteratively reviewed by the entire team for bias, comprehensibility, relevance to funding agencies, and openness to priorities identified in steps 3 and 4 above, particularly where these priorities differed from those generally prioritized by funding agencies. For example, the category Multiple Conditions was frequently mentioned in interviews with clinicians in underserved areas, and community leaders and partners agreed this was an important category of PCOR. The Families/Caregivers category was developed from interviews with community partners and leaders. All content was iteratively reviewed by academic and community partners for bias, comprehensibility, and relevance to both community members and research funders.
Because we were asking laypersons to deliberate about a topic, PCOR, about which they would not be expected to have much baseline knowledge, sessions began with a brief video describing what research is, how it is funded, and who currently decides what research questions are asked.1 The video then presented the goal of PCORI (i.e., to involve patients in decision making) and connected that goal to their task of setting PCOR priorities. Background was designed to be as neutral as possible to avoid shaping participants' priorities; for instance, no information was presented about why certain priorities (such as health disparities) might be important.
CHoosing All Together presents participants with an interactive, online game board that resembles a pie chart (see Figure 1). Each wedge of the circle represents a category of PCOR spending, and each wedge has different levels of spending (including the option of no spending at all). Thirteen categories of PCOR each had up to three levels that could be selected (see Figure 1), with higher levels associated with more research and higher cost. Costs assigned to different levels of possible spending within categories reflected knowledge gained from key informants, for instance that there would be at least some fixed costs associated with funding research within a category. The first (lowest) level of spending in every category needed to reflect both the cost of the research (grants) and some fixed costs (infrastructure, personnel). Levels were described to reflect, in general terms, how the increased spending would be used for that type of research. For instance, research might be done on larger or more diverse populations, might cover more topics, or might involve testing interventions rather than describing problems. Given the challenges of estimating relative costs for research categories and levels, descriptions of different levels of spending and estimates of resources were based on current levels of funding and interviews with leaders of research institutions. Added categories (e.g., Multiple Conditions) were assigned the minimum number of markers. Categories and levels are described in Appendix SA2. Near the end of deliberations, and in post‐CHAT surveys, participants are asked questions about what choices they would have liked to see presented or presented differently.
Figure 1.
PCOR‐CHAT Game Board from November 1, 2013, to March 30, 2014
Participants choose the level of funding for each category by allocating markers required for the level they choose. However, participants are given a limited number of markers, and thus must make tradeoffs between these categories—choosing high levels of funding in one category requires lower levels of funding in another. Participants first set priorities as individuals, then in groups of 2–4, then with the entire group (up to 16), and repeat individual choices at the end. During the exercise, the group hears and discusses scenarios (“events”) that illustrate the consequences of the priorities they chose. Events were developed based on real‐life events and vetted by community leaders, researchers, and leaders of research institutions. Participants learn from the video, other members of the group, the illustrative events, and embedded resources, and are asked to make fair decisions on behalf of fellow community members.
Sample
We convened 16 focus groups of 4–15 participants (total n = 183), with most groups containing between 10 and 12 participants. Participants were recruited in minority and underserved communities in Michigan and Missouri using flyers and local advertising in English and Spanish, and through personal contacts. Volunteers were excluded if they were health care professionals or researchers, or under 18. We aimed to recruit approximately equal numbers of men and women, and to have disproportionate representation of minority and low‐income residents. Three focus groups were conducted in Spanish. Focus groups were convened in locations familiar to and convenient for participants (e.g., churches) to encourage attendance and maximize open and frank dialog.
Data Collection
Pre‐ and postdeliberation surveys measured demographic and health characteristics. CHAT software recorded which categories and levels were selected by individuals initially and after group deliberation, and which categories and levels are prioritized by the group.
Analysis
Participant characteristics were described using proportions for categorical variables (e.g., race) and means and standard deviations for continuous variables (e.g., age). Poverty level was calculated using income (ranges) and the number identified by respondents as living in their household; the upper portion of the income range they identified was used so the portion under the federal poverty level represents a conservative (under) estimate. We describe individuals choosing each of the 13 research priorities using proportions calculated both before and after deliberation, and calculate also the percentage of groups selecting each priority.
The effect of deliberation on individual priority selection was measured using odds ratios accounting for within‐individual paired responses (i.e., individuals' predeliberation responses and postdeliberation responses), and the significance of the changes in the selection for each priority was assessed using a multilevel logistic regression model with priority selection as the response variable and with deliberation groups and individuals nested within deliberation groups included as random intercepts to adjust for potential clustering within‐groups and within‐individuals. Similarly, the effect of deliberation on changes in the level selected was tested using multilevel regression model with level changes as response variable and group as random intercepts to adjust for within‐group clustering. A multilevel logistic regression with groups as random intercepts was used to obtain estimates for independent associations between each priority selection and both various individual‐level (e.g., age) and group‐level characteristics (e.g., urban vs. rural). Each priority model always included age, race, rural residence, and gender. The remaining variables were selected based on hypotheses about what might predict priorities (e.g., income, knowledge of research, views of health disparities), and they were retained in final models based on statistical significance (p < .05) or large magnitude of the association. All analyses used Stata 13.1 (StataCorp. 2013).
Results
Deliberators ranged in age from 18 to 83 years, with over half (61 percent) women and about one‐quarter residing in a rural area (Table 1). About one‐third of deliberators were white, and one‐half black/African American. Data collection about ethnicity encountered technical difficulties leading to missed responses for 53 participants, but three focus groups conducted in Spanish included 32 participants (17.5 percent) and, in English‐speaking groups, an additional nine reported Hispanic/Latino ethnicity, so at least 41 (22.4 percent) participants were Hispanic/Latino, and possibly as high as 31 percent (if missing data are excluded from the denominator). Most participants (71.6 percent) had incomes less than $35,000; combining income with number in household, at least 39.2 percent were under the federal poverty level. Over one‐third (37.3 percent) reported fair or poor health status.
Table 1.
Participant Characteristics (N = 183)
| Participant Characteristics | N (%) |
|---|---|
| State of residence | |
| Michigan | 105 (57.4) |
| Missouri | 78 (42.6) |
| Female (n = 183) | 112 (61.2) |
| Age in years, mean (SD; range), n = 178 | 46.4 (14.7; 18–83) |
| Race (n = 183) | |
| White | 63 (34.4) |
| Black or African American | 98 (53.6) |
| Othera | 22 (12.0) |
| Hispanics (CHAT in Spanish or self‐identified Hispanics, n = 183) | 41 (22.4) |
| Education (n = 182) | |
| High school/GED or less | 72 (39.6) |
| Some college | 63 (34.6) |
| Bachelor's degree | 24 (13.2) |
| More than bachelor's degree | 23 (12.6) |
| Rural (vs. urban region, n = 172) | 45 (26.2) |
| Income (n = 174) | |
| Less than $15,000 | 67 (39.0) |
| $15,000 to $34,999 | 56 (32.6) |
| $35,000 or more | 49 (28.5) |
| No. of people in household (mean (SD; range), n = 177) | 2.9 (1.7; 1–11) |
| ≤Federal poverty level (n = 171) | 67 (39.2) |
| Living alone (n = 177) | 43 (24.3) |
| Perceived health status (n = 177) | |
| Fair or poor | 66 (37.3) |
| Good | 50 (28.3) |
| Very good or excellent | 61 (34.5) |
Cell values are N(%) unless otherwise described. N does not add to 183 when some responses are missing.
Other includes other race and mixed race.
Research Priorities Selected by Individuals
Over 80 percent of individuals, prior to group deliberations, allocated at least some markers to Causes of Disease, Quality of Life, Patient‐Doctor, and Promote Health (Table 2). After group deliberations, over 80 percent of individuals selected these same four categories; additionally, 80 percent allocated at least some markers to Health Inequity, Access, and Special Needs. Of the 13 research priority categories, Compare Approaches (OR = 2.53; 95 percent CI = (1.44, 4.44); p = .001), Special Needs (OR = 2.40; 95 percent CI = (1.15, 5.00); p = .02), Access (OR = 2.31; 95 percent CI = (1.21, 4.44); p = .01), and Families/Caregivers (OR = 2.00; 95 percent CI = (1.14, 3.53); p = .02) categories of research were all significantly more likely to be selected at least the minimum level after deliberation than they were before deliberation. No categories were less likely to be selected after deliberation.
Table 2.
Individual Priority Selections of Level in Research Categories and Markers Allocated
| Priority | Selected, n (%) | Selected Levela | Markers | |||
|---|---|---|---|---|---|---|
| 1, n (%) | 2, n (%) | 3, n (%) | Markers Needed to Select [Level 1, 2, 3] | Spent per Person | ||
| Round 1 (N = 182) | ||||||
| Causes of Disease | 160 (87.9) | 39 (24.4) | 43 (26.9) | 78 (48.8) | [2, 3, 4] | 2.9 |
| New Approaches | 138 (75.8) | 46 (33.3) | 43 (31.2) | 49 (35.5) | [4, 6, 8] | 4.6 |
| Promote Health | 155 (85.2) | 38 (24.5) | 65 (41.9) | 52 (33.6) | [2, 3, 5] | 2.9 |
| Compare Approaches | 91 (50.0) | 51 (56.0) | 26 (28.6) | 14 (15.4) | [7, 11, 13] | 4.5 |
| Patient‐Doctor | 159 (87.4) | 80 (50.3) | 50 (31.5) | 29 (18.2) | [6, 9, 11] | 7.3 |
| Quality of Life | 154 (84.6) | 49 (31.8) | 44 (28.6) | 61 (39.6) | [3, 4, 6] | 3.8 |
| Health Inequity | 141 (77.5) | 46 (32.6) | 48 (34.0) | 47 (33.3) | [3, 4, 6] | 3.9 |
| Multiple Conditions | 140 (76.9) | 35 (25.0) | 41 (29.3) | 64 (45.7) | [2, 3, 4] | 2.5 |
| Special Needs | 141 (77.5) | 38 (27.0) | 46 (32.6) | 57 (40.4) | [2, 3, 4] | 2.4 |
| Families/Caregivers | 124 (68.1) | 40 (32.3) | 43 (34.7) | 41 (33.1) | [2, 3, 4] | 2.0 |
| Access | 139 (76.4) | 46 (33.1) | 44 (31.7) | 49 (35.3) | [2, 3, 4] | 2.3 |
| Improve Research | 136 (74.7) | 63 (46.3) | 43 (31.6) | 30 (22.1) | [5, 7, 8] | 4.7 |
| Rare Diseases | 125 (68.7) | 54 (43.2) | 43 (34.4) | 28 (22.4) | [2, 3, 4] | 1.9 |
| Round 4 (N = 168) | ||||||
| Causes of Disease | 143 (85.1) | 31 (21.7) | 42 (29.4) | 70 (49.0) | [2, 3, 4] | 2.8 |
| New Approaches | 122 (72.6) | 57 (46.7) | 40 (32.8) | 25 (20.5) | [4, 6, 8] | 4.0 |
| Promote Health | 139 (82.7) | 23 (16.6) | 58 (41.7) | 58 (41.7) | [2, 3, 5] | 3.0 |
| Compare Approaches | 112 (66.7) | 75 (67.0) | 19 (17.0) | 18 (16.1) | [7, 11, 13] | 5.8 |
| Patient‐Doctor | 145 (86.3) | 56 (38.6) | 48 (33.1) | 41 (28.3) | [6, 9, 11] | 7.3 |
| Quality of Life | 144 (85.7) | 44 (30.6) | 54 (37.5) | 46 (31.9) | [3, 4, 6] | 3.7 |
| Health Inequity | 138 (82.1) | 45 (32.6) | 52 (37.7) | 41 (29.7) | [3, 4, 6] | 3.5 |
| Multiple Conditions | 133 (79.2) | 42 (31.6) | 48 (36.1) | 43 (32.3) | [2, 3, 4] | 2.4 |
| Special Needs | 144 (85.7) | 31 (21.5) | 60 (41.7) | 53 (36.8) | [2, 3, 4] | 2.7 |
| Families/Caregivers | 132 (78.6) | 30 (22.7) | 51 (38.6) | 51 (38.6) | [2, 3, 4] | 2.5 |
| Access | 143 (85.1) | 41 (28.7) | 36 (25.2) | 66 (46.2) | [2, 3, 4] | 2.7 |
| Improve Research | 122 (72.6) | 47 (38.5) | 43 (35.3) | 32 (26.2) | [5, 7, 8] | 4.7 |
| Rare Diseases | 124 (73.8) | 46 (37.1) | 44 (35.5) | 34 (27.4) | [2, 3, 4] | 2.1 |
Percentages are calculated as individuals choosing the level out of those who selected the priority.
In addition to changing which categories received any funding, individuals changed the level of investment by a significant amount in three categories (Table 3). Families/Caregivers saw an average increase of 0.33 levels (p = .03), while Access saw an average increase of 0.35 levels (p = .01). Only the New Approaches category saw a statistically significant decrease in funding level, with an average decrease of 0.28 levels (p = .05). Combined with the changes in categories selected, these results suggest that deliberation contributed to a partial shifting of priorities, with less focus on developing new approaches and more focus in particular on improving access and recognizing the particular difficulties and challenges faced by some patients and their families.
Table 3.
Within‐Participant Investment Level Changes from before Deliberation to after Deliberation for Each Priority, n (%)
| Priority | Investment Level Change as “Postdeliberation Minus Predeliberation” | Changeb (p‐value) | ||||||
|---|---|---|---|---|---|---|---|---|
| Lower Level at Postdeliberation | Higher Level at Postdeliberation | |||||||
| −3 | −2 | −1 | 0a | 1 | 2 | 3 | ||
| Causes of Disease | 9 (5.4) | 13 (7.8) | 25 (15.0) | 75 (44.9) | 23 (13.8) | 17 (10.2) | 5 (3.0) | −.04 (.73) |
| New approaches | 11 (6.6) | 23 (13.8) | 38 (22.8) | 54 (32.3) | 21 (12.6) | 10 (6.0) | 10 (6.0) | −.28 (.05) |
| Promote Health | 2 (1.2) | 12 (7.2) | 29 (17.4) | 75 (44.9) | 30 (18.0) | 17 (10.2) | 2 (1.2) | .07 (.45) |
| Compare Approaches | 2 (1.2) | 10 (6.0) | 25 (15.0) | 74 (44.3) | 39 (23.4) | 10 (6.0) | 7 (4.2) | .17 (.06) |
| Patient‐Doctor | 3 (1.8) | 11 (6.6) | 21 (12.6) | 71 (42.5) | 44 (26.4) | 14 (8.4) | 3 (1.8) | .17 (.05) |
| Quality of Life | 8 (4.8) | 11 (6.6) | 28 (16.8) | 79 (47.2) | 25 (15.0) | 7 (4.2) | 9 (5.4) | −.05 (.60) |
| Health Inequity | 7 (4.2) | 15 (9.0) | 20 (12.0) | 71 (42.5) | 35 (21.0) | 14 (8.4) | 5 (3.0) | .04 (.72) |
| Multiple Conditions | 9 (4.8) | 16 (9.6) | 32 (19.2) | 65 (38.9) | 26 (15.6) | 14 (8.4) | 6 (3.6) | −.10 (.37) |
| Special Needs | 3 (1.8) | 9 (5.4) | 31 (18.6) | 64 (38.3) | 37 (22.2) | 16 (9.6) | 7 (4.2) | .19 (.07) |
| Families/Caregivers | 6 (3.6) | 9 (5.4) | 21 (12.6) | 64 (38.3) | 36 (21.6) | 17 (10.2) | 14 (8.4) | .33 (.03) |
| Access | 6 (3.6) | 9 (5.4) | 28 (16.8) | 59 (35.3) | 28 (16.8) | 17 (10.2) | 20 (12.0) | .35 (.01) |
| Improve Research | 7 (4.2) | 13 (7.8) | 27 (16.2) | 64 (38.3) | 40 (24.0) | 7 (4.2) | 9 (5.4) | .04 (.68) |
| Rare Diseases | 5 (3.0) | 12 (7.2) | 19 (11.4) | 69 (41.3) | 40 (24.0) | 19 (11.4) | 3 (1.8) | .17 (.07) |
Includes those who did not select the priority at both rounds.
Mean change in the investment level, where the change is calculated as after deliberation level minus before deliberation level; positive values correspond to higher investment level selection after deliberation, and p‐values are adjusted for within‐CHAT group clustering using multilevel regression model.
Examining the number of markers allocated to different categories, Patient‐Doctor research had the largest allocation from individuals both before and after informed group deliberations (Table 2), and Compare Approaches had the second largest allocation of markers. Of note, Patient‐Doctor research and Compare Approaches categories required more markers than others to be selected at all (at level 1; Table 2, Markers needed to select).
Research Priorities Selected by Groups
Each of the 16 groups allocated at least some resources to Quality of Life, Patient‐Doctor, and Access categories of PCOR. Less than 70 percent of the groups allocated resources for Compare Approaches (7 of 16; 44 percent), Health Inequity (10 of 16; 63 percent), Multiple Conditions (11 of 16; 69 percent), and Rare Disease (11 of 16; 69 percent) (see Table 4). Of note, although individuals were more likely to select Compare Approaches after deliberation, still only 67 percent of the individuals selected the priority postdeliberation while 44 percent of 16 groups selected that category.
Table 4.
Individual Versus Group Selections
| Priority | Individual | Level Selected by Groups (number)c | ||||||
|---|---|---|---|---|---|---|---|---|
| Pre | Post | |||||||
| Deliberation | Groupa | Group≠Postb | ||||||
| N = 182 (%) | N = 168 (%) | N = 16 (%) | N = 168 (%) | 0 | 1 | 2 | 3 | |
| Causes of Disease | 87.9 | 85.1 | 93.8 | 24.2 | 1 | 1 | 2 | 12 |
| New Approaches | 75.8 | 72.6 | 81.3 | 36.3 | 3 | 6 | 6 | 1 |
| Promote Health | 85.2 | 82.7 | 81.3 | 33.0 | 3 | 2 | 2 | 9 |
| Compare Approaches | 50.0 | 66.7 | 43.8 | 37.9 | 9 | 6 | 1 | 0 |
| Patient‐Doctor | 87.4 | 86.3 | 100.0 | 20.3 | 0 | 3 | 9 | 4 |
| Quality of Life | 84.6 | 85.7 | 100.0 | 20.9 | 0 | 1 | 8 | 7 |
| Health Inequity | 77.5 | 82.1 | 62.5 | 44.0 | 6 | 3 | 4 | 3 |
| Multiple Conditions | 76.9 | 79.2 | 68.8 | 37.4 | 5 | 5 | 1 | 5 |
| Special Needs | 77.5 | 85.7 | 93.8 | 23.6 | 1 | 1 | 2 | 12 |
| Families/Caregivers | 68.1 | 78.6 | 87.5 | 28.6 | 2 | 4 | 4 | 6 |
| Access | 76.4 | 85.1 | 100.0 | 21.4 | 0 | 2 | 8 | 6 |
| Improve Research | 74.7 | 72.6 | 75.0 | 26.9 | 4 | 3 | 5 | 4 |
| Rare Diseases | 68.7 | 73.8 | 68.8 | 39.0 | 5 | 5 | 2 | 4 |
Percentage of groups that chose each priority.
Percentage of individuals (N = 168) whose choices in Round 4 were different from the choice made by the group.
Number of groups choosing each level in that priority.
Predictors of Priority Selection
We found no significant relationship between priorities selected and gender, residence in rural versus urban setting, ethnicity, incomes below the federal poverty level, or health status. Those with less than a high school education were less likely to prioritize New Approaches, Patient‐Doctor, Quality of Life, and Families/Caregivers (Table 5). Blacks and those of other races were less likely to prioritize research on Causes of Disease than whites. Blacks were also less likely to prioritize New Approaches and Compare Approaches than whites, and those of other races were less likely to prioritize Causes of Disease than whites. Of note, several patient characteristics were potentially associated with priority category, as indicated by large magnitudes of association, but were only marginally significant.
Table 5.
Participant Characteristics Predicting Priority Selection at CHAT Round Postdeliberation (Adjusted Odds Ratios)
| Priority | Age | Age2 | Raceb | Hispanic c | ≤High School | MI | FPL | Know | Views on Health Disparityd | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Black | Other Race | Disc | Gen | Pers | Job | Ins | ||||||||
| Causes of Disease | 0.95a | 0.998a | 0.06a | 0.05a | – | – | – | – | – | – | – | – | – | – |
| New Approaches | 0.99 | – | 0.25a | 0.40 | – | 0.44a | – | – | – | – | – | – | – | – |
| Promote Health | 0.96a | 0.998a | 0.31 | 0.37 | – | 0.47 | – | – | – | – | – | – | – | – |
| Compare Approaches | 1.00 | – | 0.12a | 0.62 | 0.17a | – | – | 3.61a | 3.53 | – | 0.56a | – | – | – |
| Patient‐Doctor | 0.99 | – | 0.42 | 0.47 | – | 0.36a | – | – | – | – | – | – | – | – |
| Quality of Life | 0.99 | – | 0.64 | 0.80 | – | 0.40a | – | – | – | – | – | – | – | – |
| Health Inequity | 0.96 | 0.998a | 0.44 | 0.15 | 0.29a | – | – | – | – | 2.21a | – | – | – | – |
| Multiple Conditions | 0.98 | – | 2.02 | 0.33 | 0.28 | 0.55 | .21 | – | – | – | 0.52a | – | – | – |
| Special Needs | 0.97a | – | 0.51 | 1.08 | – | 0.50 | – | – | – | – | – | – | – | – |
| Families/Caregivers | 0.96a | – | 1.13 | 0.70 | – | 0.38a | – | – | – | – | – | – | – | – |
| Access | 0.96a | – | 1.35 | 0.48 | – | – | – | – | – | – | – | 2.68a | 0.32a | 2.47a |
| Improve Research | 0.98 | – | 0.46 | 1.31 | 0.23 | – | – | – | – | – | – | – | – | – |
| Rare Diseases | 0.98 | – | 1.45 | 1.43 | – | – | – | – | – | – | – | – | – | – |
Models are fit separately for each priority, with selecting the priority as the response variable adjusted for clustering within CHAT group and covariates listed above. Age, race, rural residence, and gender were always included, although rural residence and gender were not predictive of any priority selections. Income, self‐perceived health status, CHAT done in Spanish, living alone, and Trust in Medical Researchers were not predictive of any priority selection and thus were not included in the final models. Final model included significant predictors (p < .05) or those with large magnitudes. Age is centered at 50 years old; an adjusted odds ratio (OR) of Age less than 1.0 indicates decreasing likelihood of selecting the priority with increasing age. OR <1.0 for both Age and Age2 indicate decreasing likelihood of selecting the priority with increasing age, but the rate of decreasing likelihood decreases with increasing age for participants older than 50. “–” refers to variables not predictive of the priority selection and thus not included in the model.
p < .05.
Reference is white race.
Hispanic ethnicity (Hispanics are either self‐identified Hispanics or CHAT done in Spanish (n = 183).
Refer to Table 2 for detailed description and other various possible explanations to views on health disparity; Sup is support, Disc is discrimination, Gen is genetics, Pers is personal, Ins is insurance.
LEHS is ≤high school education (Ref: >high school), MI is Michigan (Ref: Missouri), FPL is federal poverty level status, and Know indicates for a response of “all of it” to a self‐assessed knowledge question (“Overall, if you were to read a newspaper account or hear on TV, a breakthrough in the health research, how much of it do you think you would be able to understand”).
Discussion
Sixteen groups of residents from minority and underserved communities deliberated about priorities for PCOR. They prioritized, as groups and as individuals, research on Quality of Life, Patient‐Doctor, Access, Special Needs, and (by total resources spent) Compare Approaches. Less priority was given to New Approaches, Improving Research and Causes of Disease. As has been found in other work asking patients or members of the public about priorities for research, priorities of our deliberators differ from those typically found in research institutions and funding agencies (Tallon, Chard, and Dieppe 2000; National Science Foundation 2010), and lend support to the mission and types of research supported by PCORI. An emphasis on discovering new interventions is, arguably, less consistent with the mission of PCOR to help patients and doctors make decisions about existing discoveries.
Somewhat surprisingly, given the predominance of minority and underserved community members, deliberators did not prioritize Health Inequity research. On the other hand, the priority given to Quality of Life, Patient‐Doctor relations, Access, and Special Needs might reflect lived experiences of minority and underserved populations, who can be disproportionately affected by these problems and may have less experience with or access to new, cutting‐edge discoveries.
We found few relationships between demographic characteristics and priorities. Those in the oldest age group (>70 years old) were, surprisingly, less likely to prioritize Multiple Conditions and Families/Caregivers research than the youngest age group (≤30 years old). While we expected that rural residents might prioritize Access more than urban residents, this was not found. Analysis of dialog, currently in process, may illuminate access challenges in different locales.
Participation in deliberation changed participants' funding priorities. After deliberating, more individuals allocated research funding in the areas of Special Needs, Families/Caregivers, Access, and Compare Approaches, perhaps reflecting a better understanding, after group deliberation, of the potential impact of research in those areas for themselves and others. Individuals “paid for” these increases by making small reductions in a number of other areas, with the largest reduction for the New Approaches category. To the extent that changes resulted from a high‐quality deliberative process, these suggest that deliberative consideration of research funding priorities produces changes in the way that individuals prioritize research spending across different areas. Changes may reflect individual learning from the exercise itself, or from other members of the group, or could reflect a response to reasons articulated by others during deliberation. Future work will examine deliberative dialog for evidence of reasoning, learning, and other elements of deliberation quality.
Compare Approaches and Patient‐Doctor relations required the largest quantity of resources for even the minimum level (level 1) of research funding. Given the challenges of estimating relative costs for research categories and levels, estimates of resources were based on current levels of funding. In retrospect, using current levels of funding rather than estimates of the relative cost of different types of research (imprecise though that may be), probably biased deliberators against choosing Compare Approaches. However, another category, Patient‐Doctor, was also costly and yet was still highly prioritized by individuals and groups. We would recommend, for future work, populating levels of all categories with similar numbers of markers, with some adjustment based on rough estimates of the relative cost of different types of research. It would be interesting to see if that would affect the priority given to Compare Approaches research by deliberators, particularly given the priority given to that type of research by PCORI, among other funders.
Conclusion
Asked to make fair decisions on behalf of fellow community members, deliberators from minority and underserved urban and rural communities in two states prioritized research on Quality of Life, Special Needs, Patient‐Doctor, and comparative effectiveness. Priorities selected by individuals changed slightly after deliberation. We found education level, race, and age bore some relationship with priorities selected, but we found no significant relationship between priorities selected and gender, residence in rural versus urban setting, ethnicity, incomes below the federal poverty level, or health status. Underrepresented populations were easily and positively engaged in deliberations about PCOR priorities using an interactive device‐based tool. This method could help research institutions, funders, community groups, and advocacy organizations engage patients and stakeholders in research priority setting.
Supporting information
Appendix SA1: Author Matrix.
Appendix SA2: Research Categories and Levels in PCORI‐CHAT.
Acknowledgments
Joint Acknowledgment/Disclosure Statement: Research reported in this article was funded through a Patient‐Centered Outcomes Research Institute (PCORI) Award 1IP2PI000521‐01. The statements presented in this article are solely the responsibility of the author(s) and do not necessarily represent the views of the Patient‐Centered Outcomes Research Institute (PCORI), its Board of Governors, or Methodology Committee. Dr. Goold, Dr. Danis, and their institutions could benefit from future paid licenses (royalties) for the CHAT tool used in this study.
Disclosures: None.
Disclaimers: None.
Note
References
- Burkhalter, S. , Gastil J., and Kelshaw T.. 2002. “A Conceptual Definition and Theoretical Model of Public Deliberation in Small Face to Face Groups.” Communication Theory 12: 398–422. [Google Scholar]
- Cohen, J. 1997. “Deliberation and Democratic Legitimacy” In Deliberative Democracy: Essays on Reasons and Politics, edited by Bohman J.F., and Rehg W., pp. 321–48. Cambridge, MA: The Massachusetts Institute of Technology Press. [Google Scholar]
- Danis, M. , Ginsburg M.M., and Goold S.. 2010. “Experience in the United States with Public Deliberation about Health Insurance Benefits Using the Small Group Decision Exercise, CHAT.” The Journal of Ambulatory Care Management 33 (3): 205. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fishkin, J.F. 1997. The Voice of the People. New Haven, CT: Yale University Press. [Google Scholar]
- Fleck, L.M. 1992. “Just Health Care Rationing: A Democratic Decision Making Approach.” University of Pennsylvania Law Review 140 (5): 1597–636. [PubMed] [Google Scholar]
- Fleck, L.M. . 2001. “Healthcare Justice and Rational Democratic Deliberation.” American Journal of Bioethics 1 (2): 20–1. [DOI] [PubMed] [Google Scholar]
- Goold, S.D. 1996. “Allocating Health Care Resources: Cost Utility Analysis, Informed Democratic Decision Making, or the Veil of Ignorance?” Journal of Health Politics, Policy and Law 21 (1): 69–98. [DOI] [PubMed] [Google Scholar]
- Goold, S.D. , Biddle A.K., Klipp G., Hall C.N., and Danis M.. 2005. “Choosing Healthplans All Together: A Deliberative Exercise for Allocating Limited Health Care Resources.” Journal of Health Politics, Policy and Law 30 (4): 563–602. [DOI] [PubMed] [Google Scholar]
- National Science Foundation . 2010. “Open Government Initiative 2010” [accessed on December 2, 2010]. Available at http://www.nsf.gov/open/
- Patient Protection and Affordable Care Act. 2010.
- PPACA Sec. 6301 D Sec 1181 (d) (1)
- Rowe, G. , Rawsthorne D., Scarpello T., and Dainty J.R.. 2010. “Public Engagement in Research Funding: A Study of Public Capacities and Engagement Methodology.” Public Understanding of Science 19 (2): 225–39, 1: 1‐15. [DOI] [PubMed] [Google Scholar]
- Solomon, S. , and Abelson J.. 2012. “Why and When Should We Use Public Deliberation?” Hastings Center Report 42 (2): 17–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- StataCorp . 2013. Stata Statistical Software: Release 13. College Station, TX: StataCorp LP. [Google Scholar]
- Tallon, D. , Chard J., and Dieppe P.. 2000. “Relation between Agendas of the Research Community and the Research Consumer.” The Lancet 355 (9220): 2037–40. [DOI] [PubMed] [Google Scholar]
- Vayena, E. 2014. “The Next Step in the Patient Revolution: Patients Initiating and Leading Research.” British Medical Journal 349: g4318. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Appendix SA1: Author Matrix.
Appendix SA2: Research Categories and Levels in PCORI‐CHAT.