Demonstrating Patterns in the Views Of Stakeholders Regarding Ethically-Salient Issues in Clinical Research: A Novel Use of Graphical Models in Empirical Ethics Inquiry

Abstract

Background

Empirical ethics inquiry works from the notion that stakeholder perspectives are necessary for gauging the ethical acceptability of human studies and assuring that research aligns with societal expectations. Although common, studies involving different populations often entail comparisons of trends that problematize the interpretation of results. Using graphical model selection – a technique aimed at transcending limitations of conventional methods – this report presents data on the ethics of clinical research with two objectives: (1) to display the patterns of views held by ill and healthy individuals in clinical research as a test of the study’s original hypothesis and (2) to introduce graphical model selection as a key analytic tool for ethics research.

Methods

In this IRB-approved, NIH-funded project, data were collected from 60 mentally ill and 43 physically ill clinical research protocol volunteers, 47 healthy protocol-consented participants, and 29 healthy individuals without research protocol experience. Respondents were queried on the ethical acceptability of research involving people with mental and physical illness (i.e., cancer, HIV, depression, schizophrenia, and post-traumatic stress disorder) and non-illness related sources of vulnerability (e.g., age, class, gender, ethnicity). Using a statistical algorithm, we selected graphical models to display interrelationships among responses to questions.

Results

Both mentally and physically ill protocol volunteers revealed a high degree of connectivity among ethically-salient perspectives. Healthy participants, irrespective of research protocol experience, revealed patterns of views that were not highly connected.

Conclusion

Between ill and healthy protocol participants, the pattern of views is vastly different. Experience with illness was tied to dense connectivity, whereas healthy individuals expressed views with sparse connections. In offering a nuanced perspective on the interrelation of ethically relevant responses, graphical model selection has the potential to bring new insights to the field of ethics.

INTRODUCTION

Ethical controversy persists over the acceptability of human studies that involve people with severe, chronic, and debilitating mental and physical illnesses (Candilis et al. 2008; Grisso et al. 1995; Palmer et al. 2005). Some leaders in the field have suggested that people living with mental illness, such as schizophrenia or depression, are especially vulnerable in the context of clinical research (Appelbaum et al. 1999; Dunn and Roberts 2005; Jeste, Depp, and Palmer 2006). Moreover, evidence-based ethics researchers have posited that physically and mentally ill individuals may not fully appreciate aspects of the research situation as different from usual clinical care, and may not fully understand the scientific objectives of the clinical-investigator as distinct from the patient-centered goals of the clinician (i.e., “therapeutic misconception”) (Appelbaum, Lidz, and Grisso 2004; Lidz et al. 2004; Miller and Brody 2003).

A further concern is that ill individuals may also be greatly influenced by factors that are subjective in nature and that shape, or even drive, their research participation decisions – and yet these factors may remain untouched or unexplored in discussions of informed consent, which conventionally focus on information disclosure. Such subjective factors include personal motivations, expectations, symptoms, cognitions, and emotions that are related to an underlying illness, and these factors may be especially prominent at certain points in an illness process or stage of life. Such factors may anchor (e.g., altruism, self-awareness) or, more worrisome, may undermine (e.g., desperation, helplessness) authentic consent for research participation (National Bioethics Advisory Commission 2001; Roberts and Dyer 2004). When attitudes and motivations are based in potentially negative aspects of an illness experience, they may distort perspectives related to clinical research and introduce an added source of vulnerability in the research situation (Macklin 2003; Menikoff 2009; Shamoo 1996).

Over the past four decades, researchers have studied many aspects of informed consent for clinical research, but relatively little attention has been given to the subjective influences shaping the consent decisions of those who are ill and choose to volunteer in clinical research (Carpenter et al. 2000; Hindmarch, Hotopf, and Owen 2013; Sugarman et al. 1999; Wirshing et al. 1998). Moreover, cross-cutting studies comparing the attitudes expressed by physically ill, mentally ill, and healthy individuals are rare (Roberts and Kim 2014). Another unknown is how the views of individuals without experience in clinical research differ from those who have volunteered already for such studies.

For the current project, we investigated data from a “cross-cutting study” with a diverse set of ill and healthy stakeholders, and hypothesized that clinical research participants who live with serious illness express their world views toward the ethical acceptability of human studies in a fundamentally different manner from those without illness. Roberts and Kim (2014) investigated the differences in views expressed by these respondents in relation to attitudes on research participation.

In this manuscript, we focus on the connections (and therefore “dependence” in a probabilistic sense) amongst views expressed by ill individuals and healthy individuals regarding the ethical acceptability of human studies. We explore whether the pattern of views of mentally ill volunteers has similarities to those of physically ill and of healthy volunteers from a new angle. Our approach uses “graphical model selection,” a statistical method that assesses the underlying dependence of variables, which in our case are subjective factors. This approach allows the weight and richness of connections evident among stakeholder perspectives to be displayed in a way that is immediately visually accessible. The visual nature of these models enables us to assess whether expressed views can cluster into smaller common themes. We have elected to analyze the data in our hypothesis-driven study in this novel manner because it represents visually and structurally the perspectives emerging from the intrinsic meanings tied to the experience of being ill.

METHODS

Study Population

Eligible study participants included volunteers in clinical protocols at the University of New Mexico School of Medicine and its affiliated VA hospital with a concurrent primary diagnosis of schizophrenia, depression or anxiety, cancer, or HIV, as well as those who were in good health. Healthy comparison subjects, who were either enrolled in a clinical protocol or not enrolled in a clinical research protocol, were recruited at the University of New Mexico Health Sciences Center and various clinics. Participants with active substance dependence were excluded from the study.

In total, 179 individuals volunteered for the study: 43% (n=60) of whom were living with mental illness (i.e., schizophrenia; depression or anxiety disorder); 29% (n=43) of whom were living with physical illness (i.e., cancer, HIV/AIDS, diabetes); and 31% (n=76) of whom were in good health. Of the 76 healthy study participants, 47 had given protocol consent (i.e., previously participated in research) and 29 were protocol-naive.

Survey Instrument and Data

We created and pilot-tested a questionnaire to assess views of ethically important aspects of clinical research based on our previous work and the existing research ethics and informed consent literatures. The resultant 301-item survey was administered along with an open-ended structured interview of 23 questions, which covered a number of domains regarding the importance of medical research, subjective views on research participation, and influences and perceptions of risk.

For this paper, we focus on a subsection of the survey – 14 items concerning the personal attitudes of participants concerning the ethical acceptability of clinical research. Respondents were queried on the ethical acceptability of research participation on a 5-point scale (from 1 = “absolutely not acceptable” to 5 = “absolutely acceptable”), prompted by the question: “How acceptable is it for the following kinds of people to take part in medical research?”

Variables of interest were respondent ratings of ethical acceptability of research including patients with particular illnesses (i.e., cancer, HIV, depression, schizophrenia, or post-traumatic stress disorder [PTSD]) and ratings of ethical acceptability of research in the context of vulnerability due to illness-related sources (i.e., those very sick or dying, in a lot of physical pain, emotional pain, living in health care facilities) and non-illness related sources (i.e., children, elders, being poor, ethnic minorities).¹

Procedure

We obtained written informed consent for our project. No information from our project was shared with the clinical research teams. This study was approved by the institutional review board at the University of New Mexico School of Medicine and funded by the National Institute of Mental Health (NIMH) and the National Institute on Drug Abuse (NIDA). Interviews for this “piggy back” project were conducted within 7 days of participants” informed consent disclosure sessions for their respective clinical protocols. The survey and interview session was completed, on average, in 1.5–2 hours. Participants received $25 compensation.

Statistical Methods

To address the primary aim of assessing how perspectives expressed by participants depend on other ethically salient perspectives expressed, we used graphical model selection.

Probabilistic Graphical Models

Undirected probabilistic graphical models are visual displays of relationships among variables, allowing one to infer the conditional independence between variables by a simple visual inspection of the graph.

In graphical models, random variables are visually represented by “nodes” and two nodes may be connected by “edges.” When two nodes are connected by an edge, the edge signifies statistical meaning, namely that the two random variables are conditionally dependent given the rest of the variables present in the model. Conversely, when two nodes are not connected by an edge they are independent, conditional on the rest of the variables. More generally, if two sets of nodes are separated by a third distinct set of nodes, than first two sets of nodes are independent of each other given the third distinct set. For the sake of simplicity, we will use the label of the node (e.g., “cancer”) to refer to the variable of interest (i.e., “ethical acceptability of research involving people living with cancer as rated by the respondent”).

Graphical models can be measured in terms of topological metrics that describe relations between nodes. Density is simply the number of edges (delineating dependent relationships) divided by the total number of edges possible in the graphical model, representing the degree of connectivity in the model.

Another simple measure is the “degree” of a node, which is defined as the number of edges that emanate from a given node. Degree can be used to distinguish between nodes that are well-connected and others that are not as well-connected. Nodes with a high degree, for example, play a more significant role in the model due to their increased connectivity to other nodes. Node “importance,” therefore, is a measure that can be evaluated using degree as a defining criterion. Other criteria can be also used to define node importance, such as “betweenness centrality,” which is widely used in the graphical modeling literature. Using this criterion, importance is measured in terms of the number of shortest paths passing between other nodes and through the node considered (Brandes 2001; Edwards 2000).

Finally, modularity is a property of a graphical model that describes the type of structure of the graphical model. Specifically, modularity describes the decomposability of the graphical model into smaller subgroups of nodes that may have many connections to other nodes within the subgroup but few connections to nodes outside the subgroup. A model with high modularity for example, will have dense connections between the nodes within a particular module but sparse connections between nodes belonging to other modules. Modularity can be both positive and negative, with positive values indicating the possible presence of structure with a graphical model. (For an example, please see Figure 1, which depicts examples of graphical models with high and low modularity.)

Figure 1.

Graphical models with high and low modularity

Statistical Analysis

All statistical summaries and graphical model selection were performed using R software (R version 3.0.0, GNU project). We selected graphical models using an efficient algorithm called the “constrained l-1 minimization for inverse matrix estimation” (CLIME) proposed by Cai, Liu, and Luo (2011) via an R package “clime.” For each stakeholder group, we took responses (i.e., ratings of ethical acceptability of research) and standardized responses to have mean zero and unit variance. We then selected graphical models to represent the pattern of dependence among these perspectives. Specifically, this involved calculating variance-covariance matrices from data matrices of responses and applying the “CLIME” algorithm to estimate the inverse of the covariance matrix. Regularization parameters were chosen by determining the optimal [value] parameter that minimized a log likelihood loss function from a 5-fold cross validation procedure.

For each graphical model, we computed the following summary measures: total number of edges, density, node importance, and modularity.

In each of the graphical models we present, the size of the labels of the nodes are proportional to its node importance as defined by between-ness centrality. Line thickness corresponds to the respective weights in the graphical model.

Comparison of Graphical Models

The rest of this paper compares the graphical models selected for each of the stakeholder groups in the following comparisons: (a) healthy protocol participants versus all respondents living with serious illness (i.e., cancer, diabetes, HIV, depression/anxiety, and schizophrenia); (b) mentally ill versus physical ill protocol participants; (c) healthy individuals with research protocol-experience or those who are protocol-naive (see Figure 2).

Figure 2.

Concept map: schematic diagram of study design

Specifically, we compare graphical models by obtaining and summarizing the attributes of the graphical models described in the previous section, which confer statistical meaning regarding the dependence structure. The graphical model framework thus provides an appealing and intuitive interface to confer underlying formal statistical meaning. For example, we juxtapose total size (number of edges) of graphical models in comparison type (a) to infer the density of connections in the overall structure of patterns. We do the same with other characteristics mentioned above, such as modularity.

To statistically compare characteristics between graphical models, we performed a simulation study and bootstrapped the null distribution using a data resampling technique. By bootstrapping the null distribution we are able to determine whether differences between characteristics of graphical models were statistically meaningful, and correspondingly we provide p-values based on the statistical test of no difference for a given characteristic.

RESULTS

Characteristics of Study Population

The mean age was 44.5 years (standard deviation [SD] 13.3). A majority of respondents were white (56.5%), with the remainder Hispanic (29.9%) or other non-specified minorities (13.6%). Table 1 summarizes key characteristics of this study population.

Table 1.

Study population characteristics

	Mentally Ill in Protocol N=60	Physically Ill in Protocol N=43	Healthy in Protocol N=47	Healthy Not in Protocol N=29	p-value
Disease type (n)
Cancer	---	28% (12)	---	---
Diabetes	---	33% (14)	---	---
HIV		39% (17)	---	---
Anxiety mood disorder	42% (25)	---	---	---
Schizophrenia	58% (35)	---	---	---
Sex (n)
Women	67% (40)	56% (24)	45% (19)	55% (16)	0.16
Men	33% (20)	44% (19)	55% (26)	45% (13)
Mean Age (SD), years	42.2 (9.9)	47.1 (12.3)	44.9 (16.9)	35.9 (15.3)	0.14
Race^ (n)
Hispanic	25% (15)	47% (20)	19% (9)	31% (9)	0.05
Other	23% (14)	9% (4)	9% (4)	17% (5)
White	50% (30)	44% (19)	72% (34)	52% (15)
Brief Symptom Inventory (BSI) Global Severity Index* (SD)	1.14 (0.86)	0.64 (0.70)	0.28 (0.29)	0.38+ (0.38)	< 0.001
Short Form Health Survey (SF-36): current health subscale* (SD)	53.12 (22.11)	49.64 (25.65)	76.81 (19.26)	72.5 (26.59)	<0.001

Note. SD = standard deviation.

^{^}Missing observations (m). Mentally ill: m=1

^*Missing observations (m). Mentally ill, physically ill: m=4; healthy not in protocol: m=1 with exception ⁺m=2.

Comparison of Graphical Models for Ill and Healthy Participants

Overall Patterns in the Graphical Models

Seriously ill and healthy protocol participants revealed vastly distinct patterns of perspectives, as demonstrated in Figure 3. The most visually prominent difference is that the number of edges in the graphical model for ill participants exceeds that of healthy participants (p=0.01, for statistical test of no difference in number of edges).

Figure 3. Graphical models for (A) ill protocol participants vs. (B) healthy protocol participants.

Description: These graphical models show a clear and discernible pattern; ill participants perceive ethical acceptability as highly dependent concepts compared to healthy participants, who express a modular view.

The acceptability of research including individuals in pain, as endorsed by the ill group, was dependent on their endorsements of research including people with mental and physical illness. In the graphical model, “emotional pain” was connected to “depression,” “schizophrenia,” “cancer,” and “physical pain” (Figure 3A). Strikingly, for healthy individuals, “very sick” was not connected to “mental illnesses.” The views of healthy respondents revealed that “physical pain,” “emotional pain,” and “sick” were connected; but “pain” was conditionally independent of nodes of mental illnesses (i.e., “schizophrenia,” “depression,” “PTSD”) given their views on “cancer” (Figure 3B).

It is structurally evident that the graphical model for healthy research participants splits into at least one distinct subgroup comprised of a trio of nodes describing severe pain and illness, namely “emotional pain,” “physical pain,” and “very sick.” Perspectives expressed by ill participants showed no pattern particularly amenable to segregation. In fact, we observed that the graphical model resulting from the healthy cohort of protocol participants featured the highest degree of modularity (0.66); in contrast, the model derived from all ill protocol participants featured a very low degree of modularity (0.33) (p=0.01 for statistical test of no difference in modularity) (Figure 3).

Local Patterns in Graphical Models

In the graphical model for seriously ill participants, a large subset of nodes are revealed to be highly important, including “physical pain,” “living in facilities,” “ethnic minorities,” the “very sick,” ”PTSD,” “cancer,” and “HIV.” The graphical model for healthy participants shows that only “women” is a highly significant node.

Comparison of Graphical Models for Mentally Ill and Physically Ill Participants

Overall in the Graphical Model

Graphical models for mentally ill and physically ill protocol participants displayed similar and richly connected patterns (Figure 4). On the whole, mentally ill participants revealed patterns in their perspectives where nodes labeled by mental illness were densely connected to other nodes, and to a lesser extent this was also true of patterns for physically ill participants (p=0.30, for statistical test of no difference in number of edges). In the model for mentally ill participants, “schizophrenia” was dependent upon 7 nodes in total, which included all physical and mental illness nodes, namely “HIV,” “cancer,” “depression,” and “PTSD” (Figure 4a). In the graphical model for physically ill participants, “schizophrenia” was dependent on five other nodes in total, including “PTSD” and “depression,” but no nodes labeled by physical illnesses.

Figure 4. Graphical models for (A) mentally ill versus (B) physically ill protocol participants.

Description: These graphical models show that both groups perceive ethical concepts as highly dependent.

The interrelated perspectives of physically ill participants were slightly more agreeable to separation than they were for protocol participants diagnosed with mental illness (modularity = 0.31 vs. 0.18; p=0.79, for statistical test of no difference in modularity).

Local Patterns in the Graphical Model

The graphical model for mentally ill participants (Figure 4B) resulted in two highly important nodes: the ethical acceptability of human studies involving “women” (of child-bearing age) and “facilities” (i.e., institutionalized).

Comparison of Graphical Models for Protocol-Consented or Protocol-Naïve Healthy Volunteers

Overall Patterns in the Graphical Model

Healthy participants, irrespective of research protocol experience, revealed views that were not highly connected (p=0.39, for statistical test of no difference in number of edges) (Figure 5). The density of connections is moderately low for the two graphical models (in protocol = 0.23 vs. not in protocol = 0.26).

Figure 5. Graphical models for healthy individuals who are (A) protocol-consented versus (B) protocol-naïve.

Description: These graphical models show similar patterns of modularity.

Both graphical models also share a common pattern in that “emotional pain,” “physical pain,” and “sick” are highly connected. From protocol-naïve participants emerged a pattern where “minority” and “facility” separate the triad (“emotional pain,” “physical pain,” and “sick”) from the other nodes. In particular, “pain” is conditionally independent of age and gender related nodes (e.g., “old,” “child,” “women,” “poor”) given “facility.” Another finding worth noting is that the “pain” triad is conditionally independent of all physical and mental illness nodes (i.e., “cancer,” “PTSD,” “depression,” “HIV,” and “schizophrenia”) given “facility.”

Healthy protocol-consented individuals revealed patterns of perspectives that were moderately amenable to separation into different groups. Healthy protocol-naïve individuals showed a similar pattern but to a slightly lesser degree (0.47) (p=0.19, p for statistical test of no difference in modularity).

Local Patterns in the Graphical Model

The node “women” (for women of child-bearing age) is featured as a highly “important node” in the graphical model selected for the healthy, protocol-consented participants (Figure 5A). On the other hand, the graphical model selected for healthy individuals who are protocol-naive shows that the node “women” is only connected to “poor.”

DISCUSSION

Focusing on subjective influences on research enrollment decisions from the point of view of clinical research participants helps to fulfill the foundational principles of respect for persons, beneficence, and justice that underlie ethically sound human studies. Though aspirational and abstract, these three principles (“The Belmont principles”) are expressed in the concrete, everyday safeguards that are incorporated into clinical investigation of diseases or their treatments. Ideally, these safeguards will be informed by stakeholders of clinical research, including participants, investigators, family members, IRB members, and policy makers.

Over the past four decades, evidence-based ethics researchers have sought to understand the perspectives of these diverse stakeholders in the process of clinical investigation. This work is predicated on the belief that discerning stakeholder perspectives is valuable – some would argue necessary – for gauging the ethical acceptability of human studies, and for assuring that clinical research is conducted in alignment with societal expectations.

To discern the nature of perspectives held by clinical research participants – arguably the most crucial stakeholders – we sought to assess the structural patterns of views regarding ethical issues in research participation expressed by diverse groups of ill and healthy protocol volunteers with the use of a technique called probabilistic graphical modeling, which provides a representation of the dependence in relationships between one variable with another. This powerful class of models captures the structure of connectivity and dependence of stakeholder responses. Through graphical models, we revealed the density of connections in responses offered by individuals drawn from different participant groups (i.e. physically ill, mentally ill, and healthy). Our findings using graphical models provide support to the study’s original hypothesis: subjective factors are more intricately connected and dependent among ill individuals than among healthy individuals enrolled in clinical research.

Indeed, the primary finding that emerged from this study is that the pattern of views regarding ethical acceptability of clinical research, as expressed by ill protocol participants, is fundamentally and vastly different from that expressed by healthy protocol participants in both degrees of connectivity and modularity. Moreover, healthy individuals in research protocols are more similar to other healthy individuals (i.e., protocol-naïve healthy individuals) than they are to ill individuals in research protocols. Being in a research protocol does not appear to be associated with differences in the patterns of interrelationships. However, it is the experience of living with severe illness (versus being healthy), given participation in a protocol, that appears to shape the views expressed. More simply stated, these findings are evidence that ill protocol volunteers perceive far deeper connections than do healthy volunteers, demonstrating that the experience of illness relates to a far more complex outlook on ethical matters on research participation.

The secondary finding that emerged from this study concerns the patterns of dependence among perspectives. The endorsement of physically ill participants regarding the ethical acceptability of research involving people with schizophrenia was dependent on their endorsements of the ethical acceptability of research on other mental illnesses (i.e., PTSD, depression), but not related to their endorsements of acceptability of research involving people with physical illness. Healthy protocol participants were also able to separate ethical concepts from others in their own thematic manner, for example, dissociating the acceptability of research that involves “very sick” people from that of research that involves mentally ill individuals.

In light of these empirical findings, there are several implications for the future direction of conducting ethically sound clinical research. One is that the rich connectivity of subjective beliefs may be more salient in the outlook and ethically relevant decision making of seriously ill people. Another implication is the need for greater attention to subjective influences related to participation decisions when constructing patient safeguards. We may question, based on this evidence that relates illness to a richly interwoven perspective on research participation, whether an information-oriented approach to consent is positioned well enough to fulfill the goals of patient-centered safeguards. Current, more information-oriented approaches to consent are necessary but may not be sufficient to achieve authentic consent for people with severe illness. When seeking informed consent from potential volunteers in clinical research studies, for example, a careful effort to clarify assumptions, motivations, concerns, and issues of the individual considering participation is extremely important. Research involving people with severe illness ideally will have safeguards tailored to the experience of being ill, demonstrating a greater regard and sensitivity for the potential vulnerabilities present in the research enrollment situation.

Limitations

Our results should be interpreted in light of the limitations of the study, which include its reliance on self-report items, novelty of the questions that were posed, the potential of bias related to self-disclosure of attitudes on sensitive topics, and the inclusion of individuals from a single geographic location. Demographic differences were found between groups, but demographic information was not accounted for in the models due to the nature of the graphical modeling method we employed.

The question of whether graphical models may be different by virtue of demographics or by group membership may merit further investigation in future research. Protocol-specific data were not ascertained in this study, but do merit consideration for future inquiry, as such data could be used to examine how the dependence structure of views varies in relation to the study-specific risk levels.

CONCLUSIONS

In Roberts and Kim (2014), overall directional patterns of the ethical acceptability of clinical research were investigated using data from this cross-cutting “piggy-back” study. What emerged was a remarkable congruence in views across groups, where physically and mentally ill individuals expressed similar and positive attitudes toward research. It was also suggested that individuals with their own immediate experience in protocols view clinical research favorably and see engagement of ill and many other potentially vulnerable people in research as ethically acceptable. In view of these previous findings, our conclusions in this current paper present a far more nuanced and deeper view into the interrelationships and dependence structures.

A prominent strength of this study is the utility and value in this powerful class of models. The very purpose of ethical stakeholder research studies such as ours is to recognize patterns of perspectives of ethical salience. While conventional methods including MANOVA and regression models and other multiple testing procedures provide necessary information through summarizing and identifying overall directional trends, the use of graphical models elucidates the interrelationships among all outcomes and addresses the core intent of this ethical inquiry. Studies involving many different groups (i.e., participants, community members, and other “stakeholders”) are common in empirical ethics research, but entail multiple comparisons that make the interpretation of results potentially problematic. The value of our method lies in the advantage of understanding stakeholder data in a manner that summarizes the structure and confers intrinsic and experiential meaning underneath expressed perspectives on clinical research, with the potential to elucidate areas that are only vaguely understood – if not entirely neglected.

Providing new critical insight into ethical practice, this study may call into question the ways in which current practices of informed consent and patient-centered safeguards demonstrate a regard for seriously ill individuals. Results from this study will also motivate inquiry into areas in need of better attunement to participant experiences. We suggest, moreover, that the use of graphical modeling techniques represents a significant contribution to the field of empirical ethics. Through the use of graphical modeling techniques, we found that seriously ill individuals have a distinct outlook on ethically salient matters, which otherwise could not have been concluded by examining marginal trends with more traditional analytic methods. This method allows us to investigate a nuanced perspective on how ethically relevant responses are interrelated, and thus has the potential to bring new insights to the field of ethics.