Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Jan 1.
Published in final edited form as: Am J Intellect Dev Disabil. 2017 Jan;122(1):78–92. doi: 10.1352/1944-7558-122.1.78

What’s the Harm? Harms in Research with Adults with Intellectual Disability

Katherine E McDonald 1, Nicole E Conroy 2, Robert S Olick 3
PMCID: PMC5568892  NIHMSID: NIHMS882454  PMID: 28095059

Abstract

Scientific advances can improve the lives of adults with intellectual disability, yet concerns that research participation may impose harm impede scientific progress. What counts as harmful can be subjective and perceptions of harm may vary among stakeholders. We studied perspectives on the harmfulness of research events among adults with intellectual disability, family members and friends, disability service providers, researchers, and Institutional Review Board members. We found considerable variance. For example, adults with intellectual disability see exclusion from research as more harmful, but most psychosocial harms as less significant than others. All stakeholders agree that having someone else make the participation decision is harmful. Findings provide insights into the concept of harm and ethical research with adults with intellectual disability.

Scientific advances hold great promise for transforming the lives of people with intellectual disability ( Lakin & Turnbull, 2005 ), and the direct participation of adults with intellectual disability in research is essential to fulfilling this promise. However, social and ethical concerns present challenges to their involvement, and put them at risk of exclusion ( Brooker et al., 2014 ; Feldman, Bosett, Collet, & Burnham-Riosa, 2013 ; Northway, 2014 ). For example, the mistreatment and marginalization of people with intellectual disability and their varied degrees of cognitive impairment, give rise to concerns about the ability to make participation decisions and to safely manage research participation experiences ( Fisher, Cea, Davidson, & Fried, 2006 ; Freedman, 2001 ; McDonald, Kidney, Kimmel, et al., 2009 ).

A central ethical component of research design, participation, and consent involves risk assessments, in particular identifying ways research participants may be harmed (negatively affected), as well as the likelihood and significance of potential harm. Risk assessments drive conversations about how to ethically proceed, including whether the study should be carried out, whether particular populations or individuals should be included, and ways to appropriately minimize potential harms. Traditionally, research harms include negative physical, psychological, social, legal, and economic outcomes ( Coleman, Menikoff, Goldner, & Dubler, 2003 ). Risks can be especially challenging to identify in behavioral research given that they may be more subtle and subjective, and are less studied than biomedical research risks ( Labott & Johnson, 2004 ). Moreover, risk assessments can be influenced by perceptions, including beliefs that harms are population-dependent, and populations perceived as vulnerable are often seen as being at risk for different harms and reactions to them. People with intellectual disability have historically been seen as in need of protection, though disability rights activists argue this denies them fundamental human rights (Dybwad, 1996; Perske, 1972 ).

There is emerging evidence that research that includes adults with intellectual disability is assessed differently than research with adults with intellectual disability by researchers and Institutional Review Board (IRB) members ( Lai, Elliott, & Ouellette-Kuntz, 2006 ). The scientific community perceives a higher level of risk when they are included, an effect accelerated when greater potential harm is present; and these views are influenced by attitudes toward their research participation ( McDonald & Keys, 2008 ). The scientific community may also believe that adults with intellectual disability are uniquely vulnerable to psychological harm ( McDonald, Kidney, Kimmel, et al., 2009 ). Though safeguarding participants is paramount, these views are concerning as they may unnecessarily restrict the autonomy of adults with intellectual disability (a violation of human rights), and contribute to their exclusion from research ( Brooker et al., 2014 ; Feldman et al., 2013 ; Northway, 2014 ). Further, views of the scientific community may not fully reflect those of community members, especially those of adults with intellectual disability; divergent views could have significant implications for feelings of respect and willingness to participate in research among community members.

Like the scientific community, family members, close friends and disability service providers are concerned that adults with intellectual disability may be especially sensitive to undue influence and have a hard time understanding information about research. However, adults with intellectual disability indicate that because of prior experiences they have a hard time trusting others, are less concerned about undue influence, and are comfortable declining participation – though they do not want to be pressured. Still, they can find it difficult to assess the legitimacy of a participation opportunity ( Andre-Barron, Strydom, & Hassiotis, 2008 ; Heller, Pederson, & Miller, 1996 ; McDonald, 2012 ; McDonald, Kidney, & Patka, 2012 ; McDonald, Schwartz, Gibbons, & Olick, 2015a ). Collectively, this work suggests others’ concerns for their vulnerability to undue influence are exaggerated, if not misplaced.

Community members also identify unique concerns about harms not strongly shared by the scientific community. Family members, close friends, and disability service providers are concerned about researchers’ abilities to work effectively and safely with people with intellectual disability. Adults with intellectual disability themselves worry they may be treated disrespectfully by researchers, including being treated as a disability or a label rather than as a person ( McDonald et al., 2015a ). Despite the importance of relational dynamics to community members, this is not typically considered in IRB review. Community members are also apprehensive that adults with intellectual disability may suppress thoughts and experiences when others are present during data collection; adults with intellectual disability add that if researchers break confidentiality, they may lose privacy and be at risk for backlash from support providers, a disincentive to share freely as a research participant ( McDonald et al., 2015a ). Encouraging the presence of another person who provides support to the adult with intellectual disability may be more accepted in the scientific community ( McDonald, Kidney, Nelms, et al., 2009 ). However, this practice can blur boundaries of confidentiality for some investigators leading to inappropriate disclosures, notwithstanding that confidentiality of research data is an established ethical and legal norm.

These findings point to areas of potential divergence among stakeholders, and raise the question whether research policy and practice adequately reflect community values and preferences. Such fissures may contribute to greater harm than anticipated by the scientific community, mistrust, and decreased research participation. Although we are gaining a better sense of views on potential harms in research with adults with intellectual disability, we have yet to directly compare views on their harmfulness among scientific and community stakeholders. Because harms have subjective elements, perspectives are especially important. This knowledge is critical to informing how we think about research risks: Adults with intellectual disability may think differently about harms and their impact than others, and this information is needed to direct research policy and practices that reflect community views and facilitate participation that is experienced as safe and respectful ( Roberts, Warner, Hammond, & Dunn, 2006 ). To fill this void, we quantitatively studied the views of adults with intellectual disability, family members and close friends of adults with intellectual disability, disability service providers, researchers and IRB members on potential research harms, and their impact on research participation decisions. We limit potential harms to those relevant to self-report research given the emphasis on studying the thoughts, experiences, and behaviors of adults with intellectual disability, and the challenges associated with making risk assessments therein. Compared to other stakeholders, we anticipated that adults with intellectual disability would perceive greater harm in events that restrict their autonomy and lead them to feel treated differently in research, views consistent with human rights perspectives.

Methods

Participants and Instruments

Five hundred and twelve individuals from 5 stakeholder groups participated in this study: 101 adults with intellectual disability who self-reported their disability, 98 family members and close friends of adults with intellectual disability who had provided them nonpaid support in the previous year (more than 80% were family members), 109 professionals who have provided social services to adults with intellectual disability in the previous year, 105 researchers who have conducted self-report research with adults with intellectual disability in the past five years (about 10% of whom were current students), and 99 IRB members from the previous 2 years from institutions conducting research with adults with intellectual disability. Although some individuals fit criteria for more than one group (40%, 38%, and 12% of each of the latter three groups also had a family member or close friend with an intellectual disability), we privileged professional group membership given the additional opportunities and responsibilities of employment.

In addition to these group-specific inclusion criteria, participants had to be 18 years or older, reside or work in the U.S., and have the communication skills to make a participation decision and express opinions with or without accommodations (see below for more information on accommodations). We were able to communicate with all individuals who were interested in participating; and our inclusion criteria yielded participants from the population who would be able to participate in the research of interest (self-report research). The sample included almost three-fourths females with an average age of 43.5 years old (SD = 14.18), and people from all over the country. Most participants reported being White (87.1%). Some participants reported having additional disabilities or health conditions; mental health conditions, mobility or physical disabilities and ongoing health conditions were the most prevalent across all groups. Most participants indicated having employment, though fewer adults with intellectual disability (52.5%) and family and friends (65.3%) did. Adults with intellectual disability lived in a variety of circumstances. Most had been approved for developmental disability services (74.3%) and received Social Security (85.1%). Family members, friends, and service providers indicated supporting people with mild, moderate, and severe intellectual disability, and some researchers and IRB members indicated that they were also family members of or friends with people with mild to severe intellectual. All groups reported most commonly working and/or having relationships with people with mild to moderate intellectual disability, or people who would likely be able to participate in self-report research. Researchers reported conducting research overwhelmingly with individuals with mild and moderate intellectual disability, though just less than one-half also do so with individuals with severe intellectual disability. Service providers held a variety of positions, and researchers and IRB members were primarily professors (52%) who conduct mostly behavioral or social science research (see Table 1).

Table 1.

Participant demographic characteristics

Intellectual disability (n = 101) Family/ friend (n = 98) Service provider (n = 109) Disability researcher (n =105) IRB Member (n = 99) Total (N = 512)
Variable M (SD) M (SD) M (SD) M (SD) M (SD) M (SD)
Age 43.28 (14.66) 44.81 (14.91) 38.85 (13.41) 40.04 (12.89) 51.05 (11.72) 43.50 (14.18)
Variable % % % % % %
Gender
 Female 49.5 80.6 80.7 85.7 62.6 72.1
 Male 43.6 19.4 19.3 14.3 37.4 26.6
 Other 1.0 0 0 0 0 0.2
Race*
 Amer. Indian/Alaskan Native 14.9 1.0 1.8 1.0 1.0 3.9
 Asian 0 2.0 3.7 7.6 4.0 3.5
 Black/African American 10.9 6.1 5.5 2.9 2.0 5.5
 Hawaiian/Pacific Islander 1.0 0 0.9 0 0 0.4
 White 81.2 87.8 86.2 88.6 91.9 87.1
 Other 3.0 3.1 0.9 0.9 2.0 2.0
Latino 2.0 6.1 3.7 2.9 5.1 3.9
Education
 Never attended school 2.0 0 0 0 0 0.4
 Did not complete high school 21.8 1.0 0 0 0 4.5
 High school diploma/GED 50.5 6.1 5.5 0 0 12.3
 Some college 6.9 14.3 13.8 1.9 2.0 7.8
 Certificate/Associates 7.9 13.3 10.1 1.0 0 6.4
 Bachelor’s 0 24.5 35.8 7.6 4.0 14.6
 Graduate degree 0 38.8 32.1 89.5 93.9 50.8
Disability advocacy 60.4 57.1 72.5 69.5 21.2 56.8
Open in a new tab

Note. For items with an asterisk (*), respondents were asked to select all that apply. Participants could also select “do not wish to say”. As a result, some variables do not add to 100%.

We created the Project ETHICS survey using a multi-prong approach to survey development ( Singleton & Straits, 1999 ; Willis, 2005 ): (1) focus groups ( McDonald & Patka, 2012 ; McDonald, Schwartz, Gibbons, & Olick, 2015b ), (2) prior research ( McDonald, 2012 ; McDonald, Keys, & Henry, 2008 ; McDonald & Kidney, 2012 ; McDonald, Kidney, Kimmel, et al., 2009 ; Roberts, Hammond, Warner, & Lewis, 2004 ; Roberts, Warner, & Brody, 2000 ; Roberts et al., 2002 ; Roberts et al., 2006 ; Rubright, Cary, Karlawish, & Kim, 2011 ), (3) an Expert Panel, and (4) cognitive interviews. At the frontend of the survey, we provided definitions of intellectual disability, self-report research, and researcher, encouraged participants to make their best choice, and explained how to answer questions. The survey also included questions on benefits( McDonald, Conroy, Olick, & Panel, 2016 ) and safeguards( McDonald, Conroy, Olick, Kim, et al., 2016 ) in research though we focus in this manuscript on the following data:

Harms

We asked participants to rate how harmful each of 13 events is for adults with intellectual disability, and how likely they thought adults with intellectual disability would be to take part in research that potentially put them at risk for each harm. We asked adults with intellectual disability how harmful it is to experience each event, and other groups how harmful it is for adults with intellectual disability to experience each event (on a scale of 1–5, higher numbers indicated greater harmfulness and likelihood of participating). Items included harms in behavioral research: Exclusion from research (1 item, harmfulness only), and psychological or social (11 items) and legal (1 item) events. We did not share categorizations with participants, but use them to organize findings to reflect traditional organization of types of harms in research.

Attitude toward Research and Research with Adults with Intellectual Disability

We asked participants to indicate their level of agreement with these 4 statements: (1) research about adults with intellectual disability is very important; (2) adults with intellectual disability should make up their own minds about research participation; (3) adults with intellectual disability can make up their own minds about research participation (on a scale of 1–6, higher levels indicate greater agreement) ( McDonald et al., 2008 ); and (4) I have a positive view about research in general (on a scale of 1–5, higher levels indicate greater agreement) ( Rubright et al., 2011 ).

Personal Information

We asked participants for personal demographic information.

To improve accessibility of the survey, we: (1) included graphic representations to convey thematic content, differentiate among survey sections and show progress towards completion; (2) used plain language and concrete examples; and (3) provided response graphics (e.g., cylinders that were increasingly filled in as values increased, faces with an array of expressions from negative to positive) and named anchors for each extreme value for all scaled items. We also included “do not wish to say” as an option, and asked an open-ended question at the end of each section to capture additional information participants wished to share.

Procedures and Analysis

We received human subjects approval from our university, and designed all materials with a 15 member Expert Panel that included representatives from all five stakeholder groups. We created national sampling frames of groups and organizations by and for adults with intellectual disability, family and friends, and disability service professionals (e.g., social service organizations, advocacy groups, and social or recreational groups such as state developmental disability agencies, state developmental disability councils, national and regional self-advocacy groups, professional associations, etc.). We also created a sampling frame of researchers who had recently published self-report research with adults with intellectual disability, and added names of researchers who had presented at recent conferences. We identified IRB Chairpersons through a Freedom of Information Act (FOIA) request, retaining those from places with active intellectual disability researchers and asking Chairpersons to forward recruitment materials to other members. We supplemented this list for institutions that provided publicly available IRB member rosters. We identified over 500 groups or organizations, 300 researchers, and 500 IRB members to whom we sent recruitment materials.

We recruited nationally via postal mail, electronic mail, social media, tablings, and presentations, using a slightly different approach for each group. Participants recruited via electronic mail received multiple notifications ( Dillman, Smyth, & Christian, 2009 ). We collected data for 4 months in 2014, closing participation for each group as it reached 100 participants. Participants completed the survey online, via postal mail, in-person, or over the telephone with or without additional supports. We further promoted accessibility by providing illustrative graphics in all research materials, allowing participants to take a break, and providing one-on-one support in-person or over the telephone as requested (e.g., reading all materials, answering questions to promote understanding). For consent/assent, we also encouraged participants to take time to make their decision and to discuss their decision with a person of their choosing if desired, and we emphasized voluntariness. We also told online participants to make the decision with their legal guardian when appropriate, and connected with legal guardians via the participant for those who participated in person or over the telephone. We at no time observed anyone who seemed to be demonstrating subtle cues of resistance to participation or who did not understand the decision ( National Institutes of Health, 2009 ) though we could not of course observe online participants directly. All participants received a $40 gift card.

We subjected data to a validation process, retaining those that we determined to be valid across an array of quality indicators (e.g., completion time of at least 20 minutes, verifiable postal address, responses to at least 85% of the items, quality of responses to open-ended comments, internal consistency of responses, self-reported rating of response quality, further contact with respondent to clarify responses). We removed 96 respondents from the final dataset, and only report data on validated surveys. We then cleaned the data; missing data, including responses of “do not wish to say,” was 3.1% and analyses with imputed means yielded identical results. With one exception, the data were negatively skewed, suggesting that participants perceived all items as relatively harmful. Given the exploratory nature of the study, we used a cutoff of p < 0.05 and Tukey’s HSD test for post-hoc comparisons ( Tabachnick & Fidell, 2001 ). We thematically coded qualitative data, using multiple coders to bolster dependability ( Braun & Clarke, 2006 ).

Results

Preliminary Results

We performed a series of preliminary analyses. First, we examined whether there were differences within professional participant groups (i.e., disability service providers, researchers, and IRB members) in perceptions of harmfulness or likelihood to participate in research based on whether individuals also had a family member or friend with an intellectual disability. With the exception of four weakly correlated items, these relationships were not associated with different perceptions. Among family members, friends, disability service providers, researchers, and IRB members, we found that perceptions did not vary based on – or in rare instances were weakly correlated with -- whether the relationship was with a family member or friend. With the exception of two weakly correlated items regarding perceptions of harmfulness, perceptions were also not associated with whether they thought their family member or friend could take the survey. Among family, disability service providers, and researchers, we found weak associations between those who supported or worked with individuals with severe intellectual disability and more harmful perceptions of five of the events, though their ratings of likelihood to participate in research did not differ. Among researchers, we found that being a student or years of research experience were not associated, or were rarely weakly correlated, with perceptions of harmfulness or likelihood to participate in research.

We also examined whether select demographic variables were associated with perceptions of harmfulness or likelihood to participate in research for the overall sample and among participant groups. We found that gender, age, and education were either not associated with perceptions of harmfulness or likelihood adults with intellectual disability would participate in research, or that relationships were weak (most common) to moderate (in no instances did correlations reach .4 or higher). And lastly we found that involvement in disability advocacy, disability rights, beliefs about research in general, or research with adults with intellectual disability were all only weakly (most common) to moderately (again, in very few instances did correlations reach .4 or higher) associated with perceptions of harmfulness or likelihood to participate in research.

We also examined whether general views toward research with intellectual disability were associated with perceptions of harmfulness or likelihood to participate in research for the overall sample and among participant groups. General views about research were only weakly (most common) to moderately (again, in very few instances did correlations reach .4 or higher) associated with perceptions of harmfulness or likelihood to participate in research.

Because none of these variables emerged as meaningfully associated with outcome variables, we did not use any covariates in our focal analyses.

Perceptions of Harmfulness

Between Group Comparisons

All groups agreed that the events count as harms, with one exception: researchers reporting information to the authorities. We performed a one-way between groups ANOVA to measure differences in groups’ perceptions of the degree of harmfulness (see Table 2). We found statistically significant differences in groups’ perceptions of harmfulness for all events, with one exception: adults with intellectual disability feeling worried about what they will have to do as a participant was rated as equally harmful by all stakeholders. Among the significant differences were that adults with intellectual disability and service providers perceived exclusion from research – someone deciding not to tell the adult with intellectual disability about the research study – as more harmful than did family members and friends; and the former two groups and researchers rated exclusion as more harmful than did IRB members. Adults with intellectual disability also perceived the researcher reporting information to the authorities as more harmful than all other groups, who largely saw this as more moderately harmful. Service providers perceived someone else making the decision about research participation as more harmful than did IRB members.

Table 2.

One-way ANOVA between groups differences on perceptions of harmfulness

Intellectual disability (n = 101) a Family/ friend (n = 98) b Service provider (n = 109) c Disability researcher (n =105) d IRB Member (n = 99) e F(df), η2
How harmful is it… M (SD) M (SD) M (SD) M (SD) M (SD)
Exclusion
 Not tell about study 3.94 (1.31)b,e 3.42 (1.17)a,c 3.87 (1.13)b,e 3.80 (1.10)e 3.31 (1.16)a,c,d F(4, 501) = 5.784, η2 = .044
Legal
 Report to authorities 3.82 (1.49)b–e 2.70 (1.39)a 2.57 (1.43)a 2.42 (1.29)a 2.46 (1.32)a F(4, 495) = 17.154, η2 = .122
Psychological and social
 Feel upset 3.32 (1.43)b-e 4.05 (.98)a 4.11 (.98)a 4.09 (.99)a 4.26 (.80)a F(4, 505) = 12.181, η2 = .088
 Feel badly because of treatment 3.70 (1.43)b-e 4.55 (.85)a 4.63 (.68)a 4.75 (.69)a 4.62 (.58)a F(4, 504) = 22.457, η2 = .151
 Feel worried 3.34 (1.51) 3.72 (.94) 3.72 (1.04) 3.72 (1.00) 3.62 (.96) F(4, 505) = 2.193, η2 = .017
 Not to share because someone is at study 3.34 (1.57)b-e 3.81 (1.09)a 4.11 (.87)a 4.09 (1.00)a 3.89 (.97)a F(4, 502) = 7.693, η2 = .058
 Feel treated like a child 4.01 (1.37)b-d 4.45 (.90)a 4.72 (.52)a,e 4.54 (.72)a 4.35 (.76)c F(4, 503) = 9.044, η2 = .067
 Researcher shares personal information 4.24 (1.27)c-e 4.56 (.91) 4.75 (.60)a 4.87 (.42)a 4.84 (.58)a F(4, 504) = 10.316, η2 = .076
 Feel labeled as a disability 4.27 (1.20)b-e 4.71 (.72)a 4.94 (.28)a,e 4.76 (.47)a 4.60 (.65)a,c F(4, 505) = 11.869, η2 = .086
 Others get upset about participation 3.33 (1.55)b-e 3.95 (.97)a 3.93 (.97)a 3.82 (.99)a 3.90 (.94)a F(4, 501) = 5.387, η2 = .041
 Not understand what might happen 3.81 (1.29)d,e 4.05 (1.00) 4.01 (1.02) 4.23 (.88)a 4.32 (.86)a F(4, 504) = 3.862, η2 = .030
 Feel pressured to be 3.84 (1.43)b-e 4.49 (.65)a 4.52 (.73)a 4.59 (.62)a 4.66 (.57)a F(4, 506) = 14.962, η2= .106
 Someone else makes participation decision 4.02 (1.32) 4.03 (.94) 4.28 (.84)e 4.17 (.90) 3.85 (1.01)c F(4, 504) = 2.686, η2 = .021
Open in a new tab

Significantly but not surprisingly, adults with intellectual disability perceived 8 of the 11 psychosocial harms as less harmful than all other groups (the exceptions being feeling worried about what they will have to do, feeling treated like a child, and someone else making the participation decision). Although adults with intellectual disability perceived feeling treated like a child as more harmful than family members, friends, service providers, and researchers, they did not significantly differ from IRB members on this item. The other groups largely agreed on how harmful these potential harms are to adults with intellectual disability. However, service providers perceived feeling treated like a child and feeling labeled as a disability and not a person as more harmful than did IRB members. And IRB members perceived not understanding what might happen in a research study as more harmful than family members and friends and service providers.

Within Group Comparisons

We performed a within group ANOVA to determine statistically significant differences in perceptions of harms within groups (see Table 3). Adults with intellectual disability perceived feeling treated like a child, researchers sharing their personal information with others without their permission, feeling labeled as a disability, and someone else making the decision about research participation as more harmful than feeling upset, feeling worried about what they will have to do, not sharing their thoughts and experiences because someone else is with them, and others getting upset about their participation. They also rated feeling labeled as a disability and not a person as more harmful than feeling badly because of how they are treated and not understanding what might happen in a research study. Among the other groups, feeling badly because of how they are treated, feeling treated like a child, researchers sharing their personal information with others without their permission, feeling labeled as a disability, and feeling pressured to participate are generally viewed as more harmful to adults with intellectual disability than the other events, although there are some exceptions.

Table 3.

One-way ANOVA within group differences in perceptions of harms

Group F(df), partial η2
Adults with intellectual disability F F(13, 74) = 4.402, η2= .436
Family members and friends F(13, 79) = 23.023, η2= .791
Service providers F(13, 94) = 30.730, η2= .810
Researchers F(13, 83) = 34.478, η2= .844
IRB members F(13, 81) = 43.181, η2= .874
Open in a new tab

Predicted Likelihood to Participate in Research with each Harm

Between Group Comparisons

We performed a one-way between groups ANOVA to identify group differences in views about the influence of potentially harmful events on the likelihood of research participation among adults with intellectual disability (see Table 4). Adults with intellectual disability reported greater interest in participating in research than all other groups predicted they would, except when someone else makes the decision about research participation. Still, in this case adults with intellectual disability were more likely to participate than service providers thought they would be.

Table 4.

One-way ANOVA between groups differences in likelihood to participate in research

Intellectual disability (n = 101) a Family/ friend (n = 98) b Service provider (n = 109) c Disability researcher (n =105) d IRB Member (n = 99) e F(df), η2
Likelihood to participate… M (SD) M (SD) M (SD) M (SD) M (SD)
Legal
 Report to authorities 3.28 (1.67)b-e 2.65 (1.08)a 2.53 (1.07)a 2.76 (.97)a 2.70 (1.20)a F(4, 491) = 6.144, η2 = .048
Psychological and social
 Feel upset 3.16 (1.60)b-e 1.72 (1.10)a 1.79 (1.25)a 1.74 (1.52)a 1.55 (.82)a F(4, 503) = 29.366, η2 = .189
 Feel badly because of treatment 2.92 (1.54)b-e 1.61 (1.17)a 1.59 (1.12)a 1.50 (1.08)a 1.41 (1.29)a F(4, 504) = 28.39, η2 = .184
 Feel worried 3.10 (1.44)b-e 2.14 (1.02)a 2.01 (1.00)a 2.02 (.97)a 2.11 (.90)a F(4, 503) = 18.341, η2 = .127
 Not to share because someone is at study 3.19 (1.60)b-e 2.33 (1.01)a 2.25 (1.06)a 2.32 (1.02)a 2.17 (.81)a F(4, 501) = 13.606, η2 = .098
 Feel treated like a child 2.48 (1.58)b-e 1.78 (1.09)a 1.51 (.92)a 1.70 (.98)a 1.68 (.79)a F(4, 503) = 11.803, η2 = .086
 Researcher shares personal information 2.39 (1.56)b-e 1.82 (1.08)a 1.54 (.94)a 1.72 (.98)a 1.54 (.80)a F(4, 503) = 10.134, η2 = .075
 Feel labeled as a disability 2.31 (1.56)b-e 1.60 (1.02)a 1.39 (.79)a 1.65 (1.01)a 1.46 (.77)a F(4, 504) = 11.737, η2 = .085
 Others get upset about participation 2.78 (1.58)b-e 1.99 (.94)a 1.81 (.79)a 1.96 (.99)e 1.78 (.80)a F(4, 501) = 15.132, η2 = .108
 Not understand what might happen 2.82 (1.51)b-e 2.12 (1.02)a 1.96 (1.01)a 2.10 (.98)a 1.89 (.82)a F(4, 501) = 11.645, η2 = .085
 Feel pressured 2.32 (1.47)b-e 1.70 (.86)a 1.62 (.85)a 1.74 (.95)a 1.38 (.55)a F(4, 504) = 12.394, η2= .090
 Someone else makes participation decision 2.40 (1.51)c 2.17 (.98) 1.78 (.92)a 2.08 (.87) 2.01 (.86) F(4, 502) = 4.751, η2 = .036
Open in a new tab

Within Group Comparisons

We performed a one-way within group ANOVA to identify differences in likelihood of participation among adults with intellectual disability, F (12, 73) = 4.85, partial η2 = .444. Adults with intellectual disability indicated greater likelihood to participate in research that might result in feeling upset, feeling worried about what they will have to do, and not sharing their thoughts and experiences because someone else is with them than research that might result in researchers sharing their personal information with others without their permission, feeling labeled as a disability and not a person, feeling pressured to participate, and someone else making the decision about research participation. They were also more likely to take part in research when they might feel worried about what they will have to do or not share their thoughts and experiences because someone else is at the research study with them than if they feel treated like a child. Adults with intellectual disability were also more likely to participate in research that resulted in the researcher reporting information to the authorities than research that resulted in feeling labeled as a disability and not a person, feeling pressured to participate, and someone else making the decision about research participation. Finally, they were more likely to participate if they did not understand what might happen in a research study than if they felt labeled as a disability.

Relationships between Perceptions of Harmfulness and Likelihood to Participate in Research with each Harm

For adults with intellectual disability, we examined the relationship between perceptions of harmfulness and their likelihood of participating in research for each event. Perceptions of harmfulness were weakly correlated to the likelihood of research participation in only two situations: researchers share their personal information with others without their permission (r = -.285, p < .01) and if they felt labeled as a disability and not a person (r = −.200, p < .05).

Qualitative Data

Family members and friends, service providers, researchers, and IRB members noted that it is difficult to evaluate harmfulness without more context (e.g., the nature of the study and relationships), along with the experiences, capacities, knowledge and preferences of those participating, that the line between rights and safety can be difficult to establish, that risks must be balanced against safeguards and benefits, and that the manner in which risk is addressed is especially important. They also noted some events, such as needing to report to the authorities that someone is being hurt, may be harmful in the short-term, but positive in the long-term.

It’s hard to think about folks with intellectual disability as a homogeneous group. They each bring different life experiences. - IRB member

These are difficult questions; much depends on the cognitive capabilities of potential subjects. A subject who was not capable of consenting, but for whom an authorized agent provided consent, will likely react differently than a subject who has the capa[c]ity to consent. - IRB member

I worry about the fine line between protecting people from research harm and making decisions for others/limiting their rights, including the right to make decisions that are different from what I might personally beli[e]ve is best. - Researcher

Research on sensitive topics cannot always be completely pleasant. Some people may not be able to understand that, but others can understand that their discomfort in telling will be helping others. - Family member

Feeling upset … is not necessarily harmful in a long-term sense, as long as the upset is resolved quickly and sensitively through debriefing and support. - IRB member

Discussion

As we grapple with identifying and reducing risk in behavioral research and improving the ethical design and practice of research for once-exploited populations experiencing growing rights and opportunities, understanding research participant perspectives is an essential part of risk assessments ( Dresser, 2014 ). For the first time ever, we studied the views of adults with intellectual disability, and compared them to those of their family and close friends, service providers, researchers, and IRB members. Insight into these potentially divergent views highlights needed changes to research policy and practice, including ways the scientific community can improve outreach to community members. Not surprisingly, we largely saw considerable variance in views on how harmful each harm was, and more tempered interest in participating in research that involved potential harm among adults with intellectual disability.

Adults with intellectual disability and service providers see keeping a research participation opportunity from adults with intellectual disability as more harmful than do family and friends and IRB members, though the effect size is small. Family and friends and IRB members may see this as less harmful because the person will never be aware the gatekeeping has happened, though many would deem this protectionist ( Dybwad, 1996 ). This finding suggests that the two former groups that see gatekeeping as harmful likely favor greater agency for adults with intellectual disability to make their own decisions about their interest in research participation. These preferences are consistent with a human rights approach that values individual capacity and opportunity; it is thus possible these stakeholder groups favor a human rights perspective that prioritizes autonomy and choice, and acceptance of perhaps greater exposure to appropriate risks of harm. This finding also suggests that adults with intellectual disability favor direct recruitment by researchers, which has been found to be more effective than other approaches ( Becker, Roberts, Morrison, & Silver, 2004 ; Cleaver, Ouellette-Kuntz, & Sakar, 2010 ; Iacono & Murray, 2003 ). It should be noted however, that adults with intellectual disability sometimes trust service providers and may welcome them as facilitators of their participation ( McDonald et al., 2015a ). These preferences should be kept in mind in designing research protocols to foster trust and participation.

Adults with intellectual disability see it as more harmful than other groups if researchers need to report information – such as that they are being hurt – to the authorities. Other groups may feel that though reporting may be harmful in the short-term, they see subsequent benefits as measures are taken to keep the individual safe from abuse; this interpretation is supported by participants’ open-ended responses. It is possible this preference is driven by what they see as vulnerability among the population to understand and report on abuse. However, adults with intellectual disability may not fully agree; perhaps they wish to have greater control over handling abuse ( Lewin, 2007 ). Further work will need to be undertaken to disentangle this dynamic, and identify implications for appropriate practices for research studies, especially for researchers also engaged in clinical practice. It is possible that drawing ideas and strategies from relevant clinical and other professional practices may provide useful ideas.

Valid consent is a tenacious challenge, but we must find alternative means to promote understanding, including via environmental modifications, and agency in decision-making as there is little support for exclusion from research. When consent capacity is in question, two approaches have been used. First, a legally authorized representative may give consent, ideally involving the adult with intellectual disability, eliciting their values and preferences to the extent reasonable, and seeking assent ( Arscott, Dagnan, & Kroese, 1998 ; Cameron & Murphy, 2007 ; Iacono & Murray, 2003 ). Second, is a model of supported decision-making that promotes an interactive partnership between the adult with intellectual disability and a trusted support provider ( Kohn & Blumenthal, 2014 ). We found agreement among stakeholders that having someone else make the participation decision is harmful, although IRB members perceived this as less harmful than did service providers. Thus, our findings suggest that surrogate decision-making may be harmful to adults with intellectual disability: Even when adults with intellectual disability have a legally authorized representative, their assent and desire to participate should be given significant weight ( Brown & Thompson, 1997 ; Freedman, 2001 ). By contrast, the supportive decision-making model that is gaining currency may better serve the goals of choice, participation and inclusion, while still offering support ( Iacono & Carling-Jenkins, 2012 ; Kohn & Blumenthal, 2014 ; Northway, 2014 ). More work is needed on the comparative merits of these approaches, including power dynamics that affect decisions by and for adults with intellectual disability. As best practices are identified, IRB members will need to be educated on these shifts and their value.

Adults with intellectual disability saw almost all of the 13 events as less harmful than other groups, and had rather moderate overall interest in participating in research associated with such harms. These findings suggest their strengths as their interest in research participation was more tempered for research that might involve greater risk. They may also reveal the importance placed on being a part of community life, and the desire to accept some degree of risk to grow and contribute ( Dybwad, 1996 ; Perske, 1972 ). Conversely, other stakeholder groups appear to perceive greater fragility among adults with intellectual disability in their ability to successfully handle experiences, as indicated by their perceptions of greater harms, which may lead to favoring the protection of adults with intellectual disability ( McDonald, Kidney, Kimmel, et al., 2009 ). This may be especially true for those who support or work with individuals with severe intellectual disability. Alternatively, these groups may hold these views, at least in part, as a reaction to the mistreatment they see adults with intellectual disability experience and their knowledge of historical mistreatment, informing a desire to prevent further hurt. Future qualitative work can help unpack motivations for these views, and their implications for research policy and practice with adults with intellectual disability. Research policy and practice should keep in mind a strengths-based perspective that acknowledges and promotes capacities in informed consent processes, and the desire of adults with intellectual disability to accept some risk in order to participate in research. Related, family members and friends appear to struggle the most with their dual commitment to promote autonomy while also keeping their loved one safe ( McDonald et al., 2015a ). Educational efforts about how taking risks is a human right and helps people grow, coupled with information about researchers’ responsibilities to participant well-being, may help family members and friends better support their loved ones’ interests in research participation. These findings also reinforce that IRB members are more protective, particularly when potential psychological harms are present. Given their authority over the terms of research participation, these views are especially important to address, and efforts to help IRBs to more fully and accurately understand adults with intellectual disability should be taken so that safeguards are community-supported, respectful, and effective ( Dresser, 2014 ).

Like other stakeholders, adults with intellectual disability perceive some events as more harmful than others. These views are important to address in designing research. For example, researchers should strive to treat adults with intellectual disability as adults and as a person, not simply a label or condition; maintain the confidentiality of their information especially vis-à-vis those providing the participant with support during research participation (upfront, signed agreements may be useful); and ensure that the participation decision belongs to the adult with intellectual disability. Promoting these conditions and experiences will likely generate greater participation in research ( Heller et al., 1996 ), mutually benefitting the quality of the science and fostering efficiency of resources.

Adults with intellectual disability often perceived less harm than other groups, but their views on harmfulness are not correlated (for the most part) with their interest in participating in research. Moreover their interest in participating in potentially harmful research is, overall, rather low, though not nonexistent, yet higher than other groups predicted it would be. It appears adults with intellectual disability are least interested in participating in research that might involve negative social exchanges, decreased agency, and a loss of confidentiality. These findings suggest that the degree of harmfulness adults with intellectual disability perceive in each event is not because of an inability to identify potential harm, but because they see them as relatively less harmful and may favor the benefits of participation over these more minor negative outcomes. Given the benefits of research participation, researchers can leverage strong interest in participation among adults with intellectual disability, but must balance this interest with their responsibility to minimize risk.

Although this study is innovative, there are limitations to note. First, we focus on self-report research and asked participants to make judgments about complex situations without additional information; other types of research, additional contextual information, or alternative research approaches may lead to different conclusions. For example, it is possible that stakeholder views may vary when restricted to adults with severe intellectual disability. It is also possible that despite the information provided, some respondents did not correctly identify the focal population. We had strong representation from female respondents. For some groups, this gender disparity is representative of the gender distribution in caregiving roles and in certain professions, though females are also more likely to participate in research which may contribute to their overrepresentation in our sample ( Galea & Tracy, 2007 ). Gender role socialization theory posits that women may be more accepting of minority groups, and they may simultaneously hold more protective attitudes ( Kerns & Fine, 1994 ; McDonald et al., 2008 ). Although we did not find any gender differences in views of harmfulness, this dynamic may nonetheless matter. We also had about one-tenth of researchers report being students. Although these individuals represent the future of the field, they likely have more limited experience with research. Also of note, we studied perceptions of harmfulness; while perceptions are important to feelings of harm, they may not capture all dimensions of harm nor fully account for factors that impact willingness to participate in research. Finally, it is always difficult to discern the extent to which statistically significant differences will translate to practically meaningful differences. Qualitative comments and prior work suggest these differences may carry weight in reactions to situations and subsequent decision-making.

Science holds great promise for improving the lives of adults with intellectual disability. Achieving this promise demands that we successfully address the social and ethical challenges associated with their inclusion in research. Studying how adults with intellectual disability perceive potential harms in research can inform the design of research protocols that reflect their priorities and interests, strengthen the ethical integrity of scientific inquiry, and expand participation and inclusion. Doing so will benefit both scientific inquiry and adults with intellectual disability. We hope that Project ETHICS can assist in upholding this promise.

Acknowledgments

We thank Colleen Gibbons, Carolyn Kim, Emily LoBraico, Ellis Prather, Lyndsey Creed, Nicole Schwartz, Michael Sperling, The Self-Advocacy Association of New York State - Central Region, and the Madison Motivators for their contributions to this study. We received support for this research from the Eunice Kennedy Shriver National Institute of Child Health & Human Development of the National Institutes of Health under Award Number R21HD075078 (PI: Katherine McDonald). The content is solely ours and does not necessarily represent the official views of the National Institutes of Health.

Contributor Information

Katherine E. McDonald, Syracuse University, Public Health, Food Studies and Nutrition and the Burton Blatt Institute, 444 White Hall, Syracuse, NY, 13244, USA.

Nicole E. Conroy, Syracuse University, Public Health, Food Studies and Nutrition and Child and Family Studies, 443 White Hall, Syracuse, NY, 13244, USA.

Robert S. Olick, SUNY Upstate Medical University, Center for Bioethics and Humanities, 618 Irving Avenue, Syracuse, NY 13210, USA.

References

  1. Andre-Barron D, Strydom A, Hassiotis A. What to tell and how to tell: A qualitative study of information sharing in research for adults with intellectual disability. Journal of Medical Ethics. 2008;34(6):501–506. doi: 10.1136/jme.2006.019521. [DOI] [PubMed] [Google Scholar]
  2. Arscott K, Dagnan D, Kroese BS. Consent to psychological research by people with an intellectual disability. Journal of Applied Research in Intellectual Disabilities. 1998;11(1):77–83. doi: 10.1111/j.1468-3148.1998.tb00035.x. [DOI] [Google Scholar]
  3. Becker H, Roberts G, Morrison J, Silver J. Recruiting people with disabilities as research participants: Challenges and strategies to address them. Mental Retardation. 2004;42(6):471–475. doi: 10.1352/0047-6765(2004)42&#x0003c;471:RPWDAR&#x0003e;2.0.CO;2. [DOI] [PubMed] [Google Scholar]
  4. Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology. 2006;3(2):77–101. doi: 10.1191/1478088706qp063oa. [DOI] [Google Scholar]
  5. Brooker K, van Dooren K, Tseng CH, McPherson L, Lennox N, Ware R. Out of sight, out of mind? The inclusion and identification of people with intellectual disability in public health research. Perspect Public Health. 2014 doi: 10.1177/1757913914552583. [DOI] [PubMed] [Google Scholar]
  6. Brown H, Thompson D. The ethics of research with men who have learning disabilities and abusive sexual behaviour: A minefield in a vacuum. Disability & Society. 1997;12(5):695–708. doi: 10.1080/09687599726994. [DOI] [Google Scholar]
  7. Cameron L, Murphy J. Obtaining consent to participate in research: The issues involved in including people with a range of learning and communication disabilities. British Journal of Learning Disabilities. 2007;35(2):113–120. doi: 10.1111/j.1468-3156.2006.00404.x. [DOI] [Google Scholar]
  8. Cleaver S, Ouellette-Kuntz H, Sakar A. Participation in intellectual disability research: a review of 20 years of studies. Journal of Intellectual Disability Research. 2010;54(3):187–193. doi: 10.1111/j.1365-2788.2010.01256.x. [DOI] [PubMed] [Google Scholar]
  9. Coleman C, Menikoff J, Goldner J, Dubler N. The ethics and regulation of research with human subjects. LEXISNEXIS; 2003. [DOI] [PubMed] [Google Scholar]
  10. Dillman DA, Smyth JD, Christian LM. Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method. 3. Hoboken, NJ: John Wiley & Sons, Inc; 2009. [Google Scholar]
  11. Dresser R. Public preferences and the challenge to genetic research policy. Journal of Law and the Biosciences. 2014;1(1):52–67. doi: 10.1093/jlb/lst001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Dybwad G, Bersani H. New voices: Self-advocacy by people with disabilities. Cambridge, MA: Brookline Books; 1996. [Google Scholar]
  13. Feldman MA, Bosett J, Collet C, Burnham-Riosa P. Where are persons with intellectual disabilities in medical research? A survey of published clinical trials. Journal of Intellectual Disability Research. 2013;58(9):800–809. doi: 10.1111/jir.12091. [DOI] [PubMed] [Google Scholar]
  14. Fisher CB, Cea CD, Davidson PW, Fried AL. Capacity of persons with mental retardation to consent to participate in randomized clinical trials. American Journal of Psychiatry. 2006;(163):1813–1820. doi: 10.1176/ajp.2006.163.10.1813. [DOI] [PubMed] [Google Scholar]
  15. Freedman RI. Ethical challenges in the conduct of research involving persons with mental retardation. Mental Retardation. 2001;39(2):130–141. doi: 10.1352/0047-6765(2001)039&#x0003c;0130:ECITCO&#x0003e;2.0.CO;2. [DOI] [PubMed] [Google Scholar]
  16. Galea S, Tracy M. Participation rates in epidemiologic studies. Annals of epidemiology. 2007;17(9):643–653. doi: 10.1016/j.annepidem.2007.03.013. [DOI] [PubMed] [Google Scholar]
  17. Heller T, Pederson EL, Miller AB. Guidelines from the consumer: Improving consumer involvement in research and training for persons with mental retardation. Mental Retardation. 1996;34(3):141–148. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/8684282. [PubMed] [Google Scholar]
  18. Iacono T, Carling-Jenkins R. The human rights context for ethical requirements for involving people with intellectual disability in medical research. Journal of Intellectual Disability Research. 2012;56(11):1122–1132. doi: 10.1111/j.1365-2788.2012.01617.x. [DOI] [PubMed] [Google Scholar]
  19. Iacono T, Murray B. Issues of informed consent in conducting medical research involving people with intellectual disability. Journal of Applied Research in Intellectual Disabilities. 2003;16(1):41. [Google Scholar]
  20. Kerns J, Fine M. The relation between gender and negative attitudes toward gay men and lesbians: Do gender role attitudes mediate this relation? Sex Roles. 1994;(31):297–307. [Google Scholar]
  21. Kohn NA, Blumenthal JA. A critical assessment of supported decision-making for persons aging with intellectual disabilities. Disability and health journal. 2014;7(1):S40–S43. doi: 10.1016/j.dhjo.2013.03.005. [DOI] [PubMed] [Google Scholar]
  22. Labott SM, Johnson TP. Psychological and social risks of behavioral research. IRB: A Review of Human Subjects Research. 2004;26:11–15. [PubMed] [Google Scholar]
  23. Lai R, Elliott D, Ouellette-Kuntz H. Attitudes of research ethics committee members toward individuals with intellectual disabilities: The need for more research. Journal of Policy and Practice in Intellectual Disabilities. 2006;3(2):114–118. doi: 10.1111/j.1741-1130.2006.00062.x. [DOI] [Google Scholar]
  24. Lakin C, Turnbull A. National research goals for people with intellectual and developmental disabilities. Washington, DC: American Association on Mental Retardation; 2005. [Google Scholar]
  25. Lewin B. Who cares about disabled victims of crime? Barriers and facilitators for redress. Journal of Policy and Practice in Intellectual Disabilities. 2007;4(3):170–176. [Google Scholar]
  26. McDonald KE. “We want respect”: Adults with intellectual and developmental disabilities address respect in research. American Journal on Intellectual and Developmental Disabilities. 2012;117(4):263–274. doi: 10.1352/1944-7558-117.4.263. [DOI] [PubMed] [Google Scholar]
  27. McDonald KE, Conroy N, Olick RS, Kim C, LoBraico E, Prather E Project ETHICS Expert Panel. Is safety in the eye of the beholder? Safeguards in research with adults with intellectual disability. Journal of Empirical Research on Human Research Ethics. 2016 doi: 10.1177/1556264616651182. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. McDonald KE, Conroy N, Olick RS, Panel PEE. Is it worth it? Benefits in research with adults with intellectual disability. Intellectual and Developmental DIsabilities. 2016;54:440–453. doi: 10.1352/1934-9556-54.6.440. doi: http://dx.doi.org/10.1352/1934-9556-54.6.440. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. McDonald KE, Keys CB. How the powerful decide: Access to research participation by those at the margins. American Journal of Community Psychology. 2008;42(1):79–93. doi: 10.1007/s10464-008-9192-x. [DOI] [PubMed] [Google Scholar]
  30. McDonald KE, Keys CB, Henry DB. The gatekeepers of science: Attitudes toward the research participation of adults with intellectual disability. Am J Ment Retard. 2008;113(6):466–478. doi: 10.1352/2008.113:466-478. [DOI] [PubMed] [Google Scholar]
  31. McDonald KE, Kidney CA. What is right? Ethics in intellectual disabilities research. Journal of Policy and Practice in Intellectual Disabilities. 2012;9(1):27–39. doi: 10.1111/j.1741-1130.2011.00319.x. [DOI] [Google Scholar]
  32. McDonald KE, Kidney CA, Kimmel A, Nelms SL, Parker MR, Keys CB. Perspectives on risks and protections for the inclusion of adults with intellectual disabilties in social science research. Journal of Policy & Practice in Intellectual Disabilities. 2009;6:224–252. [Google Scholar]
  33. McDonald KE, Kidney CA, Nelms SL, Parker MR, Kimmel A, Keys CB. Including adults with intellectual disabilities in research: Scientists’ perceptions of risks and protections. J Policy Pract Intellect Disabil. 2009;6(4):244–252. [Google Scholar]
  34. McDonald KE, Kidney CA, Patka MR. ‘You need to let your voice be heard’: Research participants’ views on research. Journal of Intellectual Disability Research. 2012;57(3):216– 225. doi: 10.1111/j.1365-2788.2011.01527.x. [DOI] [PubMed] [Google Scholar]
  35. McDonald KE, Patka M. “There is no black or white”: Scientific community views on ethics in intellectual and developmental disability research. J Policy Pract Intellect Disabil. 2012;9(3):206–214. [Google Scholar]
  36. McDonald KE, Schwartz NM, Gibbons CM, Olick RS. “You can’t be cold and scientific”: Community views on ethical issues in intellectual disability research. Journal of Empirical Research on Human Research Ethics. 2015a;10(2):196–208. doi: 10.1177/1556264615575512. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. McDonald KE, Schwartz NM, Gibbons CM, Olick RS. “You Can’t be Cold and Scientific”: Community Views on Ethical Issues in Intellectual Disability Research. Journal of Empirical Research on Human Research Ethics. 2015b;10(2):196–208. doi: 10.1177/1556264615575512. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. National Institutes of Health. Research involving individuals with questionable capacity to consent: Points to consider. 2009 Retrieved from http://grants.nih.gov/grants/policy/questionablecapacity.htm.
  39. Northway R. To include or not to include? That is the ethical question. Journal of Intellectual Disabilities. 2014;18(3):209–210. doi: 10.1177/1744629514543863. [DOI] [PubMed] [Google Scholar]
  40. Perske R. The dignity of risk and the mentally retarded. Mental Retardation. 1972;10(1):24–27. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/5059995. [PubMed] [Google Scholar]
  41. Roberts LW, Hammond KA, Warner TD, Lewis R. Influence of ethical safeguards on research participation: Comparison of perspectives of people with schizophrenia and psychiatrists. American Journal of Psychiatry. 2004;161(12):2309–2311. doi: 10.1176/appi.ajp.161.12.2309. [DOI] [PubMed] [Google Scholar]
  42. Roberts LW, Warner TD, Brody JL. Perspectives of patients with schizophrenia and psychiatrists regarding ethically important aspects of research participation. American Journal of Psychiatry. 2000;157(1):67–74. doi: 10.1176/ajp.157.1.67. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/10618015. [DOI] [PubMed] [Google Scholar]
  43. Roberts LW, Warner TD, Brody JL, Roberts B, Lauriello J, Lyketsos C. Patient and psychiatrist ratings of hypothetical schizophrenia research protocols: Assessment of harm potential and factors influencing participation decisions. American Journal of Psychiatry. 2002;159(4):573–584. doi: 10.1176/appi.ajp.159.4.573. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/11925295. [DOI] [PubMed] [Google Scholar]
  44. Roberts LW, Warner TD, Hammond KG, Dunn LB. Assessments by patients with schizophrenia and psychiatrists of relative risk of research procedures. Psychiatric Services. 2006;57(11):1629–1635. doi: 10.1176/appi.ps.57.11.1629. [DOI] [PubMed] [Google Scholar]
  45. Rubright JD, Cary MS, Karlawish JH, Kim SY. Measuring how people view biomedical research: Reliability and validity analysis of the research attitudes questionnaire. Journal of Empirical Research on Human Research Ethics. 2011;6(1):63. doi: 10.1525/jer.2011.6.1.63. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Singleton R, Straits B. Approaches to social research. 3. New York, NY: Oxford University Press; 1999. [Google Scholar]
  47. Tabachnick BG, Fidell LS. Using multivariate statistics. 4. Boston, MA: Allyn & Bacon; 2001. [Google Scholar]
  48. Willis GB. Cognitive interviewing: A tool for improving questionnaire design. Sage; Thousand Oaks, CA: 2005. [Google Scholar]

RESOURCES